Adding in multi-user support was no easy feat! I’m used to the agile development of multi-sprint practices in other MVC frameworks where you can add and change any module or feature anytime. That simply did not work due to Django Framework anomalies with the user setup. The Django Auth User Model can only be set (or changed) on the initial database build (not later). As in, once you’ve done a Django migration, the auth user model setting cannot be changed:
AUTH_USER_MODEL = 'job_search.CustomUser'This complicated things as I already have over 1,000 job postings, over 200 opportunities, and over 40 job sites!
Having had experience with SQLyog, I felt confident in doing a simple export of data, rebuilding the SQLite database, then, importing the data, mostly just using SQLiteStudio. Unfortunately, due to the contents of those JSON fields and a few other default settings and null = true… this took easily 10 draining, circular hours trying to force everything in to just… work.
Then ChatGPT and I stumbled upon the consensus that SQLite has some finicky behaviors with JSON constraints and settings. This made adding new user fields and fixing the JSON fields downright impossible due to … all of it. I was stuck.
Eventually, I’ll be migrating out of SQLite and into the more robust AWS WordPress MySQL Instance database. I did not want to muddy the “production” feeling of the WordPress MySQL database just yet… so it was time to migrate the data over to MariaDB which would be much better behaved.
This still was no easy task: there were additional hacks from the normal methods to get this to work. Also, MariaDB is far more stringent than SQLite with data fitting into fields. This was a good time to review the field widths with real-world data. The final models.py file eventually evolved into the following:
# models.py
from django.db import models
from datetime import datetime
from django.contrib.auth.models import AbstractUser, Group, Permission
class CustomUser(AbstractUser):
bio = models.TextField()
user_greeting = models.CharField(max_length=24)
color_mode = models.CharField(max_length=24)
dashboard_first_date = models.DateTimeField(blank=True, null=True)
dashboard_second_date = models.DateTimeField(blank=True, null=True)
class EmailOpportunity(models.Model):
user = models.ForeignKey(
CustomUser, on_delete=models.CASCADE,
null=True, # Temporary. To be removed after migration
# default=1, # Set the default user ID
)
job_title = models.CharField(max_length=128)
opportunity_status = models.CharField(max_length=48)
recruiter_name = models.CharField(max_length=64)
recruiter_company = models.CharField(max_length=64)
email_received_at = models.DateTimeField(default=datetime.now, blank=True)
employment_type = models.CharField(max_length=24, default='')
job_duration = models.CharField(max_length=64, default='')
location_type = models.CharField(max_length=32, default='')
location_city = models.CharField(max_length=64, default='')
comments = models.JSONField(default=list, blank=True, null=True)
job_description = models.TextField(default='', blank=True, null=True)
def _str_(self):
return self.job_title
class JobSite(models.Model):
user = models.ForeignKey(
CustomUser, on_delete=models.CASCADE,
null=True, # Temporary. To be removed after migration
# default=1, # Set the default user ID
)
site_name = models.CharField(max_length=64, default='')
site_url = models.CharField(max_length=2048, default='')
site_password = models.CharField(max_length=64, default='', blank=True, null=True)
rating = models.IntegerField(default=1)
resume_format = models.CharField(max_length=32, default='')
github_field = models.BooleanField(default=False)
project_site_field = models.BooleanField(default=False)
last_visited_at = models.DateTimeField(default=datetime.now, blank=True)
resume_updated_at = models.DateTimeField(default=datetime.now, blank=True)
headline = models.CharField(max_length=512, default='')
description = models.TextField(blank=True, null=True)
def _str_(self):
return self.site_name
class JobPosting(models.Model):
user = models.ForeignKey(
CustomUser, on_delete=models.CASCADE,
null=True, # Temporary. To be removed after migration
# default=1, # Set the default user ID
)
job_site_id = models.ForeignKey(JobSite, on_delete=models.CASCADE)
company_name = models.CharField(max_length=128, default='')
posting_title = models.CharField(max_length=128, default='')
posting_status = models.CharField(max_length=32, default='')
posting_url_full = models.CharField(max_length=2048, default='')
posting_url_domain = models.CharField(max_length=32, default='')
posting_password = models.CharField(max_length=32, default='', blank=True, null=True)
pay_range = models.CharField(max_length=256, default='')
location_type = models.CharField(max_length=32, default='')
location_city = models.CharField(max_length=64, default='')
employment_type = models.CharField(max_length=32, default='')
applied_at = models.DateTimeField(default=datetime.now)
interviewed_at = models.DateTimeField(blank=True, null=True)
rejected_at = models.DateTimeField(blank=True, null=True)
rejected_after_stage = models.CharField(max_length=32, default='')
job_scan_info = models.CharField(max_length=64, default='', blank=True, null=True)
outreach_info = models.CharField(max_length=64, default='', blank=True, null=True)
time_spent = models.IntegerField(default='', blank=True, null=True)
technology_string = models.CharField(max_length=512, default='', blank=True)
technology_stack = models.JSONField(default=list, blank=True, null=True)
comments = models.JSONField(default=list, blank=True, null=True)
posting_application_questions = models.JSONField(default=list, blank=True, null=True)
job_description = models.TextField()
def _str_(self):
return self.posting_title
I was ready for data export with the models.py file correctly aligned with the data. Unfortunately, the standard export would not provide the correct data encoding for MariaDB (we needed to migrate to a stringent UTF-8 MB4 collation) and included too much of the database (auth.permission and contenttypes tables would not correctly import). To solve this, we created a script to do the export:
#export_data.py
import os
from django.core.management import call_command
from io import StringIO
import django
# Set the settings module explicitly
os.environ['DJANGO_SETTINGS_MODULE'] = 'django_react_job_search.settings' # Replace with your actual project name
# Initialize Django
django.setup()
def export_data():
# Capture the dumpdata output with excluded tables
output = StringIO()
call_command('dumpdata', exclude=['auth.permission', 'contenttypes'], stdout=output)
# Write the output to a file with UTF-8 encoding
with open('db_export_cleaned.json', 'w', encoding='utf-8') as f:
f.write(output.getvalue())
if __name__ == "__main__":
export_data()
With this version of the script set up, and the models.py max lengths, default values, and nullable settings, these commands finally ran smoothly:
python export_data.pyFinally, with the data correctly exported, I changed the Database settings in settings.py to the local MariaDB server.
# DATABASES = {
# 'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': BASE_DIR / 'db.sqlite3',
# }
# }
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql', # Use MySQL engine for MariaDB compatibility
'NAME': 'django_job_search', # Replace with your actual database name
'USER': 'django_app', # Replace with your MariaDB username
'PASSWORD': 'django_app', # Replace with your MariaDB password
'HOST': 'localhost', # Or use your MariaDB server address (e.g., IP or domain)
'PORT': '3306', # Default MariaDB/MySQL port
'OPTIONS': {
'charset': 'utf8mb4', # Set charset to utf8mb4 for better Unicode support
'collation': 'utf8mb4_unicode_ci', # Ensure the collation is utf8mb4-based
},
}
}With Django pointing to the new MariaDB schema, I ran the imports using Django:
python manage.py flush
python manage.py makemigrations
python manage.py migrate
python manage.py loaddata db_export_cleaned.jsonChecking out the application behaving in the new MariaDB finally working I was finally able to relax! A few celebratory drinks may have been consumed. 🙂