by: Brian BorninoPosted on:

Job App Tracker AWS Promotion

Getting my application finally running on AWS was its own hurdle. AWS gives us so many choices, that we can easily suffer paralysis by analysis. That happened to me a few times. Over the last few days, I pushed hard to bring up my app on AWS and was finally able to get it running, so that anybody could poke around! Here are the AWS elements which I went through:

  • Docker Revisit
  • AWS Elastic Container Registry (ECR) and Amazon Command Line Interface (CLI)
  • AWS LightSail Container Service
  • AWS EC2
  • AWS Route 53

Docker Revisit

I thought I had the whole docker concept down. What I actually understood was the docker file and compose. I have used some Linux workstations, so I can navigate around Linux and use the basic utils (like vi…). Unfortunately, through the final push to production, when you finally figure out what your docker structure will look like and start testing more fully, you realize the pieces that you are missing, and you learn more about the fundamental pieces that you are bringing in.

  • One would think that with an image name like “slim” (python:3.12-slim) that it would be the slim[mest], or smallest, right? Nope: the smallest image tag is the Alpine version. That made a huge impact on my image size!
  • Coming from the world of Apache, (think LAMP: Linux, Apache, MySQL, PHP…) everything “mostly just works”. But you don’t get that kind of out-of-the-box support for Python or React… So I learned more about the modern and fully featured Web App Server: Nginx. And its relationship with the WSGI server: Gunicorn for Python support and a RESTful API.
  • Ports: are necessary, as they help separate Nginx and Gunicorn (which isn’t just a drag-and-drop library module). The docker environment must map a port for Nginx and a separate port for Gunicorn. Like many software engineers out there, I use a Windows-based laptop (which means using the Windows-based Docker Desktop and its Docker Composer). Last month, I thought I had Docker Composer figured out: nope. Turns out, that Docker Desktop Compose has some quirks about mapping ports for Nginx. So after all that wonderful learning about Docker Compose, I had to go back to doing the non-compose command line to build and run containers. Further complicating things: AWS LightSail was not happy with the below file, either (see the AWS Light Sail section for additional details)…

By the time I was all done, I ended up with the below docker file.

# PRODUCTION Dockerfile

# First Stage - Build the React App
FROM node:20 AS react_build
WORKDIR /app/frontend

# Copy package.json and package-lock.json separately for better caching
COPY frontend/package.json frontend/package-lock.json ./
RUN npm install
COPY frontend ./
RUN npm run build

# Stage 2: Setup Django backend
FROM python:3.12-alpine AS final
WORKDIR /app/backend

# Set Python to not buffer stdout/stderr (for logging in Docker)
ENV PYTHONUNBUFFERED=1

# Install dependencies for MySQL client and other necessary build tools
COPY backend/. .
RUN apk update && \
    apk add --no-cache build-base mariadb-dev pkgconfig nginx \
    && pip install --upgrade pip \
    && pip install --no-cache-dir -r requirements.txt \
    # Remove unnecessary packages to reduce image size
    && apk del build-base \
    # Clean up APK cache
    && rm -rf /var/cache/apk/*

# Ensure the logs directory exists in the container
RUN mkdir -p /backend/logs

# Copy compiled front end from previous build stage
COPY --from=react_build /app/frontend/build/ /app/backend/staticfiles/

# Copy Nginx configuration for production (Ensure you have this file available)
COPY /frontend/default.conf /etc/nginx/http.d/default.conf

# Expose ports
EXPOSE 80 8000

# Start Nginx and Gunicorn in the background
CMD ["sh", "-c", "gunicorn --bind 0.0.0.0:8000 django_react_job_search.wsgi:application & nginx -g 'daemon off;'"]

Oh, and the default.conf file that I sort of brushed off? Turns out, that is the primary config file for the web server. It had some fun pieces to build up that I’ll later finalize:

# Nginx is a high-performance web server and reverse proxy server used to serve static files 
# and forward requests to backend services (like Gunicorn for Django apps) efficiently.

# Nginx Configuration

# The 'server' block defines a virtual server for Nginx. This block listens for incoming HTTP requests
# and determines how to handle them based on the configuration inside it.
server {
    # Listen for incoming requests on port 80 on all network interfaces
    listen 0.0.0.0:80;  # Ensures Nginx listens on all interfaces on port 80 (HTTP).
    
    # Define the server name
    server_name localhost;  # The domain name of the server, typically replaced with your real domain.

    # Handle requests to the root URL ('/')
    location / {
        # Serve static files (HTML, CSS, JS, etc.) from the /app/backend/staticfiles directory
        root /app/backend/staticfiles;  # Static files like HTML are served from this directory.
        
        # If the requested file doesn't exist, try serving the 'index.html' file (SPA fallback)
        try_files $uri /index.html;  # If a file is not found, serve the index.html (single-page app behavior).
    }

    # Handle requests to the '/api/' path, typically for API requests
    location /api/ {
        # Forward the incoming request to the backend running on localhost:8000 (typically Gunicorn)
        proxy_pass http://localhost:8000;  # Proxy requests to the backend server (like Gunicorn running Django).
        
        # Set headers to forward the real client information and protocol to the backend server
        proxy_set_header Host $host;  # Preserve the original 'Host' header.
        proxy_set_header X-Real-IP $remote_addr;  # Forward the real IP address of the client.
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;  # Forward the 'X-Forwarded-For' header (proxy chain).
        proxy_set_header X-Forwarded-Proto $scheme;  # Forward the protocol used (http or https).
    }

    # This routes traffic for /admin/ to Gunicorn (Django Admin)
    location /admin/ {
        proxy_pass http://localhost:8000;  # Gunicorn on port 8000
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
    
    location /backend/logs/ {
        alias /app/backend/logs;  # Map the URL path to the container's logs directory
        autoindex on;  # Enable directory listing (optional, so you can see files listed)
        try_files $uri $uri/ =404;  # Return 404 if the file doesn't exist
        access_log off;  # Optionally, turn off logging for this location to avoid extra logs
    }

    location /frontend/logs/ { 
        alias /var/log/nginx;  # Adjust this path if necessary
        autoindex on;  # Enable directory listing so you can see files in the logs folder
        try_files $uri $uri/ =404;  # Return 404 if the file doesn't exist
        access_log off;  # Optionally turn off access logging for this location
    }
}

With the Nginx config file done, I needed the final piece figured out, which I had accidentally not done correctly last month. Gunicorn is a Web Server Gateway Interface (WSGI) that needs to know the entry point for our Django application. The CMD line of our dockerfile needed to be adjusted to point to the below file:

import os
from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'django_react_job_search.settings')
application = get_wsgi_application()

AWS Elastic Container Registry (ECR) and Amazon Command Line Interface (CLI)

Prior to this week, I had some exposure to AWS training, IAM, Route 53, and LightSail Instances. I had earlier figured out some advanced usage of my lone LightSail WordPress instance to also use its MariaDB for the Job App Tracker database. But that only scratches the surface of the power and cost-effectiveness of AWS: Containers. The only way to get containers onto an AWS LightSail Container Service is by using ECR.

Configuring ECR was at first daunting with lots of new terms and what should I select but… turned out to be pretty easy. For wherever you are going to place a container, there is a one-to-one relationship with the repository. Kind of. For those of us with projects or just one instance, it’s a one-to-one. But when you start doing multiple deployments, it is that one image that gets sent everywhere. The only thing I really needed to figure out: the repository name. Since this is really at the docker instance stage, I named my repository job-app-tracker-image. The rest I kept at their defaults (Mutable Tags and AES-256 encryption).

But how do I upload (and download on a Linux machine) these images? That is where the Amazon CLI came in. For my Windows Laptop, I had to download the AWS CLI installer for Windows. After the quick install, I needed to configure it. On the EC2, I used Amazon Linux, which already had the CLI installed, but I still needed to configure that. That configuration used my individual account user information from AWS Identity and Access Management (IAM). From IAM, I got my AWS Access Key ID and AWS Secret Access Key. This is done using the CLI Command:

aws configure

That command is run just once on that machine. Ever.

The next command is to authenticate the user (using the input from the above configure statement). I run the authentication command once, at the beginning of every day (I leave my command prompt open all day long). This will require using your ECR instance name:

aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin [your container info here].amazonaws.com

With those out of the way, uploading my docker images to ECR was straightforward:

docker build --no-cache -f dockerfile.prod -t job-search-prod-image:staging .
docker tag job-search-prod-image:staging [your container info here].amazonaws.com/job-app-tracker-image:staging
docker push [your container info here].amazonaws.com/job-app-tracker-backend-image:staging

AWS LightSail Container Service

LightSail is misleadingly simple. The draw of having a LightSail container or instance that sits on top of a layer that Amazon maintains is very alluding. So, I tried out the Container Service in hopes that I could be up and running in an hour. A full day later of struggling to get the container service running (I had to sleep on it to come up with the debug strategy and correct Google question), I learned some key ‘features”:

  • The LightSail Container Service only allows ONE port to be opened. That includes internal ports, which were apparently blocked. Remember how Nginx communicates via port to Gunicorn? I was never able to get any Gunicorn traffic because the LightSail Container Service, despite my properly configuring the ports in the docker file, blocked the Gunicorn port.
  • The LightSail Container Service is perfect for a simple container with just one major service/daemon. So, I could have had just Nginx or just Gnicorn on the container, but not both.

AWS EC2

The LightSail Container Service, for the underlying VM hardware, is more expensive than just the EC2, where you have to manage more of the environment. Though I was initially scared away by this concept, I learned through working with the LightSail Container Service that I needed access to those controls to get my containers up and running. Plus, I could bypass ECS!

I set up the Amazon CLI and git, then built the docker container. Of course, it did not build correctly the first time. This was my first opportunity to work remotely through the docker command line to figure out what was missing: I had to make some slight changes to the Nginx configuration file and I stumbled upon a file that didn’t get picked up by the repository. No big deal, right? About the third or fourth docker build, the entire EC2 froze mid-build! Turns out, one of the ways that they make the t2.nano and t2.micro cheaper is “CPU credits”, for those rare times that you need extra CPU cycles. The t2.micro has plenty of horsepower for running a simple full-stack Web Site Application, but not building.

So that’s why you still use ECS!

With these changes, the app “worked”. Rather, the front end was perfect. The back end still required a database. Oh yeah, we can’t use the SSH tunnel that my laptop was using to connect to the AWS LightSail WordPress MariaDB Instance. This was an easy fix. On the AWS LightSail WordPress Instance, there is an IPv4 Firewall. I simply opened up the MariaDB port to just the EC2 instance IP address.

AWS Route 53

With these changes, my app was finally working on the web. Unfortunately, it had a rather ugly URL. But, that is what AWS Route 53 is for, right? I added my A NAME entry into my Route 53 registry and… sorry, I just could not wait: I had to try it… pretty much right away. Maybe, from the time that entered the additional route 53 entry and when I entered the URL in the browser, it was a… full minute? It pulled up right away.

A Final High Note!

Relief at last. It is like you can finally breathe. There is that sense of accomplishment of “yeah, I knew I could do it”. Needless to say, there was some celebrating. Here is a link to the final app (note, I have not added the HTTPS certificates yet):

http://job-app-tracker.bornino.net/

In retrospect, I wouldn’t give up any of my learning on this project at all. Yes, the next full-stack project I do will come together much faster. Anything of similar scope (or bigger) would likely take me… 40 to 80 hours. I have all my leverage points and documentation ready to go.

Of course, using the Agile Sprint Methodology… this was technically sprint 5. However, I have a few more sprints planned in the not-too-distant future enabling more AWS features. And each one of them I can now easily roll them out live, right away.