Building and Deploying a Fargate Container that runs Python and performs CloudWatch Logging

This guide walks you through building and deploying an AWS Fargate container that runs a Python script. The container will be tagged as the latest version, pushed to Amazon Elastic Container Registry (ECR), and configured to log messages to CloudWatch Logs. The process includes setting up the Dockerfile, creating the Python script, building and pushing the Docker image, and configuring AWS services to ensure your container runs smoothly on Fargate.

By the end of this tutorial, you will have a running Fargate service that logs information to CloudWatch Logs, allowing you to monitor the status and output of your containerized application in real time.

NOTE - In this guide, when you see aws_account_id replace it with your AWS account number and region with your AWS region. 

Step-by-Step Guide Overview

Setting up the Dockerfile

Define a Dockerfile that uses Amazon Linux 2023 as a base image, installs Python, sets up the environment, and configures the CloudWatch logging agent.

Writing the Python Script for CloudWatch Logging

Create a Python script that logs a message every 5 seconds to CloudWatch, ensuring you can monitor the container’s health and status. 

Building the Docker Image

Build the Docker image using the Dockerfile, tagging it as latest to make it easy to track and manage. 

Tagging the Docker Image

Tag the Docker image with the latest version and prepare it for pushing to ECR by using the appropriate repository URL.

Pushing the Image to Amazon ECR 

Push the Docker image to Amazon Elastic Container Registry (ECR) so it can be pulled by Fargate when running the container. 

Creating a CloudWatch Log Group 

Set up a CloudWatch Log Group where logs from the Fargate container will be streamed for monitoring and troubleshooting. 

Creating an ECS Cluster 

Set up an ECS cluster in Amazon ECS to serve as the environment where the Fargate task will run. 

Creating a Fargate Task Definition 

Define the task configuration for Fargate, including resource requirements, the container image, and the CloudWatch logging configuration. 

Running the Fargate Service 

Deploy the Fargate task as a service within your ECS cluster to run the container continuously. 

Monitoring Logs in CloudWatch 

Use Amazon CloudWatch to monitor the logs generated by your Fargate container, ensuring that the Python script is running as expected.

Step 1: Setting up the Dockerfile

The Dockerfile defines how the Fargate container image is built and configured. This file will use Amazon Linux 2023 as the base image, install Python 3, the AWS CLI, Boto3 for AWS SDK access, and configure CloudWatch for logging.

Dockerfile:

FROM amazonlinux:2023

# Update and install required packages, checking if curl is already installed

RUN yum update -y && \

    # Check if curl is installed, and only install if not found

    if ! command -v curl &> /dev/null; then \

        echo "curl not found, installing curl..."; \

        yum install -y curl; \

    else \

        echo "curl is already installed, skipping..."; \

    fi && \

    # Install Python3, unzip, and CloudWatch agent

    yum install -y python3 unzip && \

    # Install pip using the get-pip.py script

    curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \

    python3 get-pip.py && \

    # Clean up the get-pip script

    rm -f get-pip.py && \

    # Upgrade pip to the latest version

    pip3 install --upgrade pip && \

    # Install AWS CLI and boto3 for logging to CloudWatch

    pip3 install awscli boto3 && \

    # Install CloudWatch Agent

    yum install -y amazon-cloudwatch-agent

# Set working directory

WORKDIR /app

# Copy the Python script into the container

COPY app.py .

# Run the Python script when the container starts

CMD ["python3", "app.py"]

Step 2: Writing the Python Script for CloudWatch Logging

Now, write a Python script that logs a message to CloudWatch every 5 seconds. The script uses Python's built-in logging module to log messages with the level INFO.

Python script (app.py):

import logging

import time

# Configure logging

logging.basicConfig(level=logging.INFO)

logger = logging.getLogger("FargateLogger")

def main():

    while True:

        # Log the message every 5 seconds

        logger.info("Fargate Container is Running Successfully!")

        time.sleep(5)

if __name__ == "__main__":

    main()

Step 3: Build the Docker Image

Next, build the Docker image from the Dockerfile. Ensure that you're in the directory containing both the Dockerfile and app.py. This command will build the Docker image and tag it as my-fargate-app.

docker build -t my-fargate-app .

Step 4: Tag the Docker Image

After building the image, tag it with the appropriate ECR repository URL. This makes it easier to push the image to Amazon ECR.

docker tag my-fargate-app:latest <aws_account_id>.dkr.ecr.<region>.amazonaws.com/my-fargate-app:latest

Step 5: Push the Docker Image to Amazon ECR

Create an ECR Repository:

• Go to the Amazon ECR console and create a new repository (e.g., my-fargate-app).

Authenticate Docker to ECR

• Authenticate Docker with the ECR registry using AWS CLI.

aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<region>.amazonaws.com

Push the Docker Image

 Push the image to ECR

docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/my-fargate-app:latest

Step 6: Create a CloudWatch Log Group

Create a CloudWatch Log Group where the logs from your Fargate container will be stored. This step ensures that all log data generated by the container is centralized for easy monitoring.

1. Go to the CloudWatch Console.
2. Click Logs, then Create log group.
3. Name the log group (e.g., FargateContainerLogs), then click Create.

Step 7: Create an ECS Cluster

1. Go to the Amazon ECS Console.
2. Click Create Cluster and choose Fargate.
3. Name the cluster (e.g., MyFargateCluster), then click Create.

Step 8: Create a Fargate Task Definition

A task definition defines how the container should run in ECS.

1. In the ECS Console, click Task Definitions, then Create new Task Definition.
2. Choose Fargate as the launch type.
3. Name the task definition (e.g., MyFargateTask).

In the Container Definitions section:

1. Click Add container.
2. Name the container (e.g., my-fargate-container).
3. Enter the image URL: <aws_account_id>.dkr.ecr.<region>.amazonaws.com/my-fargate-app:latest.
4. Set the memory and CPU requirements (e.g., 0.5 GB memory and 0.25 vCPU).
5. For Log Configuration, select awslogs and enter:

• Log group: FargateContainerLogs.
• Log stream prefix: my-fargate-stream.

Click Add, then Create to save the task definition.

Step 9: Run the Fargate Service

1. In the ECS Console, go to your cluster (e.g., MyFargateCluster).
2. Click Create in the Services tab.
3. Choose Fargate as the launch type.
4. Select the task definition you created earlier (e.g., MyFargateTask).
5. Set the desired number of tasks (e.g., 1), configure networking settings, and click Create Service.

Step 10: Monitor Logs in CloudWatch

Once the service is running, you can view the logs in CloudWatch.

1. Go to the CloudWatch Console.
2. Click Logs and select the log group you created (FargateContainerLogs).
3. View the log streams, which should show messages like:
   

These logs are generated every 5 seconds by the Python script.

Summary

In this guide, you've:

1. Built a Docker container that runs a Python script logging messages every 5 seconds.
2. Tagged and pushed the image to Amazon ECR for deployment.
3. Set up CloudWatch to capture the logs from the container.
4. Deployed the container to ECS Fargate and monitored the logs through CloudWatch.

By following these steps, you'll have a fully functional Fargate service that continuously logs information to CloudWatch, giving you valuable insights into the health and operation of your application.