Building My Second CI/CD Pipeline: Lessons, Errors, and Triumphs

🚀 Project Overview I set out to build my second CI/CD pipeline from scratch using Docker, Docker Compose, and GitHub Actions. My goal was to automate the build and deployment of a custom Nginx website while learning how real-world DevOps workflows operate—including all the mistakes and fixes along the way!

🛠️ What I Built • Custom Nginx site with my own index.html and nginx.conf • Dockerfile to package my app and configs into a Docker image • docker-compose.yml to orchestrate Nginx, Prometheus, and Grafana for monitoring • GitHub Actions workflow to automate building, testing, and pushing the Docker image to Docker Hub

🐞 Errors I Made & How I Fixed Them

1. Confusion Between Dockerfile and docker-compose.yml My docker-compose.yml started empty or misconfigured (missing port dashes, wrong names). I discovered that specifying “image: nginx:latest” in Compose pulls the default image and ignores your Dockerfile. For a custom image, you must use “build: .” Lesson: Docker Compose and Dockerfile must be in sync—”build:” tells Compose to use your custom image.
2. Mounting vs. Baking Files Mounting index.html as a volume in Compose works great for local development, but only files baked into the Docker image persist when pushing to Docker Hub. Lesson: Bake configs into the image for production; mount files for local development.
3. Prometheus and Grafana Monitoring The nginx-prometheus-exporter showed no metrics (nginx_up 0) until I added a stub_status endpoint in nginx.conf and configured the exporter properly. I verified the setup using docker exec and curl http://localhost:8080/stub_status. Lesson: Monitoring needs both Nginx configuration and proper exporter setup.
4. CI/CD Workflow Issues I initially named the directory “workflow” instead of “workflows”—a simple but blocking mistake. Linter errors required fixing permissions and naming conventions. The deploy job only ran on the main branch, teaching me about proper branch management.
5. Authentication and Secrets Docker push failed from “insufficient scopes.” The fix: regenerating my Docker Hub token with correct permissions and updating GitHub secrets. Lesson: Always verify secret permissions and update them carefully.
6. Forcing CI Failures for Testing HTML linter tests weren’t triggering as expected. I learned to add explicit failure steps like “run: exit 1” to test the CI pipeline.
7. Image Caching Issues The old Nginx default page sometimes persisted after updates until I learned to use “docker pull” to fetch the latest image.

💡 What I Learned • Proper sync between Dockerfile and Compose is crucial • Production needs baked-in configs; development benefits from mounted files • Monitoring requires coordinated configuration changes • CI/CD demands attention to detail: directory names, branch triggers, and permissions • Debugging teaches through iteration • Security requires careful token and secret management

📋 My Workflow In a Nutshell

1. Write and test code locally
2. Commit to a feature branch, merge to main
3. GitHub Actions builds, tests, and pushes to Docker Hub
4. Anyone can pull and run the image without source files
5. For changes, clone the repo and rebuild locally

🏁 Final Thoughts This project revealed the practical side of DevOps and CI/CD: automating builds, managing secrets, debugging configs, and delivering production-ready applications. I’m ready to tackle more complex projects!

See the code: GitHub repo

Try the image: docker pull freddocker443/cicd_2:latest