Project-4: End-to-End DevSecOps CI/CD Pipeline Implementation

Automating Code Review, SAST, Secrets Detection, IaC Security, Container Scanning & Webhook Quality Gates with Jenkins, SonarQube, Trivy, GitLeaks & Docker

Real-Time Industry Project | Shift-Left Security & DevSecOps Pipeline

🔒 Project Overview & Shift-Left Security Philosophy

Traditional software development treated security as a final review gate right before production deployment. This delayed releases and resulted in extremely expensive post-deployment vulnerability fixes. DevSecOps (Development, Security, Operations) revolutionizes software delivery by embedding security directly into every stage of the continuous integration and continuous deployment (CI/CD) pipeline.

DevSecOps Real-Time CI/CD Project Cover
Project 4 — End-to-End DevSecOps Real-Time Pipeline Architecture Overview
Dev Code (Git)
GitLeaks / SAST
SonarQube Code Quality
Trivy / Scout Scan
Docker Hub Registry
Secure Container Deployment
DevSecOps Core Objective: Shift security left! Catch code vulnerabilities, hardcoded secrets, dependency CVEs, and container misconfigurations early in Jenkins CI/CD before code ever reaches staging or production.

🛡️ DevOps vs. DevSecOps: What's the Difference?

DevOps vs DevSecOps Difference Diagram
DevOps vs. DevSecOps vs. SecDevOps Evolutionary Comparison

Fixing Security Before Takeoff

Imagine discovering an aircraft engine defect while flying at 30,000 feet versus discovering it during routine maintenance in the hangar before takeoff! Fixing a security vulnerability in production is up to 30 times more expensive than catching it during the initial code push.

DevSecOps Pillars Development Security Operations
The 3 Core Pillars of DevSecOps (Development, Security, Operations)

👨‍💻 DEV (Development)

  • Peer Code Reviews & PR Checks
  • Git Version Control Management
  • Unit Testing & Build Automation

🔐 SEC (Security)

  • Static Application Security Testing (SAST)
  • Hardcoded Secrets & Credentials Scanning
  • Software Bill of Materials (SBOM) & CVE Audit
  • Container Image & IaC Vulnerability Scanning
DevSecOps Goals Business Value
Key Business & Engineering Goals of DevSecOps

🛠️ Complete DevSecOps Tooling Stack & Categories

To implement an effective DevSecOps pipeline, security tooling is applied across 5 distinct domains of the software lifecycle:

DevSecOps Tooling Stack Security Categories
Aqua Trivy Scanner Overview
Security Domain Primary Tool Function & Coverage
Code Security & SAST SonarQube Static Application Security Testing, code smells, bugs & quality gates.
Secrets Detection GitLeaks Detects unencrypted passwords, API tokens, and AWS keys in git repos.
IaC Security Checkov Scans Terraform, CloudFormation & K8s manifests for misconfigurations.
Container Security Aqua Trivy / Scout Audits container OS packages and application dependencies for CVEs.
Dependency Security OWASP / Snyk Identifies vulnerable 3rd party npm / maven / pip software packages.

Tool Demos & Command Examples

Aqua Trivy All in One Security Scanner
Running Trivy Image Scan on Container Image
Aqua Trivy All in One Security Scanner
Running Trivy Image Scan on Container Image
Trivy Container Scan Terminal Demo
GitLeaks Detecting Secrets & API Tokens

Hands-On Guide Step-by-Step DevSecOps Server & Pipeline Setup

Step 1: Infrastructure Prerequisites & EC2 Provisioning

Step 1 AWS EC2 Server Launch Details
Fork & Clone GitHub Source Repository (`3tierapplicationdeplyDevSecOps`)
  • Instance Name: DevOps app
  • Operating System: Ubuntu 22.04 LTS
  • Instance Type: c5.large / t3.xlarge (2 vCPUs & 8 GB RAM)
  • Storage: 30 GB GP3 EBS Volume
GitHub Repository Overview
Launching AWS EC2 Ubuntu Server for DevSecOps Tools
Terminal Command
# Clone project source repository
git clone https://github.com/CloudDevOpsHub/3tierapplicationdeplyDevSecOps.git
cd 3tierapplicationdeplyDevSecOps

Step 2: EC2 Security Group Inbound Port Rules

GitHub Repository Overview
Launching AWS EC2 Ubuntu Server for DevSecOps Tools
Protocol Port Range Purpose & Service
SSH 22 Remote Terminal Connection
SMTP 25 Mail Notifications
Custom TCP 3000 Frontend Web Container
Custom TCP 5000 3-Tier Application API Container
Custom TCP 8080 Jenkins CI/CD Automation Server
Custom TCP 9000 SonarQube Code Quality Dashboard

Step 3: Connect to EC2 Server

SSH Connection via MobaXterm Terminal
Connecting to Ubuntu Server via SSH / MobaXterm
Terminal Command
sudo su
apt update -y

Step 4: Installing OpenJDK 17 & Jenkins Server

Step 5 Installing OpenJDK 17 and Jenkins
Installing OpenJDK 17 JRE Headless and Jenkins Server
Bash Script - OpenJDK 17 & Jenkins Setup
#!/bin/bash
# Install OpenJDK 17 JRE Headless
sudo apt install openjdk-17-jre-headless -y

# Download Jenkins GPG Key
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

# Add Jenkins Debian Repository
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/" | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null

# Update & Install Jenkins
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl enable --now jenkins

Step 5: Installing Docker Engine & Configuring Permissions

Step 6 Installing Docker Engine
Executing `2nd-Docker.sh` for Docker Engine Installation
Step 7 Adding User Permissions and Python
Adding `jenkins` User to `docker` Group & Installing Python pip
Bash - Docker & User Privileges
# Add jenkins user to docker group
sudo usermod -aG docker jenkins

# Restart Jenkins service to apply group updates
sudo systemctl restart jenkins

# Verify jenkins user can execute docker commands without sudo
sudo -u jenkins docker ps

# Install Python & pip
sudo apt-get install python3-pip -y

Step 6: Installing Trivy Vulnerability Scanner

Step 8 Installing Aqua Trivy Scanner
Shell Script (`4th-Trivy.sh`) Installing Aqua Security Trivy
Bash - Trivy Setup
#!/bin/bash
sudo apt-get install wget apt-transport-https gnupg -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/publickey | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
trivy --version

Step 7: Docker Hub & Docker Scout Login

Step 9 Authenticating Docker Hub Account
Authenticating Docker Hub CLI via Personal Access Token
Step 9 Docker Scout Integration
Running Docker Scout Container Security Audits
Terminal Command
docker login -u vikas4cloud

Step 8: Deploying SonarQube Container

Step 10 Deploying SonarQube Container
Running SonarQube LTS Community Container on Port 9000
Terminal Command
# Run SonarQube LTS container
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

# Verify running container
docker ps

Access SonarQube dashboard on http://:9000. Default credentials: username admin, password admin. Set a new admin password upon first login.

⚙️ Jenkins & SonarQube Integration & Webhook Quality Gates

1. Install Required Jenkins Plugins

Step 11 Installing Jenkins Security & Docker Plugins
Installing SonarQube Scanner, Docker & Pipeline Stage View Plugins

Under Manage Jenkins ➔ Plugins ➔ Available Plugins, install:

  • SonarQube Scanner
  • Docker, Docker Commons, Docker Pipeline, Docker API, docker-build-step
  • Pipeline Stage View Plugin

2. Configure Global Tool Installations

Step 12 Configuring Global Tools in Jenkins
Configuring SonarQube Scanner & Docker Auto-Installers in Jenkins Tools

Under Manage Jenkins ➔ Tools:

  1. SonarQube Scanner: Name sonar-scanner | Install automatically (Version 7.1.0.4889).
  2. Docker: Name docker | Install automatically from docker.com (Version latest).

3. Add Docker Hub Credentials in Jenkins

Step 13 Configuring Docker Hub Credentials in Jenkins
Adding Docker Hub Credentials (Kind: Username with Password, ID: `docker`)

4. Generate SonarQube Authentication Token

Step 14 Generating SonarQube User Authentication Token
Generating User Access Token (`sonar-token`) in SonarQube Console

In SonarQube Console: Go to Administration ➔ Security ➔ Users ➔ Tokens ➔ Generate Token (Name: token). Copy the generated secret token string (e.g. squ_655a181a...).

5. Add SonarQube Secret Token to Jenkins Credentials

Step 14 Generating SonarQube User Authentication Token
Generating User Access Token (`sonar-token`) in SonarQube Console

6. Configure SonarQube Webhook for Automated Quality Gates

Step 17 Configuring SonarQube Webhook for Jenkins
Configuring Webhook in SonarQube (`http://:8080/sonarqube-webhook/`)

In SonarQube Console: Go to Administration ➔ Configuration ➔ Webhooks ➔ Create:

  • Name: jenkins
  • URL: http://:8080/sonarqube-webhook/

7. Configure SonarQube Server in Jenkins System Settings

Step 18 Configuring SonarQube Server in Jenkins System
System Configuration in Jenkins Linking SonarQube Server & Authentication Token

Under Manage Jenkins ➔ System Configuration ➔ System ➔ SonarQube servers:

  • Name: sonar
  • Server URL: http://:9000 *(Do not include trailing slash)*
  • Server Authentication Token: Select sonar-token from dropdown.

🚀 DevSecOps Pipeline Execution & Live Verification

1. Pipeline Job Creation

Step 19 Pipeline Job Creation in Jenkins
Creating Jenkins DevSecOps Pipeline Job

2. Verifying Deployed 3-Tier Web Application

Step 20 Live Web Application Verification on Port 5000
Accessing Deployed Web Application Live on Public IP Port 5000

Open your browser and navigate to http://:5000 — your secure 3-tier web application is live!

3. Connecting & Querying MySQL Container Database

Step 21 Accessing MySQL Container Database
Connecting to MySQL Database Container (`mysql_db`)
Step 22 Executing Database Queries
Executing Database Queries (`devops_exam` database)
Terminal & MySQL CLI Commands
# Connect to running MySQL container
docker exec -it mysql_db mysql -u root -p
# Enter password: rootpass

# Execute Database Queries
SHOW DATABASES;
USE devops_exam;
SHOW TABLES;
DESCRIBE results;
SELECT * FROM results ORDER BY score DESC;
SELECT AVG(score) FROM results;

💼 Resume Bullet Points & DevSecOps Engineer Responsibilities

You can add these high-impact production bullet points to your resume based on this project:

  • Architected and implemented an end-to-end Automated DevSecOps CI/CD Pipeline integrating SAST, Secrets Scanning, Container Security, and Quality Gates using Jenkins, SonarQube, Trivy, and GitLeaks.
  • Configured SonarQube Server & Quality Gates with automated webhook feedback to block builds exceeding critical vulnerability thresholds.
  • Integrated Aqua Trivy & Docker Scout container security scanners into Jenkins pipelines to audit OS packages and application dependencies before pushing images to Docker Hub.
  • Enforced automated secret detection using GitLeaks to prevent accidental exposure of API keys, AWS credentials, and database passwords in source control.
  • Configured Checkov IaC Scanning to detect misconfigurations in Dockerfiles, Terraform scripts, and Kubernetes manifests.
  • Deployed and managed containerized multi-tier web applications and MySQL databases with strict container security permissions.
  • Participated in 24/7 on-call rotation for production security incident response, vulnerability triage, and patch management.

❓ Frequently Asked Questions (FAQ) & Interview Guide

Common real-time production & interview questions on DevSecOps pipeline implementation:

Q1: What is Shift-Left Security in DevSecOps and why is it critical?

Answer: Shift-Left Security means moving security testing to the earliest stages of the software development lifecycle (during code commit and build) rather than waiting for post-deployment security audits. Detecting vulnerabilities during initial development is up to 30x cheaper and prevents security flaws from ever reaching production environments.

Q2: What is the technical difference between SAST, DAST, and SCA?

Answer: SAST (Static Application Security Testing): Analyzes source code without executing it (e.g. SonarQube).
DAST (Dynamic Application Security Testing): Tests running applications from the outside to discover runtime vulnerabilities (e.g. OWASP ZAP).
SCA (Software Composition Analysis): Identifies known CVE vulnerabilities in 3rd party libraries and dependencies (e.g. Trivy, Snyk, Dependency-Check).

Q3: How does SonarQube Quality Gate enforce pipeline failure in Jenkins?

Answer: Jenkins initiates a SonarQube analysis stage using the sonar-scanner CLI tool. SonarQube computes code metrics against defined Quality Gate rules (e.g. 0 Critical Bugs, >80% Coverage). A configured Webhook notifies Jenkins when analysis completes. If Quality Gate rules fail, the Jenkins pipeline stage aborts, preventing deployment of vulnerable code.

Q4: Why do we use Aqua Trivy alongside SonarQube in a DevSecOps pipeline?

Answer: SonarQube focuses on source code quality and static code bugs. Aqua Trivy specializes in OS container image vulnerabilities, scanning base Linux packages (Alpine/Debian/Ubuntu) and application dependencies inside Docker images for published Common Vulnerabilities and Exposures (CVEs).

Q5: How does GitLeaks prevent credential leaks in GitHub repositories?

Answer: GitLeaks scans git commits, commit history, and pull requests using regex rules to identify hardcoded passwords, API tokens, Private RSA Keys, and AWS Access Keys. Integrating GitLeaks in pre-commit hooks or CI pipelines prevents developers from committing secret credentials to source repositories.

Q6: What is Checkov and why is Infrastructure as Code (IaC) scanning necessary?

Answer: Checkov is a static code analysis tool for Infrastructure as Code (IaC). It scans Terraform, AWS CloudFormation, Helm charts, Dockerfiles, and Kubernetes manifests for security misconfigurations (e.g., publicly accessible S3 buckets, privileged container execution, missing encryption at rest).

Q7: What permissions are required to run Docker commands inside a Jenkins agent?

Answer: The jenkins service user must be added to the Linux docker group via sudo usermod -aG docker jenkins, followed by restarting the Jenkins service (sudo systemctl restart jenkins). This grants Jenkins permission to interact with the Docker daemon unix socket (/var/run/docker.sock).

Q8: How do you securely handle database passwords in production containerized apps?

Answer: Never hardcode database credentials in Dockerfiles or source code. Use Jenkins Credentials Manager, AWS Secrets Manager, HashiCorp Vault, or Kubernetes Secrets to inject credentials as encrypted environment variables or mounted secret files at runtime.

🔐
Cloud DevOps Hub Production DevSecOps CI/CD Pipeline Implementation Guide