## DevOps Pipeline Framework
### Phase 1: Pipeline Architecture
```
Design a CI/CD pipeline for:
**Application:** [Monolith, microservices, serverless]
**Stack:** [Languages, frameworks, databases]
**Deployment Target:** [AWS, GCP, Azure, on-prem, hybrid]
**Team Size:** [Number of developers]
Pipeline requirements:
1. **Build Stage**
- Source control integration
- Dependency management
- Compilation/bundling
- Artifact creation
2. **Test Stage**
- Unit tests
- Integration tests
- E2E tests
- Security scans
- Performance tests
3. **Deploy Stage**
- Environment promotion (dev → staging → prod)
- Deployment strategy (rolling, blue-green, canary)
- Rollback capability
- Feature flags
4. **Quality Gates**
- Code coverage thresholds
- Security vulnerability limits
- Performance benchmarks
- Approval workflows
5. **Observability**
- Pipeline metrics
- Deployment tracking
- Error alerting
- Audit logging
Generate pipeline architecture diagram and workflow.
```
### Phase 2: CI/CD Implementation
#### GitHub Actions
```yaml
name: CI/CD Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Lint
run: npm run lint
- name: Type check
run: npm run type-check
- name: Unit tests
run: npm run test:unit -- --coverage
- name: Upload coverage
uses: codecov/codecov-action@v3
integration-test:
runs-on: ubuntu-latest
needs: test
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: test
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run test:integration
env:
DATABASE_URL: postgresql://postgres:test@localhost:5432/test
security:
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
severity: 'CRITICAL,HIGH'
exit-code: '1'
- name: SAST scan
uses: github/codeql-action/analyze@v2
build:
runs-on: ubuntu-latest
needs: [test, security]
if: github.event_name == 'push'
permissions:
contents: read
packages: write
outputs:
image-tag: ${{ steps.meta.outputs.tags }}
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha,prefix=
type=ref,event=branch
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy-staging:
runs-on: ubuntu-latest
needs: build
if: github.ref == 'refs/heads/develop'
environment: staging
steps:
- uses: actions/checkout@v4
- name: Deploy to Kubernetes
uses: azure/k8s-deploy@v4
with:
namespace: staging
manifests: |
k8s/deployment.yaml
k8s/service.yaml
images: |
${{ needs.build.outputs.image-tag }}
deploy-production:
runs-on: ubuntu-latest
needs: build
if: github.ref == 'refs/heads/main'
environment: production
steps:
- uses: actions/checkout@v4
- name: Deploy with canary
run: |
kubectl apply -f k8s/canary.yaml
sleep 300 # Monitor for 5 minutes
ERROR_RATE=$(curl -s "http://prometheus/api/v1/query?query=error_rate" | jq .data.result[0].value[1])
if (( $(echo "$ERROR_RATE > 0.01" | bc -l) )); then
kubectl rollout undo deployment/app
exit 1
fi
kubectl apply -f k8s/deployment.yaml
```
### Phase 3: Infrastructure as Code
```
Implement infrastructure as code with:
**Tool:** [Terraform, Pulumi, CloudFormation]
**Cloud:** [AWS, GCP, Azure]
**Resources:** [List infrastructure needed]
Terraform example structure:
1. **Module Organization**
```
infrastructure/
├── modules/
│ ├── networking/
│ │ ├── main.tf
│ │ ├── variables.tf
│ │ └── outputs.tf
│ ├── kubernetes/
│ ├── database/
│ └── monitoring/
├── environments/
│ ├── dev/
│ │ ├── main.tf
│ │ └── terraform.tfvars
│ ├── staging/
│ └── production/
└── global/
└── backend.tf
````
2. **State Management**
```hcl
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "env/${var.environment}/terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "terraform-locks"
}
}
````
3. **Environment Promotion**
```hcl
module "app" {
source = "../../modules/app"
environment = var.environment
instance_count = var.environment == "production" ? 3 : 1
instance_type = var.environment == "production" ? "t3.large" : "t3.small"
}
```
4. **Secret Management**
- Use AWS Secrets Manager / HashiCorp Vault
- Reference secrets in Terraform
- Rotate secrets automatically
- Never commit secrets to code
Generate Terraform modules for the infrastructure.
```
### Phase 4: Deployment Strategies
```
Implement deployment strategy for:
**Application:** [Description]
**Availability Requirements:** [SLA, downtime tolerance]
**Risk Tolerance:** [Conservative, moderate, aggressive]
Deployment strategies:
1. **Blue-Green Deployment**
```yaml
Blue (Current): app-v1 (receiving traffic)
Green (New): app-v2 (ready, no traffic)
Switch: Update LB to point to Green
Rollback: Switch LB back to Blue
```
2. **Canary Deployment**
```yaml
Phase 1: 5% to v2, 95% to v1
Monitor: 15 minutes
Phase 2: 25% to v2, 75% to v1
Monitor: 30 minutes
Phase 3: 50% to v2, 50% to v1
Monitor: 1 hour
Phase 4: 100% to v2
Rollback trigger:
- Error rate > 1%
- Latency p99 > 500ms
- Manual override
```
3. **Rolling Deployment**
```yaml
spec:
replicas: 4
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1 # One extra pod during update
maxUnavailable: 0 # Always maintain capacity
```
4. **Feature Flags**
```javascript
// Decouple deployment from release
if (featureFlags.isEnabled('new-checkout', user)) {
return <NewCheckout />;
} else {
return <OldCheckout />;
}
```
Generate deployment configuration for chosen strategy.
```
## Kubernetes Patterns
```
Kubernetes deployment patterns:
1. **Deployment Configuration**
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: app
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
spec:
containers:
- name: app
image: myapp:v1
resources:
requests:
cpu: '100m'
memory: '128Mi'
limits:
cpu: '500m'
memory: '512Mi'
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
```
2. **Horizontal Pod Autoscaler**
```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: app-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: app
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
```
3. **Pod Disruption Budget**
```yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: app-pdb
spec:
minAvailable: 2 # At least 2 pods always running
selector:
matchLabels:
app: myapp
```
```
## Best Practices
1. **Everything as Code**: Pipelines, infrastructure, configuration
2. **Immutable Artifacts**: Build once, deploy to all environments
3. **Fast Feedback**: Fail fast, parallelize tests
4. **Security Shift-Left**: Scan early and often
5. **Observability Built-In**: Logging, metrics, tracing from day one
6. **Rollback Ready**: Always have a quick rollback path
7. **Secrets Management**: Never hardcode, always encrypt
8. **Documentation**: Runbooks for operations
## Example Session
**User**: "Set up a CI/CD pipeline for my Node.js API with PostgreSQL."
**Response**:
> I'll create a complete pipeline with GitHub Actions:
>
> **Pipeline Overview:**
> 1. **Test** → Lint, type-check, unit tests
> 2. **Integration Test** → With PostgreSQL container
> 3. **Security** → Trivy scan, CodeQL
> 4. **Build** → Docker image to GHCR
> 5. **Deploy** → Kubernetes with rolling update
>
> Key files I'll generate:
> - `.github/workflows/ci-cd.yml` - Main pipeline
> - `Dockerfile` - Multi-stage build
> - `k8s/` - Kubernetes manifests
> - `.env.example` - Environment template
>
> Want me to generate these files?
## Related Resources
- [GitHub Actions Documentation](https://docs.github.com/en/actions)
- [Kubernetes Documentation](https://kubernetes.io/docs/)
- [Terraform Registry](https://registry.terraform.io/)
- [The DevOps Handbook](https://itrevolution.com/product/the-devops-handbook/)
```