Cloud Computing and Deployment - AWS, Azure, GCP Complete Mastery

You manage: Application, Data, Runtime, Middleware, OS
Provider manages: Virtualization, Servers, Storage, Networking

Control: Maximum control, most complex
Cost: Pay per resource
Examples: AWS EC2, Azure VMs, Google Compute Engine
Use: Custom OS, specific software requirements

You manage: Application, Data
Provider manages: Runtime, Middleware, OS, Virtualization, Servers

Control: Medium control, less complexity
Cost: Pay per app/instance
Examples: AWS Elastic Beanstalk, Heroku, Google App Engine
Use: Web apps, APIs, rapid development

You manage: Just your data
Provider manages: Everything else

Control: Minimum control, maximum simplicity
Cost: Subscription model
Examples: Salesforce, Office 365, Slack
Use: Out-of-the-box solutions

                IaaS         PaaS         SaaS
Flexibility     High         Medium       Low
Complexity      High         Medium       Low
Cost Control    High         Medium       Low
Scalability     Manual       Automatic    Built-in
Maintenance     You          Shared       Provider

"Virtual machines in the cloud"
- Choose instance type (t2.micro, m5.large, c5.2xlarge)
- Choose OS (Amazon Linux, Windows, Ubuntu)
- Configure storage (EBS volumes)
- Assign security groups (firewalls)
- Launch instances → pay per hour

Pricing Model:
- On-Demand: Pay per hour ($0.05/hour for t2.micro)
- Reserved: Commit 1-3 years ($0.01/hour for same instance)
- Spot: Unused capacity (70% discount, but interruptible)

Use: Custom apps, full OS control, complex deployments
Problems: Requires server management, manual scaling

"Run code without managing servers"

Process:
1. Write function (Node.js, Python, Java, etc.)
2. Deploy to Lambda
3. Set trigger (API call, S3 upload, scheduled)
4. Function runs automatically
5. Pay only for execution time ($0.0000002 per request)

No servers to manage!
Automatic scaling!
Extremely cost-effective!

Example: Image resizing on S3 upload
1. User uploads image to S3
2. S3 triggers Lambda
3. Lambda resizes image
4. Lambda uploads to CDN

Limitations:
- 15 minute max execution time
- Stateless (no local persistence)
- Cold starts (first request slower)

ECS (Elastic Container Service):
- Manage Docker containers
- Auto-scaling
- Load balancing
- Simpler than Kubernetes

EKS (Elastic Kubernetes Service):
- Kubernetes in the cloud
- More complex, more powerful
- Industry standard
- Multi-cloud capable

Workflow:
1. Write application
2. Create Dockerfile
3. Build Docker image
4. Push to ECR (Elastic Container Registry)
5. Define ECS/EKS task
6. Deploy to cluster
7. Auto-scaling handles load

"Infinitely scalable storage"

Characteristics:
- Stores any file type
- No size limits
- 11+ nines durability
- Data centers worldwide
- Pricing: $0.023 per GB/month

Use Cases:
- Website assets (images, CSS, JS)
- Backups
- Data lakes
- CDN origin

Access Patterns:
- Public URLs: https://bucket.s3.amazonaws.com/file.jpg
- Private access via credentials
- Signed URLs (temporary access)

Versioning:
- Keep history of object changes
- Rollback to previous version
- Disaster recovery

"Managed SQL databases"

Supported Databases:
- PostgreSQL
- MySQL
- MariaDB
- Oracle
- SQL Server
- Aurora (AWS-optimized, 5x faster MySQL)

Managed Benefits:
- Automatic backups
- Read replicas (replication)
- Multi-AZ deployment (availability)
- Automatic failover
- Parameter groups (tuning)

Pricing:
- On-demand instances
- Storage per GB
- Data transfer
- Backups

Example: Multi-AZ RDS PostgreSQL
- Primary in us-east-1a
- Standby in us-east-1b
- Automatic replication
- Auto-failover on primary failure

"Serverless NoSQL database"

Key Features:
- Fully managed
- Auto-scaling
- Single-digit millisecond latency
- Encryption at rest

Pricing Model:
- On-demand: Pay per request
- Provisioned: Reserve capacity (cheaper for predictable load)

Schema:
- Partition key (main lookup)
- Sort key (optional, range queries)
- Attributes (flexible, add any field)

Example:
{
  "UserId": "user123",        // Partition key
  "CreatedAt": 1234567890,   // Sort key
  "Email": "user@mail.com",
  "Profile": { ... }         // Nested attributes
}

Query Patterns:
- Get by partition key: O(1)
- Query range: Efficient
- Scan all: Expensive, avoid

Use: User profiles, real-time data, mobile apps

"Your private network in the cloud"

Components:
┌─ VPC (10.0.0.0/16) ─────────────────┐
│ ┌─ Public Subnet ─┐ ┌─ Private ─┐   │
│ │ • NAT Gateway   │ │ Subnet    │   │
│ │ • EC2 instances │ │ • RDS     │   │
│ │ • ALB           │ │ • Cache  │   │
│ └─────────────────┘ └───────────┘   │
└─────────────────────────────────────┘

Internet Gateway: Route to internet (0.0.0.0/0)
Route Table: Defines traffic routes
Network ACLs: Subnet-level firewall (stateless)
Security Groups: Instance-level firewall (stateful)

Example Architecture:
Internet ↔ IGW ↔ ALB (public subnet) 
        ↔ EC2s (public subnet)
        ↔ RDS (private subnet, no internet access)

"Distribute traffic across instances"

Features:
- Layer 7 (application) load balancing
- Path-based routing (/api → backend, / → frontend)
- Host-based routing (api.com → backend, web.com → frontend)
- SSL termination (handles HTTPS)
- Auto-scaling integration

Workflow:
1. User sends request to ALB DNS
2. ALB routes to target group based on rules
3. Load balanced across instances
4. Unhealthy instances removed automatically

Health Checks:
- HTTP requests to /health endpoint
- Instances responding 200 → healthy
- Instances not responding → removed from rotation

"Redis/Memcached in the cloud"

Use Cases:
- Session storage
- API response caching
- Rate limiting
- Real-time leaderboards
- Job queues

Performance:
- Microsecond latency
- Reduces database load
- Massive performance improvement

Example: Caching user profile
1. User requests profile
2. Check ElastiCache → found → return
3. Cache miss → query RDS → update cache → return
4. Next request → cache hit, instant response

# Dockerfile - Define application environment
FROM node:18-alpine

WORKDIR /app

# Copy application
COPY package*.json ./
RUN npm install

# Copy code
COPY . .

# Expose port
EXPOSE 3000

# Run application
CMD ["node", "server.js"]

# Build image
docker build -t my-app:1.0 .

# Run container
docker run -p 3000:3000 my-app:1.0

# Push to registry
docker push registry.example.com/my-app:1.0

# Kubernetes deployment manifest
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3  # Run 3 instances
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: registry.example.com/my-app:1.0
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 3000
  type: LoadBalancer

Developer workflow:
1. Write code locally
2. Commit to GitHub
3. Webhook triggers build
4. Run automated tests
5. Build Docker image
6. Push to registry
7. Update status on PR

On Failure:
- Notify developer
- Block merge to main

# .gitlab-ci.yml
stages:
  - test
  - build
  - deploy

variables:
  DOCKER_DRIVER: overlay2
  REGISTRY: registry.gitlab.com

test:
  stage: test
  image: node:18
  script:
    - npm install
    - npm run lint
    - npm test
  only:
    - merge_requests

build:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker build -t $REGISTRY/my-app:$CI_COMMIT_SHA .
    - docker push $REGISTRY/my-app:$CI_COMMIT_SHA
  only:
    - main

deploy:
  stage: deploy
  image: alpine:latest
  script:
    - kubectl set image deployment/my-app 
      app=$REGISTRY/my-app:$CI_COMMIT_SHA
  only:
    - main

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]

jobs:
  test-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Setup Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '18'
      
      - name: Install dependencies
        run: npm install
      
      - name: Run tests
        run: npm test
      
      - name: Build Docker image
        run: docker build -t my-app:${{ github.sha }} .
      
      - name: Push to registry
        run: docker push registry.example.com/my-app:${{ github.sha }}
      
      - name: Deploy to production
        run: kubectl set image deployment/my-app app=registry.example.com/my-app:${{ github.sha }}

┌────────────────────────────┐
│        AWS Region          │
│                            │
│  ┌────────────────────┐    │
│  │  CloudFront (CDN)  │    │
│  └─────────┬──────────┘    │
│            │               │
│  ┌─────────▼──────────┐    │
│  │ ALB (us-east-1a)  │    │
│  └────┬───────┬──────┘    │
│       │       │           │
│  ┌────▼─┐ ┌──▼────┐       │
│  │EC2-1 │ │EC2-2  │       │
│  └────┬─┘ └──┬────┘       │
│       │      │            │
│  ┌────▼──────▼──┐         │
│  │ RDS Multi-AZ │         │
│  └──────────────┘         │
│                            │
│  ┌──────────────┐          │
│  │  S3 Bucket   │          │
│  └──────────────┘          │
└────────────────────────────┘

Costs: Low
Latency: Varies by location
Availability: Regional

┌──────────────────────────┐    ┌──────────────────────────┐
│   US Region (N.Virginia) │    │ EU Region (Frankfurt)    │
│ ┌──────────────────────┐ │    │ ┌──────────────────────┐ │
│ │ CloudFront + Route53 │ │    │ │ CloudFront + Route53 │ │
│ └──────────┬───────────┘ │    │ └──────────┬──────────┘ │
│            │             │    │            │            │
│ ┌──────────▼──────────┐  │    │ ┌──────────▼─────────┐  │
│ │ ALB + Auto Scaling  │  │    │ │ ALB + Auto Scaling │  │
│ └──────────┬──────────┘  │    │ └──────────┬────────┘   │
│            │             │    │            │            │
│ ┌──────────▼──────────┐  │    │ ┌──────────▼─────────┐  │
│ │   DynamoDB Global   │◄──────►│   DynamoDB Global   │  │
│ │   Table (Read/Write)│  │    │   Table (Read/Write)│  │
│ └─────────────────────┘  │    │ └──────────────────┘   │
└──────────────────────────┘    └──────────────────────────┘

Benefits:
- Lowest latency (pick closest region)
- Disaster recovery (one region down, other continues)
- Compliance (data localization)

Costs: High (redundancy)
Data Sync: Critical complexity

┌─────────────────────────────────────┐
│         API Gateway (Route53)       │
└──────────────┬──────────────────────┘
               │
    ┌──────────┼──────────┐
    │          │          │
┌───▼──┐  ┌───▼──┐  ┌───▼──┐
│Users │  │Orders│  │ Inv  │
│Service   │Service   │Service
└────┬─┘  └───┬──┘  └───┬──┘
     │        │         │
┌────▼──┐ ┌──▼───┐ ┌──▼───┐
│Users  │ │Orders│ │Produc│
│DB     │ │DB    │ │tDB   │
└───────┘ └──────┘ └──────┘

│         │         │
└─────────┴─────────┘
          │
    ┌─────▼──────┐
    │ Message    │
    │ Queue      │
    │ (async)    │
    └────────────┘

Benefits:
- Independent scaling
- Technology diversity
- Faster deployments
- Fault isolation

Challenges:
- Distributed tracing
- Complex debugging
- Network latency
- Consistency issues

// Using AWS CloudWatch
const cloudwatch = new AWS.CloudWatch();

// Log custom metrics
cloudwatch.putMetricData({
    Namespace: 'MyApp',
    MetricData: [{
        MetricName: 'UserSignups',
        Value: 42,
        Unit: 'Count'
    }]
}).promise();

// Create alarms
cloudwatch.putMetricAlarm({
    AlarmName: 'HighCPU',
    MetricName: 'CPUUtilization',
    Threshold: 80,
    ComparisonOperator: 'GreaterThanThreshold',
    AlarmActions: ['arn:aws:sns:...']
}).promise();

Application logs → CloudWatch Logs
                ↓
            Aggregation
                ↓
    Dashboard, Alerts, Analytics

Key metrics to track:
- Request latency (p50, p95, p99)
- Error rates
- Throughput (requests/sec)
- Resource utilization
- Business metrics (conversions, signups)

1. Right-sizing Instances
   - Monitor actual CPU/memory usage
   - Downsize over-provisioned instances
   - Can save 30-50%

2. Reserved Instances
   - Commit to 1 or 3 years
   - Up to 70% discount vs On-Demand

3. Spot Instances
   - Use unused capacity
   - Up to 90% discount
   - For interruptible workloads

4. Data Transfer Optimization
   - Use CloudFront CDN
   - Compress responses
   - Consolidate data centers

5. Database Optimization
   - Close unused databases
   - Right-size database instances
   - Archive old data

Example: $50,000/month to $15,000/month
- Move to Spot: -40%
- Right-size: -20%
- Reserved instances: -30%
- Archive data: -10%

Principle of Least Privilege:
- Users have MINIMUM permissions needed
- No blanket "admin" access

Example Policy:
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::my-bucket/*"
  }]
}

This allows reading S3 objects only

Security Groups (Stateful):
- Inbound: Port 443 (HTTPS) from 0.0.0.0/0
- Outbound: All ports/protocols

Network ACLs (Stateless):
- Public subnet: Allow 80, 443 inbound
- Private subnet: Allow only from public subnet