# SmoothSchedule - Production Deployment Manual

Complete step-by-step guide for manually deploying SmoothSchedule from scratch on a production server. This guide is useful when you need to reset the entire production deployment or troubleshoot deployment issues.

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Complete Fresh Deployment](#complete-fresh-deployment)
3. [Docker Build & Startup](#docker-build--startup)
4. [Database Initialization](#database-initialization)
5. [Static Files & Migrations](#static-files--migrations)
6. [Verification Steps](#verification-steps)
7. [Troubleshooting](#troubleshooting)

---

## Prerequisites

### Required on Production Server

```bash
# Check these are installed and running
docker --version              # Docker 20.10+
docker compose --version      # Docker Compose 2.0+
```

### Required Locally (Before Deployment)

1. All code changes committed to git (`main` branch)
2. All missing files added to git and pushed
3. Production environment variables backed up locally
4. `.envs/.production/` directory exists with proper credentials

---

## Complete Fresh Deployment

### Step 1: Save Production Environment Variables Locally

**On your local development machine:**

```bash
# Backup current production environment variables
scp poduck@smoothschedule.com:~/smoothschedule/smoothschedule/.envs/.production/.django \
    /tmp/production_django_env_backup

scp poduck@smoothschedule.com:~/smoothschedule/smoothschedule/.envs/.production/.postgres \
    /tmp/production_postgres_env_backup
```

### Step 2: Prepare & Commit All Code Changes Locally

**On your local development machine:**

```bash
cd /home/poduck/Desktop/smoothschedule2

# Check for any missing files
git status

# Add all changes and new files
git add -A

# Review staged changes
git diff --cached

# Commit changes
git commit -m "Production deployment: Add missing files and config updates"

# Push to main branch
git push origin main
```

### Step 3: Bring Down Production Containers

**SSH into production server:**

```bash
ssh poduck@smoothschedule.com

cd ~/smoothschedule/smoothschedule

# Stop all containers (preserves volumes)
docker compose -f docker-compose.production.yml down

# Or to completely reset and remove volumes (careful!):
docker compose -f docker-compose.production.yml down -v
```

### Step 4: Remove Production Codebase

**On production server:**

```bash
cd ~

# Remove everything
rm -rf smoothschedule
```

### Step 5: Pull Fresh Code from Git

**On production server:**

```bash
cd ~

# Clone the repository
git clone https://git.talova.net/poduck/smoothschedule.git smoothschedule

cd smoothschedule
```

**Note:** If git authentication fails, configure credentials on the server:

```bash
git config --global user.email "poduck@smoothschedule.com"
git config --global user.name "Poduck"

# Or store credentials
git credential approve
# Then paste: protocol=https
#            host=git.talova.net
#            username=poduck
#            password=chaff/starry
```

### Step 6: Restore Production Environment Variables

**On production server:**

```bash
cd ~/smoothschedule/smoothschedule

# Create the .envs/.production directory if it doesn't exist
mkdir -p .envs/.production

# Restore from backups on local machine or recreate them
# Option A: Use SCP to copy from local machine
scp /tmp/production_django_env_backup poduck@smoothschedule.com:~/smoothschedule/smoothschedule/.envs/.production/.django
scp /tmp/production_postgres_env_backup poduck@smoothschedule.com:~/smoothschedule/smoothschedule/.envs/.production/.postgres

# Option B: Manually recreate the files on the production server
# (see Environment Variables section below)
```

---

## Docker Build & Startup

### Step 7: Build Docker Images

**On production server:**

```bash
cd ~/smoothschedule/smoothschedule

# Build all production Docker images
docker compose -f docker-compose.production.yml build
```

**Expected output:**
```
✓ Successfully built:
  - smoothschedule_production_django
  - smoothschedule_production_celeryworker
  - smoothschedule_production_celerybeat
  - smoothschedule_production_flower
  - smoothschedule_production_postgres
  - smoothschedule_production_traefik
  - smoothschedule-awscli
```

### Step 8: Start Containers

**On production server:**

```bash
cd ~/smoothschedule/smoothschedule

# Start all containers in detached mode
docker compose -f docker-compose.production.yml up -d

# Wait for services to stabilize (10-15 seconds)
sleep 15

# Check container status
docker compose -f docker-compose.production.yml ps
```

**Expected status:** All containers should be `Up` or `Healthy`

---

## Database Initialization

### Step 9: Run Database Migrations

**On production server:**

The application uses django-tenants for multi-tenant support. Migrations must be run in this order:

```bash
cd ~/smoothschedule/smoothschedule

# 1. Migrate the public/shared schema (all shared apps)
docker compose -f docker-compose.production.yml exec -T django \
  python manage.py migrate_schemas --shared

# Expected output should show multiple migrations applied to "public" schema
```

**If you get "service django is not running":**

The Django container may not be fully started. Wait a bit longer:

```bash
sleep 30  # Wait for Django to initialize
docker compose -f docker-compose.production.yml exec -T django \
  python manage.py migrate_schemas --shared
```

---

## Static Files & Migrations

### Step 10: Collect Static Files

**On production server:**

```bash
cd ~/smoothschedule/smoothschedule

docker compose -f docker-compose.production.yml exec -T django \
  python manage.py collectstatic --noinput
```

This collects all static files (CSS, JS, images) and uploads them to:
- **Local filesystem:** `smoothschedule/staticfiles/`
- **DigitalOcean Spaces/S3:** Configured via `DJANGO_AWS_*` environment variables

### Step 11: Create Superuser (First Time Only)

**On production server, if this is a fresh installation:**

```bash
cd ~/smoothschedule/smoothschedule

docker compose -f docker-compose.production.yml exec django \
  python manage.py createsuperuser

# Follow the prompts to create the admin user
```

### Step 12: Create Initial Tenant (First Time Only)

**On production server, if you need a demo tenant:**

```bash
cd ~/smoothschedule/smoothschedule

docker compose -f docker-compose.production.yml exec django python manage.py shell
```

Then in the Django shell:

```python
from core.models import Tenant, Domain
from django.utils.text import slugify

# Create a tenant
tenant = Tenant.objects.create(
    name="Demo Company",
    slug="demo",
    is_free_trial=False,
    is_temporary=False,
)

# Create a domain for the tenant
Domain.objects.create(
    domain="demo.smoothschedule.com",
    tenant=tenant,
)

print(f"Created tenant: {tenant.name} with domain: demo.smoothschedule.com")
exit()
```

---

## Verification Steps

### Step 13: Verify All Services Are Running

```bash
cd ~/smoothschedule/smoothschedule

# Check container status
docker compose -f docker-compose.production.yml ps

# View logs for any errors
docker compose -f docker-compose.production.yml logs --tail=50

# Check specific service logs
docker compose -f docker-compose.production.yml logs django --tail=30
docker compose -f docker-compose.production.yml logs postgres --tail=30
```

### Step 14: Test API Endpoints

**From your local machine:**

```bash
# Test the backend API
curl -s "https://smoothschedule.com/api/health/" | jq

# Test tenant access
curl -s "https://demo.smoothschedule.com/api/resources/" | jq

# Test platform admin
curl -s "https://platform.smoothschedule.com/api/admin/businesses/" | jq
```

### Step 15: Verify Nginx Frontend

The frontend should be accessible at:
- **Main site:** `https://smoothschedule.com`
- **Platform dashboard:** `https://platform.smoothschedule.com`
- **Tenant subdomain:** `https://demo.smoothschedule.com`

---

## Environment Variables Reference

### Django Configuration (`.envs/.production/.django`)

```bash
# Security & Secrets
DJANGO_SECRET_KEY=your-secret-key-here
DJANGO_DEBUG=False
DJANGO_ALLOWED_HOSTS=smoothschedule.com,platform.smoothschedule.com,*.smoothschedule.com

# Admin & URLs
DJANGO_ADMIN_URL=<random-slug>/
FRONTEND_URL=https://platform.smoothschedule.com
PLATFORM_BASE_URL=https://platform.smoothschedule.com

# Celery & Redis
REDIS_URL=redis://redis:6379/0
CELERY_BROKER_URL=redis://redis:6379/0

# AWS/DigitalOcean Spaces (for media storage)
DJANGO_AWS_ACCESS_KEY_ID=your-access-key
DJANGO_AWS_SECRET_ACCESS_KEY=your-secret-key
DJANGO_AWS_STORAGE_BUCKET_NAME=smoothschedule
DJANGO_AWS_S3_ENDPOINT_URL=https://nyc3.digitaloceanspaces.com
DJANGO_AWS_S3_REGION_NAME=nyc3
DJANGO_AWS_S3_CUSTOM_DOMAIN=smoothschedule.nyc3.digitaloceanspaces.com

# Email Configuration (optional)
MAILGUN_API_KEY=your-mailgun-key
MAILGUN_DOMAIN=mg.smoothschedule.com

# SSL/Security
DJANGO_SECURE_SSL_REDIRECT=True
DJANGO_SECURE_HSTS_SECONDS=60
DJANGO_SECURE_HSTS_INCLUDE_SUBDOMAINS=True
DJANGO_SECURE_HSTS_PRELOAD=True
DJANGO_SESSION_COOKIE_SECURE=True
DJANGO_CSRF_COOKIE_SECURE=True

# Sentry (optional - error tracking)
SENTRY_DSN=
SENTRY_ENVIRONMENT=production
SENTRY_TRACES_SAMPLE_RATE=0.1
```

### PostgreSQL Configuration (`.envs/.production/.postgres`)

```bash
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=smoothschedule
POSTGRES_USER=<random-username>
POSTGRES_PASSWORD=<random-secure-password>
```

---

## Troubleshooting

### Django Container Won't Start

**Symptom:** `docker compose ps` shows Django as exited

**Solution:**

```bash
cd ~/smoothschedule/smoothschedule

# View full error logs
docker compose -f docker-compose.production.yml logs django --tail=100

# Check if database is accessible
docker compose -f docker-compose.production.yml logs postgres --tail=20

# Restart Django after fixing the issue
docker compose -f docker-compose.production.yml restart django
```

### Migration Fails with "role does not exist"

**Symptom:** Error when running `migrate_schemas --shared`:
```
FATAL: role "postgres" does not exist
```

**Solution:** The database volume is corrupted. Reset and restart:

```bash
cd ~/smoothschedule/smoothschedule

# Stop and remove all volumes
docker compose -f docker-compose.production.yml down -v

# Restart (this will recreate the database)
docker compose -f docker-compose.production.yml up -d

# Wait for database to initialize (30 seconds)
sleep 30

# Try migrations again
docker compose -f docker-compose.production.yml exec -T django \
  python manage.py migrate_schemas --shared
```

### Cannot Connect to Production Server

**Check SSH access:**

```bash
ssh poduck@smoothschedule.com "echo 'Connection successful'"
```

**If that fails:**
- Verify your SSH key is authorized on the server
- Check firewall rules allow SSH (port 22)
- Verify DNS resolves `smoothschedule.com` to the correct IP

### Traefik Certificate Issues

**Symptom:** SSL certificate errors, HTTP redirects failing

**Solution:**

```bash
cd ~/smoothschedule/smoothschedule

# Check Traefik logs
docker compose -f docker-compose.production.yml logs traefik --tail=50

# Restart Traefik to request new certificates
docker compose -f docker-compose.production.yml restart traefik

# Wait for Let's Encrypt to issue certificates (5-10 minutes)
# Check via: https://smoothschedule.com (should show valid cert)
```

### Frontend Not Loading

**Verify nginx container is running:**

```bash
# Check if nginx image was built
docker images | grep smoothschedule

# If missing, rebuild
docker compose -f docker-compose.production.yml build nginx

# Restart the service
docker compose -f docker-compose.production.yml restart

# Check logs
docker logs $(docker ps -q --filter "name=nginx")
```

---

## Quick Reference Commands

```bash
# SSH to production server
ssh poduck@smoothschedule.com

# Navigate to project
cd ~/smoothschedule/smoothschedule

# View all container logs
docker compose -f docker-compose.production.yml logs -f

# View Django logs only
docker compose -f docker-compose.production.yml logs django -f --tail=100

# Stop all containers
docker compose -f docker-compose.production.yml down

# Start all containers
docker compose -f docker-compose.production.yml up -d

# Restart a specific service
docker compose -f docker-compose.production.yml restart django

# Run a Django management command
docker compose -f docker-compose.production.yml exec -T django python manage.py <command>

# Access Django shell
docker compose -f docker-compose.production.yml exec django python manage.py shell

# Create a database backup
docker compose -f docker-compose.production.yml exec -T postgres \
  pg_dump -U $POSTGRES_USER smoothschedule > backup.sql

# Execute arbitrary SQL
docker compose -f docker-compose.production.yml exec -T postgres \
  psql -U $POSTGRES_USER smoothschedule -c "SELECT version();"

# Check Docker resource usage
docker stats
```

---

## Advanced: Rollback Procedure

If a deployment fails catastrophically:

```bash
# 1. Stop everything
cd ~/smoothschedule/smoothschedule
docker compose -f docker-compose.production.yml down

# 2. Restore from database backup (if available)
docker compose -f docker-compose.production.yml exec -T postgres \
  psql -U $POSTGRES_USER smoothschedule < backup.sql

# 3. Checkout previous git commit
cd ~/smoothschedule
git checkout <previous-commit-hash>

# 4. Rebuild and restart
cd smoothschedule
docker compose -f docker-compose.production.yml build
docker compose -f docker-compose.production.yml up -d
```

---

## Monitoring Checklist

After deployment, monitor these for 24 hours:

- [ ] No error logs in `docker compose logs django`
- [ ] API endpoints respond with 200 status
- [ ] SSL certificates are valid (https://smoothschedule.com)
- [ ] Frontend loads without console errors
- [ ] Tenant subdomains work correctly
- [ ] Static files are being served (CSS, JS load)
- [ ] Background tasks execute (check Celery worker logs)
- [ ] No database connection errors

---

## Important Notes

1. **Always backup before major changes:**
   - Database: `pg_dump`
   - Environment variables: Copy `.envs/.production/` locally
   - Code: Git commits

2. **Never modify code on production directly:**
   - Change code locally → Git push → Production git pull
   - Only edit `.env` files directly on production

3. **Multi-Tenancy Considerations:**
   - Each tenant gets a separate PostgreSQL schema
   - Migrations apply to "public" schema (shared) first
   - Tenant migrations happen automatically on first request to new tenant

4. **Docker Compose File:**
   - Always use `-f docker-compose.production.yml` for production
   - Never use `-f docker-compose.local.yml` on production

5. **Persistence:**
   - Database: `production_postgres_data` volume
   - Redis: `production_redis_data` volume
   - Traefik certs: `production_traefik` volume
   - These are preserved when using `down` (not `down -v`)

---

**Last Updated:** December 2025
**Deployment Version:** Manual v1.0