The CI/CD Pipeline

Shipping a change to Praxis924 means one git push to main. No SSH, no manual docker compose on the box, no "did someone remember to run the migrations?" The whole path from commit to live is two GitHub Actions workflows chained in series: CI proves the code is sound, and only if it goes green does Deploy touch the server. Here is how that pipeline is built, and the traps that shaped it.

The problem

The naive deploy is a script that SSHes in, pulls main, and restarts containers. It works until it doesn't. Someone pushes a change that passes locally but breaks against a real Postgres. A model gets a new column but nobody wrote the Alembic migration, so create_all silently skips it and the app half-works. A next build type error slips through because the deploy script never compiled the frontend. Each of these ships a broken box, and now you are debugging production instead of a red check on a pull request.

The fix is to make green tests a hard precondition for deployment, and to make the deploy itself defensive — back up first, verify the database is in a known state before migrating, and health-check after.

The design

Two workflows, one gate between them.

ci.yml runs on every push and PR. It does three things that matter: it runs pytest against a real pgvector service (not a mock — vector queries and migrations behave differently on the real engine), it runs a migration-drift check that fails if a SQLAlchemy model changed without a matching migration, and it runs a type-checked next build so a tsc error is a red build, not a runtime surprise.

1# ci.yml (excerpt)
2jobs:
3  test:
4    services:
5      db:
6        image: pgvector/pgvector:pg16
7        env: { POSTGRES_PASSWORD: postgres }
8        ports: ["5432:5432"]
9    steps:
10      - uses: actions/checkout@v4
11      - run: uv sync
12      - run: uv run pytest -q
13      - run: bash scripts/check_migration_drift.sh
14  build-frontend:
15    steps:
16      - run: npm ci && npm run build   # type-checked; tsc errors fail here

deploy.yml does not run on push. It runs on a workflow_run trigger, gated on CI having succeeded on main:

1# deploy.yml (excerpt)
2on:
3  workflow_run:
4    workflows: ["CI"]
5    types: [completed]
6    branches: [main]
7
8jobs:
9  deploy:
10    if: github.event.workflow_run.conclusion == 'success'
11    runs-on: ubuntu-latest
12    steps:
13      - name: Deploy over SSH
14        run: |
15          ssh $DEPLOY_USER@$DEPLOY_HOST <<'EOF'
16            cd "$DEPLOY_PATH"
17            bash scripts/db-backup.sh              # 1. snapshot the DB first
18            git reset --hard origin/main           # 2. exact tree, no drift
19            docker compose up -d --build           # 3. rebuild + restart
20            docker compose exec -T backend \
21              python -m app.scripts.preflight_migration_check   # 4. safe to migrate?
22            docker compose exec -T backend alembic upgrade head # 5. apply migrations
23            curl -fsS http://localhost:8000/health              # 6. verify
24          EOF

The ordering is the whole point. Back up before anything mutates. git reset --hard (not git pull) so the server tree is byte-identical to origin/main — hand-edits on the box get erased, which is intended. The preflight check runs before alembic upgrade head: it aborts if the production database is unstamped or its stamp doesn't match its real schema, so you fail loudly with a clear message instead of crashing mid-migration with a DuplicateColumn. Then the health-check confirms the container actually came up.

The gotchas

These are the traps that earned their place in the pipeline:

Build it yourself

The minimal version is two files. In ci.yml, add a services: block with the same database image you run in prod, then run your test suite and your frontend build. In deploy.yml, use on.workflow_run with if: github.event.workflow_run.conclusion == 'success' so deploy can only fire after CI is green on main. The deploy job holds four secrets — DEPLOY_HOST, DEPLOY_USER, DEPLOY_KEY, DEPLOY_PATH — and its body is the SSH heredoc above: backup, reset, build, preflight, migrate, health-check. That is a complete, safe pipeline.

What changes at scale

Today the box runs the same Docker Compose stack as dev, which keeps the mental model tiny. As traffic grows you would add a staging environment that deploys first and runs smoke tests before promotion; blue-green or rolling restarts so compose up --build doesn't drop requests; and a rollback step that redeploys the previous image tag if the health-check fails instead of leaving a half-broken box. You would also move infra into IaC — right now the EC2 instance and its data EBS volume are provisioned by hand in the AWS console, which is fine for one box and dangerous for ten. None of that changes the core invariant: nothing ships unless tests and build are green, and the database is snapshotted before it is touched.