api
Express 5 behind API Gateway. The same app runs in docker compose locally.
// CASE STUDY — STUDY PROJECT
SERVERLESS
> E-commerce API on AWS Lambda
An e-commerce API on AWS Lambda with infrastructure as code, CI/CD on GitHub Actions, full observability and Machine Learning product recommendations. The goal: every decision has a written reason, and every guarantee has a test that proves it.
- Node.js 24
- Express 5
- AWS Lambda
- API Gateway
- RDS PostgreSQL
- ElastiCache (Valkey)
- SQS
- EventBridge
- S3
- SageMaker
- Terraform
- GitHub Actions
- OpenTelemetry
- X-Ray
- Docker Compose
- Python
// OVERVIEW
More than a Lambda hello world.
A study project built to practise how a serverless backend is really made: transactions, idempotency, async events, least-privilege IAM and a real deploy pipeline.
What the API covers
- sign-up and login with JWT
- paginated, cached catalogue
- transactional, idempotent orders
- image upload straight to S3
- asynchronous event processing
- personalised product recommendations
Runs without an AWS account
The whole stack runs locally with docker compose, and the same application runs on Lambda without changing a line.
// ARCHITECTURE
4 Lambdas, 4 different triggers.
A private VPC across 2 availability zones. RDS and ElastiCache have no public access, and every connection uses TLS.
client ─HTTPS→ API Gateway ─→ λ api (Express 5)
├─→ RDS Proxy ─→ PostgreSQL
├─→ ElastiCache (Valkey)
├┄→ S3, private presigned POST
└┄→ SageMaker serverless InvokeEndpoint
EventBridge ─→ λ dispatcher ─→ SQS ─→ λ worker
└┄→ DLQ
λ ops no HTTP route: migrations and admin tasksdispatcher
Scheduled by EventBridge. Publishes pending outbox events to SQS.
worker
Consumes SQS with partial batch failure, so one bad message goes to the DLQ without blocking the batch.
ops
No HTTP route at all: migrations, training data export and admin tasks.
10 Terraform modules
Three environments (dev, staging, prod), each with its own state.
RDS Proxy
Each Lambda execution environment opens its own connections and can exhaust the database in a traffic spike. The proxy keeps the pool and multiplexes them.
// PROBLEMS SOLVED
Each guarantee has a test behind it.
The failures a real store hits under concurrency, retries and outages, and how the API handles each one.
No negative stock, even under concurrency
Stock is debited with UPDATE … WHERE stock >= quantity inside the order transaction, taking locks in a fixed order to avoid deadlocks. A database constraint is the last line of defence.
test: 4 concurrent buyers fight over 1 unit, and exactly one gets 201.
No duplicate order from a client retry
The Idempotency-Key header is mandatory. The same key with the same body returns the same order with no new stock debit. The same key with a different body returns 409.
test: 3 concurrent retries with the same key become a single order.
No event lost between the database and the queue
Transactional outbox: the event is written in the order's transaction, and a dispatcher publishes it with FOR UPDATE SKIP LOCKED, so two dispatchers never publish the same event. The worker deduplicates by event id.
test: With SQS down, the event waits in the outbox and is published when the queue comes back.
A cache that can't take the app down
Cache-aside on Redis treats every error as a miss, with short timeouts. If Redis goes down, the catalogue keeps answering straight from the database.
Safe uploads
Presigned POST instead of PUT, because only the POST policy can cap the file size. The server generates the object key inside the user's own prefix, so no client can write into another user's space.
Database connections from Lambda
RDS Proxy holds the connection pool so a burst of cold Lambda environments doesn't exhaust PostgreSQL.
// MACHINE LEARNING
Recommendations from what similar customers bought.
GET /api/v1/recommendations suggests products from item-item collaborative filtering, served by SageMaker Serverless Inference.
PostgreSQL ─→ anonymised export ─→ S3 ─→ Python training ─→ model.tar.gz
│
λ api ─InvokeEndpoint, 800 ms→ SageMaker Serverless ←─────────────┘
└─ timeout · no history · model down ─→ best sellersThe right model for the problem
Item-item collaborative filtering with cosine similarity, trained from scratch on the purchase history. Fine-tuning was ruled out on purpose: it needs a pre-trained model, a GPU and data volume a small store doesn't have.
Pay per invocation
SageMaker Serverless Inference with the official scikit-learn container. No instance running 24 hours a day.
An optional dependency
With an 800 ms timeout, no customer history or the model down, the API answers with best sellers. The X-Recommendation-Source header says where the list came from.
Consistent with the store
The model only returns ids. Price and stock always come from the database, so an out-of-stock product is never recommended.
Private and reproducible
The training export turns the customer id into an HMAC with a fresh random salt each time. The same data produces a byte-identical artifact, and the model version is the hash of its content.
Tested end to end
One test exports from PostgreSQL, trains, starts the inference server and checks that the API switches from “best sellers” to “model”.
// INFRASTRUCTURE AS CODE
Terraform with no secrets in the state.
10 modules, 3 environments, and a test suite that runs without an AWS account.
Secrets out of the state
RDS generates and keeps its own password (manage_master_user_password). The JWT key is an ephemeral value written through a write-only argument, so it never appears in the Terraform state.
Least-privilege IAM
One role per function, each reaching only what that function uses. api can only invoke the recommendation endpoint; ops can only write under training/.
Configuration guards
A check block refuses a production plan without Multi-AZ.
Native state locking
Locking happens natively in S3, with no DynamoDB table.
terraform test with a mocked provider
Validates all three environments across 8 scenarios, with no AWS account and no cost.
Generated docs
Each module has a README produced by terraform-docs and checked in CI.
// CI/CD
No AWS keys in GitHub.
GitHub Actions authenticates with OIDC, and production needs a human to approve.
pull request ─┬─→ tests · lint · build · compose + smoke · terraform
└─→ terraform plan × 3 environments (read-only role)
push main ───→ deploy dev ─→ deploy prod (manual approval)OIDC instead of keys
The production role can only be assumed by a job in the prod environment, which requires manual approval.
Pull requests never apply
The plan role is read-only.
Deploy from the saved plan
Serialised per environment, followed by migrations and a smoke test.
Full local stack in CI
docker compose comes up and the smoke test goes from sign-up to order, through the trained recommendation.
// OBSERVABILITY
Follow one order from HTTP to the worker.
Structured logs, end-to-end correlation, tracing and alarms on what actually breaks.
Structured JSON logs
Every line carries the correlation ID and trace ID. Password, token and email are redacted automatically.
End-to-end correlation ID
It crosses the HTTP request, the outbox, the SQS message and the worker, so one saved CloudWatch Logs Insights query shows an order's whole path.
OpenTelemetry and X-Ray
Locally, an OpenTelemetry collector runs as a sidecar. On Lambda, where there are no side-by-side containers, the equivalent is X-Ray through a Lambda Extension.
Dashboard and alarms
A CloudWatch dashboard and alarms for 5xx, p95 latency, errors per function, any message in the DLQ and messages ageing in the queue.
// QUALITY
Tested against the real thing.
Row locks, constraints, transactions and SKIP LOCKED don't exist in a mock, so the tests use a real database and cache on purpose.
- backend tests against real PostgreSQL and Redis
- 69backend tests against real PostgreSQL and Redis
- line coverage (~83% branches, CI floor of 80%)
- ~95%line coverage (~83% branches, CI floor of 80%)
- ML tests: training, inference contract and HTTP server
- 9ML tests: training, inference contract and HTTP server
- terraform test scenarios
- 8terraform test scenarios
| Layer | Coverage |
|---|---|
| Backend (Jest + Supertest) | 69 tests against real PostgreSQL and Redis |
| Machine Learning (unittest) | 9 tests: training, inference contract and HTTP server |
| Infrastructure | terraform fmt, validate and test (8 scenarios) |
| Artifact | The Lambda zip is unpacked the way AWS does it and invoked with a real API Gateway event |
| Integration | Smoke test against the full Docker Compose stack |
// HONEST STATUS
What is proven, and what isn't yet.
The pipeline is ready. A deploy to a real AWS account has not run yet.
Verified
- tests, lint, build, Terraform validate/test and the Docker stack, all in CI
- recommendations end to end, locally and in CI
Pipeline ready, not run yet
- deploy to a real AWS account: apply, SageMaker, X-Ray
Estimated cost
About US$ 0.12/hour for the dev environment. It's an estimate, not a bill. Fixed costs dominate: NAT, RDS Proxy, RDS and ElastiCache, not the Lambdas. That's why the project runs 100% locally and goes up to AWS only to test, torn down the same day.
What I learned
- The gap between “written to the database” and “published to the queue” is real, and the outbox closes it.
- Idempotency has to be designed in from the start, not added later.
- Serverless has hidden fixed costs: NAT, Proxy and the database bill while idle, and the Lambdas barely register.
- Pick the ML model that fits the problem. Simple, explainable collaborative filtering beats a big model when data is scarce.
- Optional dependencies (cache, ML model) need a timeout and a fallback, so their failure never becomes the user's failure.
- Infrastructure security: OIDC instead of keys, secrets out of the state and minimal IAM per function.
- Writing down why each decision was made is worth as much as the code.