Skip to content
Event-Driven Serverless Walkthrough

Staged Apply

A staged apply ensures processing resources ( SQS
Amazon Simple Queue Service — buffers ingest and reprocess messages between S3/API and the processor Lambda. , processor Lambda
Processor Lambda container — runs PyTECGg calibration per SQS message; adopted when a Batch on Fargate parallel-execution quota increase was rejected. Image from `ghcr.io/platformfuzz/tec-processor-image` mirrored to ECR. , DynamoDB
Amazon DynamoDB — Jobs table tracks reprocessing job status by `job_id` (queued → processing → completed/failed). ) are ready before the EventBridge Scheduler
AWS EventBridge Scheduler — triggers ingest-sync Lambda on a recurring UTC schedule (default: hourly). starts automated ingest. This avoids raw files landing in S3
Amazon Simple Storage Service — data lake bucket for raw ingest and processed TEC output; S3 events trigger processing. before the processor can consume them.

Prerequisites

Section titled “Prerequisites”

Stage 1 — Processing layer

Section titled “Stage 1 — Processing layer”

Create ingest bucket, SQS pipeline, DynamoDB Jobs table, and processor Lambda:

Terminal window
terraform -chdir=terraform apply -var="region=ap-southeast-2" -target=module.processing

This also creates the ingest module resources the processing layer depends on (S3 bucket, notifications).

Stage 2 — Enable ingest scheduler

Section titled “Stage 2 — Enable ingest scheduler”

When processing is healthy, enable scheduled ingest:

Terminal window
terraform -chdir=terraform apply -var="region=ap-southeast-2" -target=module.ingest_scheduler

Stage 3 — Full reconciliation

Section titled “Stage 3 — Full reconciliation”

Apply the remaining resources (presentation, observability, and any drift):

Terminal window
terraform -chdir=terraform apply -var="region=ap-southeast-2"

Type yes to confirm. Terraform creates resources in dependency order:

  1. Ingest layer (S3, ingest-sync Lambda)
  2. Processing layer (SQS, DynamoDB, ECR, processor Lambda)
  3. Presentation layer (API Lambdas, API Gateway, Amplify app)
  4. Portal build + manual Amplify deploy (when deploy_amplify_on_apply=true)

Skip Amplify on apply

Section titled “Skip Amplify on apply”

If Node.js is unavailable in your CI environment:

Terminal window
terraform -chdir=terraform apply -var='deploy_amplify_on_apply=false' -var="region=ap-southeast-2"

Deploy the portal manually — see Amplify Portal.

Single-shot alternative

Section titled “Single-shot alternative”

If you prefer one apply (after ECR sync):

Terminal window
terraform -chdir=terraform apply -var="region=ap-southeast-2"

Staged apply is recommended for first-time deploys to control when automated ingest begins.

Verify apply outputs

Section titled “Verify apply outputs”
Terminal window
terraform -chdir=terraform output
terraform -chdir=terraform output -raw api_url
terraform -chdir=terraform output -raw bucket_name
terraform -chdir=terraform output -raw processor_image_uri

See Terraform Outputs for all output values.

Next steps

Section titled “Next steps”