Skip to content

Processor Lambda

The processor Lambda
Processor Lambda container — runs PyTECGg calibration per SQS message; adopted when a Batch on Fargate parallel-execution quota increase was rejected. Image from `ghcr.io/platformfuzz/tec-processor-image` mirrored to ECR.
container consumes one SQS
Amazon Simple Queue Service — buffers ingest and reprocess messages between S3/API and the processor Lambda.
message per invocation from Process_Queue
SQS standard queue — receives S3 ObjectCreated notifications for new raw RINEX files; triggers processor Lambda ingest path.
and Reprocess_Queue
SQS standard queue — buffers reprocessing job messages from the Reprocess API with lower concurrency than ingest.
. Monitor its health and throughput during verification.

From the demo repo root:

Terminal window
aws lambda get-function-configuration --function-name processor \
--query '{State:State,LastUpdateStatus:LastUpdateStatus,PackageType:PackageType,Timeout:Timeout,MemorySize:MemorySize}'

Expect:

  • State: Active
  • PackageType: Image
  • Timeout: 900
  • MemorySize: 2048
Terminal window
aws sqs get-queue-attributes \
--queue-url "$(terraform -chdir=terraform output -raw queue_url)" \
--attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible
aws sqs get-queue-attributes \
--queue-url "$(terraform -chdir=terraform output -raw reprocess_queue_url)" \
--attribute-names ApproximateNumberOfMessages

Effective throughput is capped by:

  1. Per-queue concurrencyprocessor_maximum_concurrency (default 15) on ingest; reprocess_maximum_concurrency (default 2) on reprocess
  2. Account-level Lambda quota — shared across all functions in the region

If ingest messages accumulate while the processor shows throttles, request a Lambda
AWS Lambda — serverless compute for ingest-sync, query-api, reprocess-api, and the processor container image.
concurrency quota increase for your account/region.

Terminal window
aws cloudwatch describe-alarms --alarm-names \
processor-lambda-errors processor-lambda-throttles \
--query 'MetricAlarms[].{Name:AlarmName,State:StateValue}'
Terminal window
terraform -chdir=terraform output -raw processor_image_uri

Should point to your ECR
Amazon Elastic Container Registry — stores the mirrored processor Lambda container image at deploy time.
repository with the tag synced in ECR and Processor Image.

After processor runs complete, verify output in S3 Processed Output.