Pipeline overview
See Why volcano cameras? for dataset context. This demo pins to one Te Kaha
Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon. camera (TKAH.01); the pipeline below is generic.
End-to-end flow in ap-southeast-2:
- Source — GeoNet Aotearoa New Zealand Data
GeoNet — Aotearoa New Zealand's geological hazard monitoring programme. Sensors, cameras, and open data (including s3://geonet-open-data via the AWS Open Data Registry). (AWS Open Data Registry,s3://geonet-open-data) publishes Te Kaha
Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon. volcano camera JPEGs (no account required to read). - Ingest —
volcano-ingestLambda
AWS Lambda — serverless compute. This demo uses volcano-ingest (copy + annotate) and volcano-api (query + presign). lists the last 7 UTC days of keys, copies each JPEG into a private bucket, derives simple tags from the key path and pixel brightness, and writes an S3 Annotation
Amazon S3 Annotations — attach structured JSON metadata to S3 objects via named namespaces (put_object_annotation / get_object_annotation). ( environment
S3 Annotations namespace on each object in this demo (ANNOTATION_NAMESPACE). Holds the JSON metadata payload. namespace) on the same object. - Store — The private bucket is the source of truth: each JPEG carries its JSON metadata via S3 Annotations. An optional DynamoDB
Amazon DynamoDB — optional denormalized mirror of annotation fields for faster reads at larger scale; S3 remains canonical. table mirrors annotation fields for faster reads at larger scale. - Query —
volcano-apiLambda (behind API Gateway
Amazon API Gateway HTTP API — exposes GET /images with CORS; invokes the query Lambda.GET /images) reads annotations, applies filters (day_phase,utc_day,visibility), paginates withoffset+limit, and returns presigned image URLs. - Gallery — Amplify
AWS Amplify Hosting — serves the React gallery SPA that filters images using annotation-backed metadata. -hosted React SPA calls the API, shows a result count, paginates, and renders tag chips from the same annotation JSON you see in the S3 console.
Components
Section titled “Components”| Component | AWS service | Role |
|---|---|---|
| GeoNet open bucket | S3 (public) | Source JPEGs — camera/volcano/images/.../TKAH/TKAH.01/... |
| EventBridge Scheduler | EventBridge Amazon EventBridge Scheduler — optional one-time trigger to run the ingest Lambda at a chosen UTC time. | Optional one-time ingest trigger (enable_scheduler) |
| Ingest Lambda | Lambda (Python 3.14) | Copy images, put_object_annotation |
| Private bucket | S3 | Images + annotations (canonical metadata) |
| DynamoDB | DynamoDB | Optional denormalized mirror (enable_dynamodb) |
| API Lambda | Lambda (Python 3.14) | get_object_annotation, filter, presign |
| HTTP API | API Gateway v2 | GET /images with CORS |
| Gallery | Amplify Hosting | React UI — filters, count, pagination |
Data source prefix: camera/volcano/images/{YYYY}/TKAH/TKAH.01/{YYYY.DDD}/*.jpg — TKAH
Volcano site code for Te Kaha in GeoNet object key paths (TKAH/TKAH.01). This demo filters ingest to this prefix only. site code for the Te Kaha
Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon. camera (TKAH.01).
Terraform outputs
Section titled “Terraform outputs”After apply:
cd terraformterraform output amplify_app_urlterraform output api_invoke_urlterraform output private_bucket_nameOr from the demo repository root:
terraform -chdir=terraform output amplify_app_url