Skip to content

Pipeline overview

See Why volcano cameras? for dataset context. This demo pins to one Te Kaha
Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon.
camera (TKAH.01); the pipeline below is generic.

S3 Annotations demo architecture — GeoNet ingest, private S3 with annotations, API, and Amplify gallery

End-to-end flow in ap-southeast-2:

  1. Source GeoNet Aotearoa New Zealand Data
    GeoNet — Aotearoa New Zealand's geological hazard monitoring programme. Sensors, cameras, and open data (including s3://geonet-open-data via the AWS Open Data Registry).
    (AWS Open Data Registry, s3://geonet-open-data) publishes Te Kaha
    Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon.
    volcano camera JPEGs (no account required to read).
  2. Ingestvolcano-ingest Lambda
    AWS Lambda — serverless compute. This demo uses volcano-ingest (copy + annotate) and volcano-api (query + presign).
    lists the last 7 UTC days of keys, copies each JPEG into a private bucket, derives simple tags from the key path and pixel brightness, and writes an S3 Annotation
    Amazon S3 Annotations — attach structured JSON metadata to S3 objects via named namespaces (put_object_annotation / get_object_annotation).
    ( environment
    S3 Annotations namespace on each object in this demo (ANNOTATION_NAMESPACE). Holds the JSON metadata payload.
    namespace) on the same object.
  3. Store — The private bucket is the source of truth: each JPEG carries its JSON metadata via S3 Annotations. An optional DynamoDB
    Amazon DynamoDB — optional denormalized mirror of annotation fields for faster reads at larger scale; S3 remains canonical.
    table mirrors annotation fields for faster reads at larger scale.
  4. Queryvolcano-api Lambda (behind API Gateway
    Amazon API Gateway HTTP API — exposes GET /images with CORS; invokes the query Lambda.
    GET /images) reads annotations, applies filters (day_phase, utc_day, visibility), paginates with offset + limit, and returns presigned image URLs.
  5. Gallery Amplify
    AWS Amplify Hosting — serves the React gallery SPA that filters images using annotation-backed metadata.
    -hosted React SPA calls the API, shows a result count, paginates, and renders tag chips from the same annotation JSON you see in the S3 console.
ComponentAWS serviceRole
GeoNet open bucketS3 (public)Source JPEGs — camera/volcano/images/.../TKAH/TKAH.01/...
EventBridge Scheduler EventBridge
Amazon EventBridge Scheduler — optional one-time trigger to run the ingest Lambda at a chosen UTC time.
Optional one-time ingest trigger (enable_scheduler)
Ingest LambdaLambda (Python 3.14)Copy images, put_object_annotation
Private bucketS3Images + annotations (canonical metadata)
DynamoDBDynamoDBOptional denormalized mirror (enable_dynamodb)
API LambdaLambda (Python 3.14)get_object_annotation, filter, presign
HTTP APIAPI Gateway v2GET /images with CORS
GalleryAmplify HostingReact UI — filters, count, pagination

Data source prefix: camera/volcano/images/{YYYY}/TKAH/TKAH.01/{YYYY.DDD}/*.jpg TKAH
Volcano site code for Te Kaha in GeoNet object key paths (TKAH/TKAH.01). This demo filters ingest to this prefix only.
site code for the Te Kaha
Te Kaha (TKAH.01) — the single GeoNet volcano camera this demo ingests. West-facing site on the Bay of Plenty coast, Aotearoa New Zealand; captures a JPEG every 10 minutes with Whakaari/White Island on the horizon.
camera (TKAH.01).

After apply:

Terminal window
cd terraform
terraform output amplify_app_url
terraform output api_invoke_url
terraform output private_bucket_name

Or from the demo repository root:

Terminal window
terraform -chdir=terraform output amplify_app_url