Amazon S3
Send2Bucket writes to your bucket by assuming an IAM role in your account. The role trusts Send2Bucket's AWS principal only when the request carries the destination's external ID, which stops another Send2Bucket customer from pointing a stream at your role.
What you provide
| Setting | Example |
|---|---|
bucket | analytics-raw |
region | us-east-1 |
prefix | raw |
role_arn | arn:aws:iam::123456789012:role/s2bucket-writer |
You also need two values from Send2Bucket. The dashboard shows both, with the trust policy below filled in:
- Send2Bucket's AWS principal, shown in the dashboard on the destination;
- the external ID, generated for each destination.
1. Create the role
Create an IAM role with this trust policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"AWS": "<aws_principal>"},
"Action": "sts:AssumeRole",
"Condition": {"StringEquals": {"sts:ExternalId": "<external_id>"}}
}
]
}
2. Allow writes under the prefix
Attach this permissions policy to the role. s3:PutObject writes delivery objects. s3:GetObject covers reading back the setup test object and the HEAD request Send2Bucket makes to check an object after a lost upload response.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject"],
"Resource": "arn:aws:s3:::analytics-raw/raw/s2b/*"
}
]
}
If the bucket uses SSE-KMS with a customer managed key, also allow kms:GenerateDataKey and kms:Decrypt on that key.
3. Add the destination
In the dashboard, enter the four settings and select Verify. Send2Bucket assumes the role, writes a test object under raw/s2b/ and reads it back. The destination becomes active and the stream accepts data once that check passes.
Objects
Objects land at raw/s2b/stream=<stream_id>/date=YYYY-MM-DD/hour=HH/<delivery_batch_id>.ndjson.gz. See Delivery format. Lifecycle rules, replication and retention on the bucket are yours to set.
Query your data
- DuckDB: reads the objects in place.
- Trino: external table over the bucket; Athena uses the same table definition.
- BigQuery: through BigQuery Omni.
- PostgreSQL: load into
jsonb. - MySQL: load into
JSON. - Python: Polars, PyIceberg or the duckdb package.
- R: the duckdb package.
- Amazon Athena
- ClickHouse
- Amazon Redshift
- Snowflake
- Microsoft Fabric
- Databricks
- Apache Spark
- StarRocks
- Dremio
To explore in a notebook or BI tool, see Integrations. The querying overview compares engines.