Skip to main content

Amazon S3 Tables

Send2Bucket appends to a table in an S3 table bucket through the regional S3 Tables Iceberg REST endpoint, https://s3tables.<region>.amazonaws.com/iceberg, signing requests with SigV4 for the s3tables service. It authenticates by assuming an IAM role in your account.

What you provide​

SettingExample
table_bucket_arnarn:aws:s3tables:us-east-1:123456789012:bucket/analytics
regionus-east-1
role_arnarn:aws:iam::123456789012:role/s2bucket-tables
namespaces2b (must exist)
tableevents

There are no stored credentials: Send2Bucket assumes the role.

1. Create the role​

Use the same trust policy as for Amazon S3: the principal is Send2Bucket's AWS principal, shown in the dashboard, and the condition is the destination's external ID. The dashboard shows the filled-in policy.

2. Allow table writes​

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3tables:GetTableBucket",
"s3tables:GetNamespace",
"s3tables:CreateTable",
"s3tables:GetTable",
"s3tables:GetTableMetadataLocation",
"s3tables:UpdateTableMetadataLocation",
"s3tables:GetTableData",
"s3tables:PutTableData"
],
"Resource": [
"arn:aws:s3tables:us-east-1:123456789012:bucket/analytics",
"arn:aws:s3tables:us-east-1:123456789012:bucket/analytics/table/*"
]
}
]
}

3. Add the destination​

Create the namespace in the table bucket, then enter the settings in the dashboard and select Verify. Send2Bucket creates the table with the columns in Delivery format if it is missing. An incompatible existing table is left untouched, and setup fails with incompatible table: ….

A batch is delivered when its Iceberg append snapshot commits. After a lost response Send2Bucket checks the table for the batch's delivery_batch_id before retrying.

Query your data​

To explore in a notebook or BI tool, see Integrations. The querying overview compares engines.