Skip to main content

Delivery semantics

What a 200 means​

Send2Bucket answers 200 (or gRPC OK) only after the batch is stored durably. From then on the batch is Send2Bucket's to deliver.

A 200 does not mean the records are in your bucket yet. Batches are delivered at least every 60 seconds; each record then carries its batch_id and the delivery_batch_id of the object or table commit it arrived in (delivery format).

At least once​

Every acknowledged record is delivered at least once. A record can appear twice in two cases:

  • Client retries. Your client sent the same records again after a timeout or a lost response. Send2Bucket does not deduplicate requests: it sees two requests and stores both.
  • Ambiguous table commits. Send2Bucket checks the catalog for a delivery batch ID before retrying a table append whose outcome is unknown. That check covers Send2Bucket's own retries; it cannot detect a client resend.

To deduplicate downstream, put your own unique ID in each event, or for OTLP use the trace and span IDs or a record attribute you control.

Retries to your destination​

When a write fails, Send2Bucket keeps the batch and retries with backoff. These retries:

  • write the same object key with the same contents, so they never create a second object;
  • never add usage: bytes are counted once, when the request is accepted;
  • keep the batch pending until the destination confirms the write.

A destination that fails is isolated: its stream builds a backlog while other streams keep delivering. When that backlog reaches its limit, new requests get 503 destination_backlog with Retry-After: 30; batches already accepted are kept.

Destination status​

StatusMeaning
unverifiedNew or changed. The stream rejects data with 409 destination_not_ready until you verify it.
activeVerified, and the last write succeeded.
blockedCredentials or the bucket policy refused access. Organization owners are emailed. Batches are kept, and delivery resumes after you verify again or the next write succeeds.
errorThree transient failures in a row, such as timeouts or 5xx responses. Send2Bucket keeps retrying.

The dashboard shows each stream's status with its last error and consecutive failures, pending batches and bytes, the age of the oldest pending batch and the last delivery. It updates live as Send2Bucket records them, and warns when it has heard nothing for 30 seconds.

Ordering​

There is no global ordering. Records from one request stay together in one delivery batch, but delivery batches can be written out of order, and two streams are never ordered relative to each other. Use received_at, or your own timestamps, to order records.

Partitions and late data​

The date and hour in an object key are the UTC hour Send2Bucket received the records, not the time inside them. A delayed delivery still lands under the hour it was received.

After you disconnect​

Objects and table rows already delivered stay in your account. Disconnecting removes Send2Bucket's access; it does not delete or move your data.