Power BI
Power BI reads data through Power Query connectors, either by importing it into a semantic model or with DirectQuery. Connect it to the engine that already reads your Send2Bucket data.
| You have | Power BI connector |
|---|---|
| A Microsoft Fabric lakehouse or warehouse | Lakehouse or Warehouse, or Direct Lake in Fabric |
| Snowflake | Snowflake |
| BigQuery | Google BigQuery |
| Records loaded into PostgreSQL | PostgreSQL database |
| Athena over the bucket or S3 Tables | Amazon Athena, with the Athena ODBC driver |
| Redshift | Amazon Redshift |
| Databricks | Azure Databricks or Databricks |
This page follows Microsoft's Power Query connector documentation and OneLake's Iceberg documentation. The steps in Power BI and Fabric were not run by us. The SQL is from the engine pages, where we ran it against our sample data.
Microsoft Fabric
Fabric can show an Iceberg table that lives in Amazon S3, Google Cloud Storage or S3-compatible storage as a lakehouse table, without copying it:
- In a lakehouse, create a table shortcut under Tables that points at the folder of the Iceberg table Send2Bucket writes, the folder that contains
metadata/anddata/. Use a connection with a read-only key. - OneLake adds Delta Lake metadata for the table. It then appears in the lakehouse and its SQL analytics endpoint, with the columns from Table columns.
- Build a semantic model on it in Direct Lake mode, or connect Power BI Desktop with the Lakehouse connector.
OneLake converts the latest Iceberg metadata after each commit, which takes from a few seconds to two minutes, and only converts tables with fewer than 5,000 commits. Send2Bucket commits once per delivery batch, so expire old snapshots in your catalog regularly, and expect the Fabric view to trail the table by a few minutes.
For bucket destinations, create a shortcut under Files to the <prefix>/s2b/ folder instead, and load the gzip NDJSON objects into a Delta table with a notebook or pipeline on a schedule.
payload is JSON text. In the SQL analytics endpoint, JSON_VALUE(payload, '$.data.event') reads a field. Create views with flat columns for the report, as in the next section.
Warehouses and databases
Power BI's drag-and-drop fields cannot reach inside a JSON column, so create views with flat columns in the engine and grant the Power BI user SELECT on the views only. For example, the s2b_events view from Metabase for PostgreSQL, or the events_flat view from Tableau for Athena and Trino.
- Snowflake: server
<account>.snowflakecomputing.com, a warehouse, and a role withUSAGEon the database and schema andSELECTon the views. - Google BigQuery: sign in with a Google account or a service account that has BigQuery Data Viewer on the
s2bdataset and BigQuery Job User on the billing project. See BigQuery for the views. - PostgreSQL database: server and database of the database you load into, as in PostgreSQL. Power BI Desktop includes the driver.
- Amazon Athena: install the Athena ODBC driver and create a system DSN with the region, workgroup and S3 output location. Select the DSN in the connector, then query the tables from Athena.
- Amazon Redshift: server and database of the cluster or workgroup, with a user that can read the tables from Redshift.
- Databricks: server hostname and HTTP path of a SQL warehouse, with a personal access token or Microsoft Entra ID sign-in. Databricks reads Iceberg tables through Unity Catalog, or bucket objects through an external location.
Refresh
- Import copies query results into the semantic model. Set a scheduled refresh in the Power BI service; data is as fresh as the last refresh.
- DirectQuery sends a query to the engine for every visual. Use it for Snowflake, BigQuery, Athena or Databricks when you need current data, and filter on the date or
ingested_atcolumn so each visual reads little data. - Sources reached through an ODBC driver, such as Athena, and databases inside a private network need an on-premises data gateway for refreshes in the Power BI service. Install the driver and create a system DSN, not a user DSN, on the gateway machine.
Notes
- Every DirectQuery visual on Athena or BigQuery runs a query that you pay for. Prefer Import with a scheduled refresh for dashboards that many people open.
- OTLP attributes are lists of key and value pairs, and timestamps are nanosecond strings. Extract them in the engine's view, as in the engine pages, not in Power Query.