Welcome back to Data This Week!
This edition is all about interoperability and the collapsing boundary between operational databases and the lakehouse. Microsoft is backing a vendor-neutral semantic layer standard, Cloudflare is challenging the cloud data platforms with zero egress fees, and both AWS and Snowflake are turning Postgres into a first-class lakehouse citizen. On the architecture side, we dig into a notorious Iceberg compaction failure mode, Supabase’s agentic-era roadmap, and what practitioners really think about dbt 2.
Here are the top reads, tools, and community discussions for senior data folks this week.
📚 Blogs to Read
Apache Ossie: Microsoft Ecosystem Support & Power BI Converter
Microsoft has officially joined the Apache Ossie (Incubating) community to support a vendor-neutral, open standard for exchanging semantic metadata across analytics, business intelligence (BI), and AI platforms. The key contributions are significant for anyone with a heavy Power BI footprint:
- Power BI Converter: A two-way open-source converter that translates Power BI semantic models (BIM and PBIP formats) into standard Ossie definitions, preserving relationships and metric calculations.
- DAX Integration: Establishing DAX as an Ossie-recognized query language so calculation logic can transfer seamlessly across tools.
- Ecosystem Alignment: Connecting Fabric, Snowflake, and Databricks to prevent “metric drift,” making it significantly easier to build consistent AI agents on top of your semantic layer.
Cloudflare Launches Basin: A Serverless Data Platform
Cloudflare has officially entered the data platform wars with Basin, built entirely on Apache Iceberg and Cloudflare R2. The biggest draw for data teams? Zero egress fees. Basin lets developers ingest, store, and query analytical data using open standards without vendor lock-in, effectively removing the cloud-to-cloud transfer tax that typically punishes multi-cloud data stacks.
Amazon Aurora PostgreSQL Integrates DuckDB for Direct Iceberg Querying
AWS just eliminated a massive chunk of reverse-ETL workloads. Aurora PostgreSQL now ships with DuckDB embedded natively, allowing you to run analytical scans directly against Apache Iceberg and Parquet files in S3. You can join live operational database tables with petabytes of historical data in your lakehouse in a single PostgreSQL query—no data duplication required.
Snowflake Open-Sources pg_lake for Postgres
Continuing the “Postgres as a Lakehouse” trend, Snowflake (via their Crunchy Data acquisition) has open-sourced pg_lake. The extension allows Postgres to act as a native catalog for Iceberg: you can create, manage, and write to Iceberg tables directly from Postgres while maintaining full transactional semantics, effectively bridging the gap between row-based operational data and columnar analytics.
Why Streaming Updates Break Iceberg Compaction
A highly technical deep-dive from Etleap on a notorious Iceberg edge case. If you’re building low-latency CDC pipelines, you’ve likely hit Cannot commit, found new delete for replaced data file errors. The post explains the mechanics of how continuous streaming delete files (both position and equality deletes) fatally conflict with long-running compaction jobs that are simultaneously trying to rewrite those exact files.
Turso Joins Supabase for the “Agentic Era” & Supabase Select 2026 Recap
Supabase just made massive moves to position itself as the default database platform for AI, highlighted by a major acquisition and a slate of deep architectural upgrades announced at their Select 2026 event.
- The Turso Acquisition: Supabase has officially acquired Turso, the pioneers behind the distributed “many-database” SQLite architecture. The strategic vision is clear: every AI agent gets its own isolated database. Provisioning millions of lightweight databases instantly is a workload Postgres traditionally struggles with, making Turso’s serverless SQLite the perfect backend engine for autonomous, agentic workflows.
- Scale Without Limits (Multigres & OrioleDB): For heavy production workloads, Supabase announced Multigres (a multi-node Postgres high-availability layer with built-in connection pooling, built by the team behind Vitess) and OrioleDB (a new storage engine that replaces the default Postgres heap storage with an undo log, effectively eliminating the need for
VACUUM). - Code-First Architecture: With Declarative Schemas 2.0, your schema and config can now live entirely in your codebase as SQL files, with
pg-deltagenerating migrations automatically. They also rolled out local development without Docker, allowing Supabase to run directly as native processes. - Agentic Tooling & Compute: Supabase launched an App MCP Server that lets users connect their own agents directly to your application (while strictly respecting your Row Level Security policies) and Supabase Compute, allowing long-running agents and web services to run right next to your data.
Read more → | Turso announcement →
🛠️ Tools
Babelfish for Aurora PostgreSQL
What it is: While not brand new, Babelfish is seeing rapid maturation. It acts as a built-in translation layer that allows Aurora PostgreSQL to understand queries written for Microsoft SQL Server (T-SQL). For senior architects plotting a migration away from expensive SQL Server licenses, Babelfish lets you swap out the database backend without rewriting your legacy application tier.
Splink (by MoJ)
What it is: An incredibly fast, highly scalable open-source package for probabilistic record linkage and entity resolution. If you need to deduplicate billions of records or merge datasets that lack unique identifiers (relying on fuzzy matching of names, addresses, or dates), Splink uses the Fellegi-Sunter model and runs natively on DuckDB, Spark, and Athena.
💬 Community Sentiments
Thoughts on dbt 2 Maturity?
Following the recent release of dbt 2 (and the confusing renaming of “dbt Fusion”), a discussion on r/dataengineering highlights that the ecosystem is heavily fractured right now.
- The Good: The new parsing engine is noticeably faster (nearly 2x faster for some 300+ model projects), and stricter YAML parsing catches typos that used to fail silently. Built-in linting (aiming to replace SQLFluff) is also a welcome addition.
- The Bad: Senior engineers are largely warning against using it in enterprise production yet. Major ecosystem tools (Astronomer, Snowflake, and vital packages like Elementary) are not fully compatible. Furthermore, changes to how database adapters are handled and corporate firewall issues with downloading the new experimental parser are causing major friction.
The Verdict: If you are running a 500+ model production environment, hold off on migrating to v2 until the ecosystem packages catch up.
That’s all for this week! See you in the next edition.