Tag: data-engineering

All the articles with the tag “data-engineering”.

• Data This Week #30

  6 Sept, 2026 · 7 min read

  Amazon acquires DuckDB Labs, AngelList's self-updating Markdown semantic layer, managing AI coding costs at Databricks, federated MCP access for AI agents, Glue 6.0 real-time mode, Polars 2.0 RC, and community advice on extracting from legacy on-prem databases.

• Data This Week #29

  23 Aug, 2026 · 5 min read

  DuckDB v2.0's new PEG-based SQL parser, Cassandra's ACCORD consensus protocol for ACID transactions, migrating multilingual full-text search to PostgreSQL, stream deduplication techniques, the hidden joins inside Snowflake MERGE, and the community debate on enterprise data catalog alternatives.

• Data This Week #28

  16 Aug, 2026 · 5 min read

  Instacart's Postgres-native search replacing Elasticsearch and FAISS, Netflix migrating batch workloads to Kueue on Kubernetes, AWS Dogwood for AI agent governance, distributed DuckDB with Quack, Debezium for CDC-driven event architecture, and the community debate on dbt Cloud pricing.

• Data This Week #27

  9 Aug, 2026 · 6 min read

  Safely migrating Iceberg catalogs without moving data, DuckDB's MySQL engine stress-tested at 500 GB, the end of the Hive Metastore era, hands-on S3 Tables evaluation, why dbt tests don't equal data trust, and an ontology+LLM approach to legacy data modernization.

• Data This Week #26

  2 Aug, 2026 · 4 min read

  Amazon MSK delivers Kafka data directly to Iceberg streaming tables, Iceberg v3 introduces native row-level lineage for CDC, Spotify builds a custom indexing layer for online point queries on the data lake, and a Rust terminal database GUI called Rainfrog.

• Data This Week #25

  26 Jul, 2026 · 6 min read

  S3 Tables compaction myths debunked, DuckDB's vectorized execution internals, Snowflake's managed Iceberg at scale, NOT EXISTS rewritten as anti-joins, partition affinity over Redis, AWS Glue view automation, and Supabase Pipelines in public alpha.

• Data This Week #24

  19 Jul, 2026 · 5 min read

  Databricks bakes native AI and semantic primitives into Spark 4.2, Netflix rebuilds LLM serving with vLLM on Triton, multi-cloud lakehouse architecture on AWS for agentic AI, Kubernetes internals deep dive, and Debezium's log-based CDC architecture.

• Data This Week #23

  12 Jul, 2026 · 5 min read

  Apache OSSIE enters ASF incubation to standardize semantic layers, HubSpot scales to 20B vectors with Qdrant, cloud-native financial search with Iceberg and Turbopuffer, Lakekeeper's Generic Table API for multi-format lakehouses, and versioning Power BI with Git.