for Data Enthusiasts
Spark’s fixed costs dominate small jobs. How DuckDB, Polars and Spark itself changed in 2026, with independent benchmarks, real AWS and Databricks prices, and where Spark is not overkill.
A data platform is built so nothing surprising happens. An agent platform is built so surprising things can happen safely. Six differences, a tested one-run-versus-1,000-runs simulation, five places the two are merging, and what is GA and open source at Microsoft.
A beginner-friendly deep dive into LoRA fine-tuning: why weight updates are low-rank, the exact math, LoRA rank and alpha, merging, QLoRA and serving, with animations and a real experiment.
Your table can only be sorted one way, but your users filter it many ways. See how Z-order and Hilbert curves solve that, square by square with 24 orders and one map, why Hilbert opened up to 39% fewer files in a million-row test, and how to use it in Apache Iceberg.
Your coding agents are out of hand. Good. Claude Code is refactoring something. Codex is halfway through a test run. A third agent has been sitting on an approval prompt for eleven minutes, waiting for a keystroke you didn’t know it wanted. And in a tab you forgot existed, a benchmark has been running since … Read more
Spark job suddenly slow? Learn to find data skew in the Spark UI and fix it with AQE, broadcast joins and salting. Real screenshots and interview answers.
It’s 9:14 PM on a rainy Friday in Bengaluru. Nikita has just wrapped up a long week at work and wants to get home. She taps Request Ride. Within about 400 milliseconds her request has to be authenticated, rate-limited, priced, safely recorded and handed to dispatch. Within a few seconds, a dozen systems in her … Read more
Some system design questions sound small and turn out to be huge. “A third party sends us data; put it on a dashboard” is one of them. There’s no ride to follow and no fancy product. There’s a vendor you don’t control, a network you don’t share, a sale that turns a trickle into a … Read more
We adopted a practical Medallion-style approach to structure our data flows – segmenting data flows into Bronze, Silver, and Gold layers. As part of this redesign, we needed to optimize how curated data was exported to Snowflake. That’s when we hit performance issues with external tables. I know the common suggestion is to use the … Read more
By the end of this read, you’ll understand why Apache Iceberg is not just another open table format — it’s the seismic shift that’s redefining the role of Delta Lake in the modern data ecosystem and transform how the world thinks about data lakes. From Data Lakes to Data Icebergs: A New Era Begins Over … Read more