MEHDIOMehdio — practical data and AI education by Mehdi Ouazza

DuckDB 2.0 alpha is out! With any major release of a database or a data framework, you need to figure out what actually changed and how to get the benefit. Some improvements are pure magic under the hood, nothing to change in your query. Others should make you question how you model your data. Because yes, for any query, how your data is modeled drives performance. Take the VARIANT type. It shipped already in 1.5, but it's in 2.0 we see the performance shows up. If you keep querying the same keys of an event (structured logs are the typical case), VARIANT is a no brainer. The gain comes from shredding: when DuckDB writes a row group, it detects the fields that show up in most rows with a consistent type and stores each of them as its own typed sub-column. The rest of the event, the rare or inconsistent fields, stays together in a binary remainder. Still queryable, just slower. I also cover async I/O and the recursive CTE rewrite in the post. https://motherduck.com/blog/why-duckdb-20-is-faster/