Reynold Xin (@rxin) / X

Pinned

I find the Lakebase design for serverless Postgres very elegant, so I spent some time explaining how it works in this blog. The blog starts by explaining how databases really persist data (with a write-ahead-log and data files that are updated async), and how Lakebase separates

From monolith to Lakebase to LTAP: rethinking the database from storage up | Databricks Blog

Databricks has 4 immigrant founders. We moved to the US for the opportunity to work with the best minds in the world. Our projects power all sectors of the US economy. We employ 1400 people & growing, most in the US. None of these would've happened if we were to start in 2020.

Reynold Xin

@rxin

Nov 21, 2023

Today, more than 80% of the table metadata updates on Databricks are AI-assisted. It took 2 engineers, 1 month, and less than $1000 in compute cost to develop the custom LLM for this task.

Bespoke LLM for AI-Generated Documentation | Databricks Blog

The DeWitt clause is anti-competitive and we should foster competition and progress. At @databricks, we are removing the DeWitt Clause from our service terms to encourage open benchmarks and research, and calling upon the rest of the industry to follow.

Eliminating the DeWitt Clause for Database Benchmarking

The Unreasonable Effectiveness of Deep Learning on Spark databricks.com/blog/2016/04/0… This is super exciting - next major chapter in big data.

Reynold Xin

@rxin

Jun 15, 2022

Apache Spark has won the SIGMOD Systems Award, and Photon won the Best Industry Paper Award! This is a great testament to the adoption and impact of these two systems. If you are at SIGMOD this wk, please come by and say hi.

Apache Spark and Photon Receive SIGMOD Awards

Delta Lake is now part of the Linux Foundation! EBs of data/month, in production 1000s of organizations. Can't wait to see how the community will shape its future and establish it as a standard for data lakes. databricks.com/blog/2019/10/1…

Reynold Xin

@rxin

Nov 15, 2021

"We stand by our blog post and the results: Databricks SQL provides superior performance and price performance over Snowflake, even on data warehousing workloads (TPC-DS)."

Snowflake Claims Similar Price/Performance to Databricks, But Not So fast!

English is the new programming language. Generative AI is the new compiler. Python is the new byte code.

Want to write your Spark job in C#? Microsoft just published Spark .NET CLR language binding spark-packages.org/package/skaart…

Reynold Xin

@rxin

Jun 14, 2024

The best data warehouse is a lakehouse! Databricks SQL (our data warehousing product on top of the Lakehouse) became GA 2.5 yrs ago, and just crossed $400m ARR. One of the fastest growing enterprise software product ever.

Tomasz Tunguz

@ttunguz

Jun 13, 2024

Databricks revealed some sensational growth this week, as they did last year. Exiting this quarter to $2.4 billion annual run rate, the company’s revenue growth is accelerated year-over-year by 10 percentage points. Net dollar retention is a major driver of growth at 140%, which