io_uring is hot but hard to make work well in production. I will show you how I revamped a production database system based on Arrow and DataFusion to get 14GB/sec read speeds end to end, in Rust, using compio + Tokio. We will go over I/O APIs, HUGETLB and kernel page faults, why memory and scheduling are the true bottlenecks.
Evan has been building high performance data systems for 20+ years. He is working on custom databases and streaming systems serving trillions of events per day, using Rust, Apache DataFusion, Arrow, Tantivy, etc. He led a team developing FiloDB, an open source distributed real-time time series database, used as a production telemetry DB at Apple. He has architected, developed, and productionized large scale data and telemetry systems at companies including Apple, and loves solving the most challenging technical problems at both large and small scales, from advanced custom data structures to distributed coordination. He has led the design and implementation of multiple big data platforms based on Rust, DataFusion, Arrow, Spark, Kafka, Cassandra, and Scala/Akka. He has contributed to Spark and DataFusion/Arrow, and was a two-time Datastax Cassandra MVP.
