We’re thrilled to welcome Madelyn Olson, Principal Software Engineer at AWS — and Valkey co-creator/maintainer — to the P99 CONF community this year. Madelyn will be presenting “The Vertical Scaling Wall, and How Valkey Gets Over It.”
Note: P99 CONF is a technical conference on performance and low-latency engineering. It’s virtual, free, and highly interactive. This year’s agenda spans Rust, Tokio, Zig, C++, Go, eBPF, observability, io_uring, kernel, distributed systems, and of course AI/ML.
We hope you’ll join us live on October 21-22 to hear the talk and chat with Madelyn. In the meantime, let’s get to know more about her!
How do you answer the dreaded “tell us about yourself” question?
I care a lot about open source and trying to solve real world problems. I was a maintainer of Redis for many years, and when the license changed I got together with my colleagues to create Valkey primarily because I cared about keeping the community together. Conveniently, my day job is being a principal engineer at AWS, where a decade on ElastiCache and MemoryDB taught me every way an in-memory database breaks when you run a lot of them, and left me caring a great deal about where the microseconds go.
What’s the most interesting project that you’re working on right now, or hoping to start soon?
Data tiering in Valkey. We have always been DRAM-only, and that assumption is baked deep into the design. It was the right call when the alternative was a spinning disk, or an SSD slow enough to wreck your tail latency. NVMe has been steadily changing that. Latency keeps dropping, bandwidth keeps climbing, and DRAM is the biggest driver of cost in Valkey deployments.
We built a version of this in ElastiCache a few years ago, and now we’re taking a fresh pass at building it for the open source Valkey community. Interest is coming from two directions. Workloads like KV caching, where you cache the prefill tensors from AI inference, have extreme throughput and capacity requirements. And the cost of DRAM has teams looking anywhere for savings. It’s a fun real world problem, with a lot of little optimizations that need to be made.
What will you be talking about at P99 CONF?
The vertical scaling wall, and how Valkey gets over it. There’s a persistent argument that you pick one: scale up by adding cores, or scale out by adding hosts. In Valkey we’ve been building both. One large node means one large blast radius, longer replication, and wasted money when your hardware shape doesn’t match your workload. But scaling out has historically been painful too, with slow slot migration and disruptive rebalancing landing exactly when an operator least wants to touch the system. I’ll cover how we benchmark, which metrics we recommend scaling on, and why command execution is staying single threaded for now.
What other P99 CONF talks are you most looking forward to, and why?
One of the things I’ve historically liked about P99 CONF is it tends to draw out talks that challenge my worldview. The ones I’m curious about this year:
- Well Designed Databases are CPU Bound (Tyson Brown). My tiering argument from the durability side. If a write pipeline on NVMe and direct I/O is CPU bound, then “disks are slow” was never a claim about hardware.
- Memory is Slow, Disk is Fast (Jared Hulbert). That’s my current project as a talk title, so I want to see where he lands.
- Escaping the Gossip (Luigi Tagliamonte). A Redis cluster that ran into horizontal scaling limits Valkey shares too. I’d rather hear how they thought about it than not.
Two of them support the argument I’m making and one of them undercuts it. That’s a good ratio.
Any performance-related resource recommendations for the P99 CONF community?
I think like a lot of folks, how I’ve been thinking about performance has changed significantly with AI and its ability to search and quickly prototype ideas. So instead of recommendations, I’ll make a pitch: go look at how some legacy systems got rebuilt in the last few months, in a codebase you don’t work on. Valkey is a reasonable place, and yes, I’m biased, but we’re small, we have almost no dependencies, and it’s C you can realistically finish.
Two examples from this year. We threw out an ordered index built on a skiplist, which had been there since the beginning, and replaced it with a B+ tree. We also rebuilt the main hashtable as a Swiss table. Both took a well trodden design and bent it around our own constraints, because Valkey is a strange shape: we’re memory hungry, and we go far out of our way to avoid latency stalls, since nobody wants a cache that times out. That combination forced a lot of real tradeoffs, and all of them are written down. Read the design issues and the PRs, or point an agent at them and have it walk you through why each call was made. That’s the reading I’d actually recommend, and it’s the one I’ve been enjoying both in Valkey and other leading OSS projects.
