[Speaker 1]: i don't think i said this publicly before, but i just call lucky was like local lucky. like if this doesn't have PMF by the end of the year, like we'll just like return all the money to you just saying that i don't want to work on this. i'm less into really working. so we want to give it the best shot this year. and i we're really going to go for it. we're going to hire a bunch of people who were going to be honest with everyone, like when i don't know how to play a game. i just play with open cards. lucky was the only person that didn't that didn't freak out.

[Speaker 2]: he was like i've never heard anyone say that before. everyone welcome to the latent space podcast. this is alessio on their kernel, and i'm shawn wang at latent space.

[Speaker 3]: so so they recording in the kernel studio for the first time, very excited. and today, we're joined by simon eskildsen, founder of turbopuffer. thank you so much for having me. and i do have to mention that like you are one of you're not my US member, the danish coders mafia, where like there's a lot of legendary programmers have come out of it like bjarne stroustrup, rasmus lerdorf, anders hejlsberg. and the V8 team and google maps team. you're mostly a canadian now, but is it not interesting? there's so much like strong danish presence?

[Speaker 1]: yeah, i was writing a post not that long ago about sort of the influences. so i grew up in denmark. i left it when i was eighteen to go to canada to to work at shopify. um and so i would still say that i feel more danish than than canadian. this is also the weird accent. i can say TH, because this is like that. i don't you know, my wife is also canadian. um and i think i think like one of the things in in denmark is, like there's just such a ruthless pragmatism. and there's also a big focus on just aesthetics like they're like very people really care about like where what things look like.

[Speaker 2]: i think i would love to get promote the position of turbopuffer, because i think you could be a vector DB, which maybe a bad word now on some circles, you could be a search engine like let's start there. and then will maybe run through the history of how you get to this point.

[Speaker 1]: for sure. yes, a turbopuffer is at this point in time a search engine, right? we do full text search and we do vector search. and that's really what we're specialized. then if you're trying to do much more than that, then this might not be the right place yet. but turbopuffer is all about search. the other way that i think about it is that we can take all of the world's knowledge, all of the exabytes and exabytes of data that there is. and we can use those tokens to train a model, but we can compress all of that into a few terabytes of weights, right? how to reason with the world, how to make sense of the knowledge, but we have to somehow connect it to something external. it actually holds that like in full fidelity and truth and that's the thing that we intend to become right. being the search engine for unstructured data is the focus of turbopuffer at this point in time.

[Speaker 2]: and let's break them. so people may say, well, then an elasticsearch already do this. and then some other people may say, is this search on my data? is this like closer to rag? and it's like a public search thing, how do you segment like the different types of search?

[Speaker 1]: the way that i generally think about this is like there's a lot of database companies. and i think if you want to build a really big database company, sort of you need a couple of ingredients to be in the air and it only happens roughly every fifteen years. you need a new workload. you basically need an ambition that every single company on earth is going to have data in your database multiple times. you look at a company like like oracle, snowflake and databricks. fifteen years later, there is not a company on earth that doesn't indirectly or directly is consuming snowflake or databricks. and i think you're in that kind of moment now. you're going to find a company over the next few years that doesn't directly indirectly have all their data available for search and connected to AI. so you need that new workload. the second thing you need is some new underlying change in the storage architecture that is not possible from the databases that have come before you. the architecture is now possible that wasn't possible fifteen years ago is to go all in on NVMe SSDs, a particular type of architecture for the database that is difficult to rather fit onto the databases that are already there. the second thing is to go all in on object storage. if someone has data in the database, they over time expect to be able to ask it more or less every question. those are the three conditions.

[Speaker 3]: i just wanted to get a little bit of motivation, right? like so you left shopify you're like principal infra guy. and then you consulted for readwise. and it kind of give you that idea. i just want to you to tell that story.

[Speaker 1]: for sure. so yeah, i spent almost a decade at shopify. i was on the infrastructure team from the fairly early days around 2013. the database that was the most difficult for me to scale during that time and that was the most aggravating to be on call for was elasticsearch. it was very, very difficult to deal with. and i saw a lot of projects that were just being held back in their ambition by using it. so i left, and i decided i was going to, i call it like angle engineering, where i just hopped around my friends' companies for three months and just help them out and solved some interesting infrastructure problem. readwise was one of them. i built a small recommendation engine. it was good enough that when i ran it on one of the co founders, i found out i got articles about having a child. but the company was spending maybe five grand a month in total on all their infrastructure. and when i did the napkin math on running the embeddings of all the articles putting them into a vector index, it's going to be like thirty grand a month. that just wasn't tenable. if the cost had been a tenth, we would have shipped that. and that haunted me.

[Speaker 1]: i couldn't help myself like i didn't know what like a vector index is. i sat down. i have this napkin math. why wasn't anyone build the database, you just put everything on object storage. and then you pull it into DRAM when you use the data. this seems fairly obvious. and actually S3 only became consistent in december of 2020. so now it means you don't have to have like big foundation DB, or like zookeeper, whatever sitting there contending with the keys. snowflake and others have to do it exactly just gone, right?

[Speaker 3]: when you say consensus layer, you strongly relying on S3, strong consistency, you are okay. so that is your consensus layer.

[Speaker 1]: it is the layer. so what compare-and-swap allows you to do is basically just download the file, you make the modifications, and then you write it only if it hasn't changed while you did the modification. that primitive was not available in S3 until late 2024, but it was available in GCP. we got really lucky, and we started on GCP because shopify ran on GCP. and when we started building the database, we really thought we had to build a consensus layer, like have a zookeeper. but then we discovered the compare-and-swap. we were closing deals with notion, actually, that was running in AWS and we were like trust us. you really want us to run this in GCP. and we had so much conviction that we bought dark fiber between the AWS regions in oregon and GCP. i don't want state in two systems. the worst outages are the ones where you have state in multiple places that not syncing up.

[Speaker 3]: they were really our biggest our second big customer after cursor, which also was a lot of late nights. right?

[Speaker 1]: the way that i remember it was that the day after after relaunch, i was just locked in on building it. one of the cursor co-founders reached out. they just speak in bullet points. so many, many TBs we have. this is what we're paying. i went over the office. postgreSQL was down. i was trying my best to see if i could help. they migrated everything over the next week or two, and we reduced their cost by ninety five percent, which i think kind of fixed per user economics. cursor is code a different workload than normal text. they will embedding entire codebase. they have their own embedding model.

[Speaker 1]: let's talk about you have this concept of the p99 engineer.

[Speaker 0]: the p99 engineer was just a term that we started using internally to talk about candidates and talk about how we wanted to build the company. i have a document called traits of the p99 engineer, and it's a bullet point list. i look at that list after every single interview. the default should be we're definitely not hiring this person. i want to see people fight for this person. if one person says it, then okay, let's do it. the p99 engineer has some history of having bent their trajectory or something to their will, some moment where they just made the computer do what it needed to do. our chief architect nathan bent the software to his will. ANNV3 can search a hundred billion vectors with AP50 of 1.4 milliseconds.

[Speaker 0]: i think another trait is that p99 spends a lot of time looking at maps. generally, it's their preferred UX. this is weaponized autism is what i call it. i love looking at maps. p99 is obsessive.

[Speaker 1]: you kindly gifted us your favorite. this is from the green tea shop in toronto.

[Speaker 0]: i consume lot of caffeine. i have an airtable table with hundred types of tea i've tried over the past fifteen years. i generally prefer chinese green tea. i have a little travel kit where i bring a thermometer. last friday, when we do demos, i spent twenty minutes walking through my airtable and going through my entire tea travel kit, including the temperature monitor. because you need eighty degrees for it.

[Speaker 1]: thank you so much for having me. it was a pleasure.