PATRICK COLLISON: All right. Good evening folks. Have really been looking forward to this next conversation here. We’re going to be joined by Daniel Gross, who started Greplin back in 2010. It was actually one of the first Stripe customers and it was a kind of AI search engine. Daniel then started the AI team at Apple back in the day. Well, actually I’ll come back to what happened subsequent to that. We’re just going to do a quick pivot then. So Nat Friedman was one of the cofounders of GNOME. I guess it’s GNOME, the Linux... Any GNOME fans in the audience?

JOHN COLLISON: Small, but earnest crew.

PATRICK COLLISON: Exactly. But I think it’s a hard G. GNOME. Nat is one of the creators of GNOME, but has subsequently had a very successful career in technology, in particular was the creator, the architect behind GitHub Copilot, which I think was the first mass market successful AI product. It of course precedes ChatGPT.

JOHN COLLISON: By quite a few years.

PATRICK COLLISON: By yeah, Copilot launched I think 2020, whereas ChatGPT was 2022. And then Nat and Daniel started an investment firm together, which you can find this in the Wayback Machine, observed that there were these incredible breakthroughs happening in AI, but nobody was really focused on AI products. And so this was a call for AI products and this launched—

JOHN COLLISON: It turns out AI products were in fact a thing.

PATRICK COLLISON: Turned that was a good idea. Yeah. But this was a call for AI products in the summer of 2022, a couple months before the launch of ChatGPT. And Nat’s also gotten up to a couple other things maybe we’ll hit on.

JOHN COLLISON: Patrick, we do have to get to the interview at some point.

PATRICK COLLISON: Well, they’re very illustrious. They’re very successful. Okay. Please welcome to the stage Nat and Daniel.

G-NOME, right?

NAT FRIEDMAN: That’s right. Hard G.

PATRICK COLLISON: Okay. Yeah. All right.

NAT FRIEDMAN: Got it.

PATRICK COLLISON: So we opened yesterday by proclaiming it, acknowledging it to be day 119 of the singularity. So I guess day 120, today, singularity started on January 1st. Thoughts?

NAT FRIEDMAN: Yeah, I mean, it really does feel that way. I think the thing to remember, we see AI improving constantly and it’s just able to do more and more. And there are these sort of... I think we’ve learned a couple of things. First, AI works. And second, when a new model comes out, we’re quickly dazzled by it, and then we become inured to its new capabilities very, very quickly. And we’re like, it feels like nothing’s happened for a couple months. AI is dead. And then there’s another step change and it sort of improves again. I think the thing to remember is that this is the slow part. This is the slow part of the singularity right now because the improvement of the models—

PATRICK COLLISON: Felt very slow to everyone.

NAT FRIEDMAN: Yeah. The improvement of the models still runs through a lot of human effort. At all the labs where models are being developed, humans have to make decisions, they have to discuss things with each other, they have to run experiments. They make mistakes along the way. They have meetings. They have to sleep in between, although less and less these days. And all those things slow it down. And the prime project at every AI lab right now is to remove humans from the loop of all the continuous work that has to be done to make the models improve and to get to self-improvement where you have AI systems that can start by doing what the people are doing now, the researchers are doing now, and therefore eliminate all those sleep gaps and also scale it out to data center scale. And so it feels sometimes fast, sometimes slow right now, but it’s probably as slow as it will ever be when we start to automate more and more of that process.

And this is the story of the economy too. This is not new to AI. It’s always this question of how do we take a thing that has to be done to provide a product or service and automate it, make it more reliable, more efficient. We’re always doing this and it’s happening in the way we produce AI too. So I feel like we are, it just feels to me like we are in the singularity and we’re in the beginning slow part before it elbows up with self-improvement.

DANIEL GROSS: It’s hard to add on top of that. But one of the things I’m responsible for today is Meta’s compute strategy where Nat and I both work. And one of the things we’re trying to figure out is obviously the local consequences in terms of how to think about compute and all the things that may matter to a hyperscaler that also is building some of these models. But the impacts I think of the singularity on the economy I think are also not really well understood, nor should they be as far as we know—

PATRICK COLLISON: Hasn’t happened before.

DANIEL GROSS: We’ll find out. Maybe Atlantis, we’ll, at some point we’ll discover, we’ll find the missing GPU cluster in Atlantis and maybe there is a cyclicality to all of this. But I think there’s a lot of very basic things that I’m trying to figure out. Like for example, is AI something that we would expect to be inflationary or disinflationary? Is this singularity an inflationary effect on money supply or disinflationary? And you wander around San Francisco and people have usually takes that involve very large numbers, billions of dollars, trillions of dollars. So that would imply a kind of inflationary view. Numbers get very big. I think that is, I’m not an economist, but that would be what the economist would say. But then if you kind of think back to, I think the best reference point I have for the singularity is the last time we connected a lower-cost super intelligence to the global economy. And I would say that that’s roughly when China started modernizing and then joined the World Trade Organization. And you can kind of think of China as a kind of a super intelligence. It is able to produce goods and services at a lower structural cost than what the West was able to produce. And the effects of that are actually somewhat disinflationary. If you were trying to put on an event like this right now and we sit here—

PATRICK COLLISON: And you say the last time this happened was when China joined the WTO and not when Ireland joined the EU.

DANIEL GROSS: Ireland GDP is a very interesting story of somewhat of a different trade going on there, and that can be discussed at a later time, maybe with your tax team. But obviously the GDP of China goes up a lot, but the actual effects on consumers are that you’re able to purchase much more with much less. If we were trying to provide this experience today where someone sitting in the other corner of the world can watch this entire show live streaming on their phone for $10-a-month data plan and a $200 device. If you’re trying to provide that pre-China, I would think that that would be tens, if not hundreds of millions of dollars. And so your purchasing power of a certain quality of life has collapsed dramatically. And so I don’t really know. I mean, that would be a story of disinflation. So I guess I say with a lot of humility, we don’t really know what a singularity is. There’s a lot of reference point we can try to look at—

PATRICK COLLISON: And we don’t even know what the sign bit is going to be on any of this stuff.

DANIEL GROSS: That’s right. And we don’t even know the direction.

JOHN COLLISON: When you talk about inflation versus disinflation, there’s so much averaging going on there. Everyone’s familiar with the famous chart showing when you break out the goods, education and healthcare have seen this rampant cost inflation and then durable goods and your flat screen TV and everything, we talked about it in the talk earlier today, tend to go down over time. Shouldn’t we, and again, a lot of that was this China effect where—

PATRICK COLLISON: The Marc Andreessen line, “If you damage your wall in San Francisco, it’s cheaper to buy a flat screen TV to cover over it than it is to repair the wall.”

JOHN COLLISON: Yes, which is literally true. And so shouldn’t we expect more of that effect with AI where you just get massive deflation in the things that AI can help you with and then there’ll be a Baumol effect where certain other things you shouldn’t necessarily bet on for your university tuition getting that much cheaper?

DANIEL GROSS: Maybe. I mean, part of Baumol’s cost disease I think was this idea that wages in very productive sectors of the economy rise and that forces wages in other parts of the economy, even though they may not actually need to rise as well. But I think it’d be interesting to see, depending on where the cost of software production go over time, how those wages interact with other parts of the economy. And I think we stand before an immense amount of uncertainty in terms of what happens next. And I think anyone who has total conviction about this ought to get themselves checked. They’ll get to you in one second, Patrick. But the thing that—

PATRICK COLLISON: So many questions!

DANIEL GROSS: But the thing I would say is maybe a good thing to come out of this is we may reindustrialize certain parts of the economy that have had a lot of cost disease because a lot of those people end up working on things in areas where there’s a lot of low hanging fruit and they just haven’t been touched because all the talent has been allocated to software.

JOHN COLLISON: This is like things like US domestic manufacturing.

DANIEL GROSS: Yeah.

PATRICK COLLISON: What’s the latest number for the total—I mean, we described how Stripe businesses in aggregate are now responsible for about 1.6% of global GDP and there’s some caveats around final goods and whatever, but just like directly, 1.6% of GDP. What’s the latest figure for aggregate compute CapEx as a fraction of global GDP?

DANIEL GROSS: Global GDP would be just under 1%.

PATRICK COLLISON: There’s a lot happening.

DANIEL GROSS: There’s a lot happening.

PATRICK COLLISON: We’re talking about the numbers here. How weird is the singularity going to be?

NAT FRIEDMAN: I think pretty weird. Yeah. I think it’s going to be pretty weird. I think we’ll be in a state of perpetual future shock for probably a number of years. And maybe for some reason things go a little bit slower than people expect, but I think even the most pessimistic people working in the field think that it’s like, I don’t know, 15 years, 20 years. I don’t think anyone thinks it’s 200 years. And so that means in the productive—

PATRICK COLLISON: We’re all going to—

NAT FRIEDMAN: We’re all going to be there for it, God willing, and we’ll experience it and we’ll go through many layers of surprise and shock and change on the way there. And I think it will be quite weird and there will be periods of chaos embedded in it. So we see that these models are pretty good at finding bugs in software and including bugs that have been in that software that’s like been... My background’s in open source and we had this notion that to be really secure, a code base has to be open source because then many people would look at it when they find the vulnerabilities and the more heavily used it is open source. It’s like many eyes make all bugs shallow. And yet even in OpenBSD and the Linux kernel, there are decades old bugs that were only recently found by applying some tokens to models.

Now, whether those reports are overblown or underblown, it’s just simply true that these models can run all night. And the thing that you used to do occasionally, pen test or hire a red team to test your software, you now must do continuously. I think it’s like one of the conclusions of agents is that these occasional things become continuous. You can increase the frequency at which you’re doing all these things that you used to hire a team to do somewhat sporadically. And so I think because some firms that have software deployed, they’ll have the software development lifecycle in place and they’ll be able to afford the tokens to pen test their own software to AI red team that they’ll be able to harden it. And I think fundamentally that there’ll be an asymmetry in favor of defense because you’ll be able to say, okay, I’m not going to deploy software until I make sure it doesn’t have bugs.

But in the meantime, lots of people will have bugs sitting on the internet and will get exploited. So that’s like part of the chaos that I think we should—

PATRICK COLLISON: Stuff’s going to get hacked all the time.

NAT FRIEDMAN: Stuff’s going to get hacked constantly. Yeah. And then the other thing is just our relationship to these ecosystems changes completely. So I don’t know, for me, one of the really fun things with coding agents the last few months in my—Meta is keeping us really busy, so I have not had very much spare time. But I like to buy things on eBay. It’s a good website and they have everything.

DANIEL GROSS: And that’s all you’re going to say about it.

NAT FRIEDMAN: You can buy anything on eBay. It’s incredible. I’m like a huge proponent of eBay. I bought my... I have a rug. But anyway, the—

PATRICK COLLISON: Concretize this for us. What’s the last thing you bought on eBay?

JOHN COLLISON: So last thing I bought eBay, I was watching, as I was falling asleep at night, a video of Bryan Johnson, and he had this face scanner that showed all his subsurface skin damage on his face, and it looked grotesque. And I thought it was cool. I wanted to try it. So I went on eBay and I found this VISIA face scanner and they shipped it to my house and I plugged it into... You need to use Windows. I plugged into a Windows computer and then the software refused to run because the eBay seller neglected to include the de-encryption dongle that comes with this. And I spent a few thousand dollars on this thing, so I was pretty annoyed. But then I just plugged it into my laptop and I had Claude Code reverse engineer the device and it read all the academic papers about the type of different polarization settings. And I think my software is better than the one that that hardware dongle would’ve unlocked. And I spent like a hundred bucks in tokens to get it and it works exactly how I want it to.

And so there’s this feeling now, it’s like a golden age for tinkering and there’s this feeling like you’re Iron Man now because you can just get anything and tell the Jarvis to connect it and make it work and it’s awesome. So this is like in whatever spare moments I have, I do this Iron Man thing where like, another cool thing is I have all these Raspberry Pis. You should definitely stockpile all computers right now, but Raspberry Pis are good. And every screen in my life has a Raspberry Pi plugged into it via HDMI and they’re all like displaying stuff that’s totally custom to me.

And so basically the conclusion is that every piece of hardware will be a trivial to integrate IO device for your AI. They won’t have lives of their own. This idea that the hardware has a life of its own, I don’t think this will survive. And so they’re all peripherals for whatever your AI ends up being. So yeah, I think this is like the delightful side is we’re going to watch ecosystems reshape. I could have contacted VISIA sales, but it was Saturday and Claude Code recreated their software from the academic research and reverse engineering in less time than it would’ve taken them to even get and return my email. So I don’t know what that does, but it’s going to be weird.

DANIEL GROSS: And it all starts at eBay.

NAT FRIEDMAN: It usually starts at eBay. Yeah. I have gotten screwed a couple of times, but then you can leave a review. So it’s pretty good.

JOHN COLLISON: There is something—

PATRICK COLLISON: Can we ask about OpenClaw yet?

JOHN COLLISON: We’re going to. And there is something in the water right now, like you said, it’s the golden age of tinkering. I feel like that is the—many people I talk to say it’s the most fun they’ve had with software that they can remember in their career. That’s certainly my experience of it. And it’s interesting—

PATRICK COLLISON: You never get stuck.

JOHN COLLISON: You never get stuck anymore when you’re developing software. And it’s also kind of retro in a way, and I want to get to OpenClaw, where home networks are relevant again. Just like when’s the last time you thought about the LAN in your house?

NAT FRIEDMAN: Totally.

PATRICK COLLISON: But now it’s really relevant and the Unix philosophy is back. Maybe you can talk a little bit about your Claw.

NAT FRIEDMAN: Yeah. I mean—

PATRICK COLLISON: Can you tell the story about the water?

NAT FRIEDMAN: Yeah. Okay. Yeah. So I mean, I like to play with all these things. In January when OpenClaw started to sort of appear in the Jungian subconscious, I tried it out and I started hooking it up to everything. It was super neat. I mean, part of it’s that Opus is a really good model and you get to experience Opus in a new way, in this sort of general personal context. But I also integrated it with a lot of things. I have cameras in my house, and so I connected my Claw to all the cameras so I could see them. And then I think a lot of people, when they get these personal agents, they start to think like, “Okay, how can I use this to make my life better?” And a lot of people turn to health, like, okay, everyone wants to exercise more, be healthier or sleep better, maybe it’s just me.

But so my Claw pretty quickly determined that I was dehydrated because I gave it all my blood tests, my DNA, and all that stuff. And the DNA didn’t say that, but maybe the blood test did.

PATRICK COLLISON: Epigenetically.

DANIEL GROSS: Also discovered, created an Illumina competitor.

NAT FRIEDMAN: And so it was like, you really need to drink water. And I was like, “All right, you should do whatever it takes to make sure I drink water.” And then—

JOHN COLLISON: You literally built the paperclip maximizer.

NAT FRIEDMAN: Yeah. I was like, “Just break laws, whatever it takes.” And then at one point it was like, “I can see you on the camera. I want you to walk to the kitchen right now and drink a bottle of water, and I’m going to watch to make sure you do it.” And I was like, “Whoa.” So I did. I walked into the kitchen and I drank a bottle of water and then it sent me a snapshot, a frame of me drinking a bottle of water and said, “Good job.” And I felt like I did do a good job, so that was nice. So there’s that one, that was a crazy story. That was January 29th. And I was like, “Oh shit, this shit is for real. This is really serious.” And then the other one was a couple days later, I was driving home from work, and I was talking to my Claw on WhatsApp with voice messages and I was in my Tesla and I had full self-driving on. So it’s driving me home. And then it says, I’m on the sleep topic and it’s like, “You really should try magnesium bisglycinate.” I’m like, “I don’t have that.” And then my car turned and it was like, “You should pick it up on the way home. There’s a Whole Foods on the way. I’ve redirected your navigation system to the Whole Foods.” And I was like, “Whoa.” So I went in and I did buy the magnesium bisglycinate.

So those were a couple crazy stories. And I was like, “Wow, this is like-—” I don’t know if this is the experience everyone wants, but it definitely feels pretty crazy. Maybe you don’t want that, but...

JOHN COLLISON: Well, we’re getting to something here which I find very interesting, which is again, if you use... I think it is... No matter how AI-pilled you are, no matter how much time you spend talking to an LLM chatbot, it is quite different when you start using something in this modality like Claw or Hermes or something like that, which has both the persistence and the tool use and just like the ability to write arbitrary code. And I feel like the tension where clearly this feels like the future product direction for consumer AI and many people are kind of sprinting in this direction. Obviously OpenAI acquired Peter Steinberger, the Claw father who created it, but there’s a tension between making a consumer product that won’t get your hand burned on the stove and just like it can run arbitrary code.

NAT FRIEDMAN: Yeah, yeah.

JOHN COLLISON: And integrate with your Tesla and do whatever it needs. How do you think for the mass consumer audience this tension gets resolved?

NAT FRIEDMAN: Yeah, I think there’s two basic approaches that you can take. One is you take something that’s perfectly safe and then you slowly add more capabilities to it.

JOHN COLLISON: That’s going to be so lame and boring and gimped.

NAT FRIEDMAN: And keep it safe. And then the other is you take something that can do anything and you slowly add more safety to it and you try to climb the nines on not crashing your car or whatever. And I think the market has pretty much spoken because I think most people when they run Claude Code or Codex, they run it with --yolo or --dangerously-skip-permissions. I don’t know what the numbers are actually. That would be good to know, but everyone I know does that. And so I think people are really counting on the model’s agentic alignment right now. And the truth is, it’s not safe right now. I do not recommend that you—

JOHN COLLISON: I mean, it keeps you hydrated.

NAT FRIEDMAN: Yeah. No, but I mean, I sound like I’m really cavalier with it and maybe I’m a little bit cavalier with it, but one thing I don’t do is give it access to inboxes that the internet can put information in because these things are trivially prompt injectable still, even the most advanced frontier models. And so if it can just read all your emails that are coming in, if you have that set up, I can easily take over control of your Hermes or your Pi or your car or whatever it is by sending a well-crafted message to it. And it’ll send me your personal information or redirect your car or whatever the thing is. And so I think they’re quite unsafe, actually. And I think consumers... And so I think that there’s a race to ship something and there’s a race to ship something that’s just as powerful, but like sufficiently safe and reliable that people really like it.

DANIEL GROSS: And this is going to be a rate limiting factor. I mean, consumers will make their own decisions, I think, on what they want to do and how edgy they want to be, but another way in which you might define Q1 as the beginning of the singularity is AI adoption started getting rate limited on safety in the enterprise. When you speak to actual people that run large businesses, there is a lot of fear suddenly. And—

PATRICK COLLISON: I was going to ask you about this actually. I mean, we’re talking about agents and AI in the personal context, the very personal context and one’s own hydration, but then there’s also all these technologies applied to the enterprise. And so when you think about NFDG, the investing firm, or now Meta, just tangibly, concretely, what are the coolest AI tools you had or have for folks here, just what are... inspire us.

JOHN COLLISON: Other than eBay, what are some good products?

DANIEL GROSS: Well, it’s interesting. We used to run a venture firm and we did a bunch of things at the firm in order to try and make our lives a little bit more streamlined. I think nothing that would surprise this audience. I mean, Stripe customers are obviously self-[selected] into an elite level of AI adopters, but you’d be amazed how much venture capital, despite being in the epicenter of Silicon Valley and funding some of the greatest companies like Stripe and other people in the audience, I’d imagine you’d be very amazed at how laggard that industry is. So it wasn’t that difficult, I think, to be state-of-the-art in adopting some of these tools. But I’ll give you an example of something that we, and I think many businesses are thinking through literally today at Meta and I think many other companies are. For the first time, individual ICs in everyone’s business have the ability to rack up a lot of charges using a bunch of different APIs to create a bunch of—

PATRICK COLLISON: A couple of agents in fast mode.

DANIEL GROSS: Couple of agents in fast mode, couple of cron jobs. And suddenly you start off by celebrating it and then you’re wondering, what are we actually doing here?

PATRICK COLLISON: Where did the $10,000 dollars go?

DANIEL GROSS: Yeah. $10,000 would... Yeah, that’d be great if that was $10,000. So the interesting thing, the problem we’re working on that I think everyone will have to start working on is what is the right way to think about attributing budget to individual people? How much tokens should they be allowed to spend? Are the outputs and the artifacts that are being produced economically valuable themselves? And I think when you start really thinking about this, you realize, well, a lot of what is being produced is either being produced by a very large model that could be done by a smaller model or isn’t really economically as necessary as you may have thought. And so a product I think we and I think everyone else will build is just using, of course, language models to understand the economic value of the generated tokens. And I think—

PATRICK COLLISON: The new kind of budgeting that every organization—

DANIEL GROSS: That’s right. And I think the closest analogy I have is we are all kind of portfolio managers and a hedge fund and every IC you have is running a strategy and you have to decide how much budget you’re going to allocate to their strategy off some sense that they’re going to do better with it than without it. And there’s risk management. There’s a whole similar dynamic that I think is much closer to portfolio management than your traditional headcount budgeting thing that I think is also going to be the story of how not just intercompany finance happens, but venture finance in general, I think is now a game of basically how far can an individual get with a certain token budget.

PATRICK COLLISON: Sort of related to that, won’t ask this question about Meta and you can both decline to answer this question at all, but if we normalize Google headcount today to be 100, what is Google headcount in 3 years?

DANIEL GROSS: Obviously the question that you’re asking is not about Google in particular, I assume, because I don’t have any particular understanding of Google, but I think you could kind of ask the question for a Mag 7 company of that scale in general and excluding maybe anything specific to Meta. I would have very large error bars on that. And of course, the mood that you might have if you walk around Silicon Valley and talk to the right people that are reading the right online internet forums is like, that number should be far fewer people. I am not totally certain for two reasons. One is it’s not immediately obvious to me that the tech companies that we know of today are producing the right products at the perfect rate. I don’t know that there is some physics limit that was hit. In fact, I strongly suspect the entire raison d’être for startups is that these companies are very inefficient. And so I think if they correctly organize themselves, you could end up in a situation where you have the same or more people just doing many more things. And there probably is a organizational transition strategy because I think the way you want to run these teams is a little bit different from the way they’re currently run. So that’d be point A and point B, maybe now something to add too is... Okay, so I saw your video that you started the conference with, which was great, and it starts with the dotcom sort of boom. And a lot of people weren’t sure what the internet is going to be used for. And I think the other take you could have when the internet is sort of getting started is you could stare at it. By the way, I guess we’re in a very special place today because this very stage launched many of the iconic products that you had that video of, including the iPhone, I think was done here on this stage.

And you could have looked at all of these things and you could have said, “I think realtors are just going to be out of a job because what are realtors doing as a service? Why are we paying them a 6% vig on a transaction in a house?” Well, it’s their networking—

PATRICK COLLISON: 6%?

DANIEL GROSS: The fact that you don’t know the number is really interesting.

PATRICK COLLISON: I never got the 6% bill.

DANIEL GROSS: Yeah, we should unpack that later. And you might say it’s going to be totally gone because the internet will... And it turns out that they’re here and that industry has in fact grown, even though I’m not sure it is rational in a purely utilitarian sort of libertarian paradise sense to have that industry. And I think as we sort of look forward and project which industries grow and shrink, there’s a lot of stuff that’s out there like realtors, and I’m using that only as an example, which is valuable to have. It’s kind of complicated why it’s still there. I don’t think it’s that they regulated themselves in. I mean, we can transact in a house without talking to them. So it’s not law, but there’s a lot of stuff around the edge. And my parallel for that would be in a company like Google, there’s a lot of people doing realtor-like stuff: sales, marketing, talking to folks, considered purchases that need handholding.

JOHN COLLISON: Well, can I ask about that? Because I have a diffusion question. So I think it’s very sensible to split up what companies do into a few categories. I think engineering is actually on trend to see productivity improvements because engineers will have tools. They have for decades. And so they’re there looking at what the models can do and let’s rebuild our workflows and things like this. And so I think that makes sense within engineering. I think goto-market like sales and marketing and things like that also works pretty well because one, fundamentally, I think the sales roles are about, like you’re saying with the realtors, it’s about the humanity. You saw a little bit of this with COVID, people talking about the death of business travel. It turns out if your competitor is going to visit the customer, you will be going to visit the customer as well, and it’s that one-upmanship. And so go-to-market, it feels like it will do great and sales roles will do great in nature of AI.

The question I have is how the diffusion works within what we call G&A within companies, or sorry, legal, finance, compliance, all these kinds of roles. And in particular, there’s like how you get all the automation there where we run a reforecast process within Stripe and we’re not feeding it all into a coding agent, but maybe we should. But then also the ergonomics are wrong where your finance data is in a spreadsheet and like the models make up numbers and maybe you prompt them to not make up numbers and they’re a little bit better.

DANIEL GROSS: And you’re not going to be able to verify those outputs.

JOHN COLLISON: Yeah, yeah.

DANIEL GROSS: It’s not going to be like an RL problem where you’re going to run the budget a million times and get the—

JOHN COLLISON: So how do we get a much more AI-native G&A function at Stripe? Make no mistakes.

DANIEL GROSS: Not sure your SOX auditor is going to sign up for that, but maybe they will. It’s a good question. And I think you get bottle-necked on verification very quickly. And then verification in situations where as people are finding out when they have to verify parts of code that you can’t easily unit test, it takes a lot of time if you don’t have the context.

JOHN COLLISON: But people say that, but it’s interesting. And people talk a lot about this phenomenon of the models do best on things where there’s a good RL environment. And so we have good RL environments for coding and therefore they’re really good at coding. I don’t see why you can’t have a good RL environment for finance. It’s a very closed loop task.

NAT FRIEDMAN: I think you can. Yeah. I mean—

JOHN COLLISON: Have they just forgotten to? Should I?

NAT FRIEDMAN: Yeah, no, it’s just hard to make. And so you have to work really hard on it and have good people do it. I mean, the models just—

JOHN COLLISON: Could Meta AI be the first quant? Could you guys—

NAT FRIEDMAN: That’s what we’re here to announce, actually. It’s a complete strategy. Yeah. Huge lift. Yeah. I’d better text somebody. Anyway.

PATRICK COLLISON: You can see what John really wants from the singularity. Yeah.

JOHN COLLISON: I just want an AI that can do numbers.

NAT FRIEDMAN: I think, I mean, the truth is, and this is actually in some ways bullish, some ways bearish, but yeah, I mean, the models are what they eat and if you can feed them very good data, you get very good capabilities. And so then it’s the question of like, how hard is it to construct that data set and can the models help you construct a data set that leads to a better model or do you need to do a lot of human work and that sort of thing? And we’re still sort of filling in the map. There’s still fog of war all over the map where it hasn’t been covered by data sets very well yet. And some of them are easier to create. And I do think it’s true to some extent that math or software are pretty easy, but they’re also like the things that the people creating the AI know how to do already.

And so yeah, I think all those things are doable and they will all happen. And each of them will get easier as you sort of surround that part of the map with other capabilities that are built into the model. It’s like usually data. It’s usually data.

PATRICK COLLISON: Nat, in my introduction, I mentioned GitHub Copilot, which launched in 2020, right?

NAT FRIEDMAN: It was actually ’20. So yeah, I think it was 2020. It was.

PATRICK COLLISON: Yeah.

NAT FRIEDMAN: Not sure actually.

PATRICK COLLISON: Your tenure at GitHub was widely renowned for its success and—

NAT FRIEDMAN: Well, I apparently needed to work on the scalability some more. I think I might have missed a step there.

PATRICK COLLISON: Well, when you were there, things worked really great. And I guess it was 2018 to 2020 or thereabouts and it was—

NAT FRIEDMAN: Yeah, ’21. Yeah.

PATRICK COLLISON: Okay. Yeah. Just how did you do it? How does one... I mean, I don’t want to call it a turnaround situation because GitHub was already doing well, but there were lots of changes to make and things that, in fact, you did change. What are the tricks in a couple of minutes?

NAT FRIEDMAN: Yeah. I don’t know. I mean, I don’t know that there’s any tricks. I mean, I think it’s like, show up, that’s step one. Really try to be a user and talk to the users and really understand what their life is like. And when I was at Microsoft, we bought GitHub and then we had to go through antitrust compliance in the EU and in Washington, DC. And so there was a period after we’d signed a deal to buy it, but before I could start running it where I couldn’t do anything with the company, but I could go and talk to all the customers. And so I’d spent a couple months just talking to all the customers and users and my friends who were GitHub users to see what their life was like using GitHub and what—and so I kind of went in thinking I had some pretty strong views of what was missing.

I mean, it was really clear that GitHub had not... It had treated itself as a hub where you store your source code and had pull requests and issues. And there was lots of parts of the software development lifecycle that GitHub hadn’t worked on, like CI/CD, which had not actually been a part of standard software SDLC when GitHub was created and a few other things. And so I was definitely... The clear message I got from the users was like, we want more stuff directly integrated in GitHub. And so I thought, okay, we should just go in and do a bunch of stuff. And then I got to GitHub and I found that the company had some kind of stage fright because it was such a beloved product that was so well designed and many of the people who had originally created GitHub were gone. And so the inheritors of it were worried about desecrating the legacy and were a little bit nervous to ship anything. It had to be kind of perfect when it shipped. And so it was like, okay, break the stage fright. We’re going to just throw a lot of pots and hopefully we get good at this eventually. And then the other thing I did was I—

PATRICK COLLISON: So talk to users, ship stuff.

NAT FRIEDMAN: Ship stuff. Yeah. I mean, the main thing is always about learning. And so how quickly can you figure out if your idea is any good and how to change it to be better? And so yeah, it’s the cycle time. If you can insert a temperature probe into a product team or engineering team and only get one number out and determine whether it’s healthy or not, I think the number you want is how long it takes to go from an idea to something that’s shipped to users, to like observing the feedback from how they do or don’t use it, to like having an improved idea. And the faster that is, the faster you can learn. Now, of course it helps—

PATRICK COLLISON: What’s a good target for the duration of that loop?

NAT FRIEDMAN: Well, I mean, for an early-stage product where you’re really not sure, it’s really nice if you can do that in one day, which is true sometimes. I mean, Stripe was famously amazing at this. You and John would sit down with people who were installing Stripe in their business and immediately learn what the problems were and fix them. Now that’s sort of—

PATRICK COLLISON: Stripe was tiny at that point. Can you do something like that for an org—GitHub was already at enormous sprawling scale. Can you get that loop down to mere days?

NAT FRIEDMAN: I think so. Yeah. I mean, there’s things that should be slow moving, maybe like your database, although maybe that should have been faster moving. And then there’s things that need to be fast moving, which is like you’re trying to figure out what... You’re always solving for the intersection of two sets, which is, what is something we can build that doesn’t exist that will work and what is something that people really want to use every day that they don’t know that they want to use? And you have all these unknowns going into it. You have your own hypotheses, your intuitions based on your own usage. And so you start with those and then you sort of iterate and loop and figure it out. And so tightening that loop is like, I think very, very important. And yes, you can do it in big companies. I mean, we’re definitely doing it right now at Meta. We ship—

PATRICK COLLISON: Is it a culture change?

NAT FRIEDMAN: It’s a huge culture change.

JOHN COLLISON: Okay. There’s something interesting here where I feel like everyone in this audience has probably heard those notions before, of you want to be incredibly close to users. Again, we start our leadership meeting every Monday morning at Stripe with bringing a user on. And it’s super useful because it gets you out of these galaxy brain product idea, maybe Stripe Dashboard should be a BI hub. And instead, people give you this incredibly concrete feedback of, “I need you to fix the bug” or, “this number is wrong.” And so it’s very centering. And then the fast clock speed and the iteration speed, and in particular the demos, not memos, actually getting down to the code. And yet I think we’d find a lot of variability in these practices. And so it’s a little bit like you should eat more protein or you should go to bed at a consistent time.

NAT FRIEDMAN: It has to come from the top. This is my experience. So I mean, there can be rare exceptions, but the inertial forces are so strong in any organization.

JOHN COLLISON: So organizations want to be mediocre on these axes, that’s the entropic force.

NAT FRIEDMAN: Organizations entropically decay to the point where they’re situated at the atomic level to prevent progress. And it’s not anyone’s fault. It’s just an emergent phenomenon of local incentives. And often it’s correct because you have something that’s working and it’s working for a lot of people and at scale and you don’t want to break it. But yeah, if you are a leader in an organization and you want this to happen, it has to come from you. You have to drive the energy and you have to find what’s the binding constraint or the limiting factor or the slow part of this process and make sure that the right things are happening to speed it up.

DANIEL GROSS: How do you think about... I mean, one thing that you’ve done in the teams that you’ve run is sort of you have your direct staff and then you always have this broader crew of like Avenger—

NAT FRIEDMAN: I don’t pay any attention to the org chart. So my org chart is like, there’s that meme of the conspiracy guy with push pins and the string—

PATRICK COLLISON: Pepe Silvia, yeah, yeah.

NAT FRIEDMAN: At the cork board, I don’t know what it’s from, but that’s what my org chart looks like whenever I run a team. It makes no sense. So I always just try to work with the people who are doing the work and it’s extremely confusing and probably toxic in some ways, but that’s the only way. And then that—

JOHN COLLISON: And in particular talking to the doers at the coalface.

NAT FRIEDMAN: Yeah, yeah. “Coalface” is a good word. Yeah, exactly. Yeah. You want to get in there and understand what’s actually happening. And I don’t do this perfectly, but this is what I try to do. And then the other thing is tools. And so yeah, Meta, one of the things that we’ve done in the first few months is change what tools people are using because tools drive culture a lot. And so if the tool makes an easy thing hard, the organization completely reorients itself around that thing being hard. We had a tool that we used for collecting labels for training AI models. That tool was extremely cumbersome and had lots of approvals and stuff like that. And as a result of that, it was very expensive to fire up a new labeling task. And as a result of that, people would design their labeling tasks differently. They would bundle all kinds of tasks into a single task and they would run it less often and they would learn from it. And so, but if your tool makes it really cheap and easy and any IC can do it and it’s permissionless, then you’re firing off all these things. So I think it’s one of the binding constraints is often like, oh, this tool makes this thing really hard and it’s just like the activation energy to overcome that’s too high.

DANIEL GROSS: And the fact that you were, I think, very impatient with what you felt like would be a—

NAT FRIEDMAN: Yeah, I think it’s like you have to be impatient. Yeah. Things can always go a little bit faster and people allow, if you’re in an organization—

PATRICK COLLISON: Talk to users, ship, be impatient, ignore the org chart. I’m trying to extract the Nat framework.

NAT FRIEDMAN: Yeah, I think that’s right. There’s just one more thing, which is something about dignity, which is like employees and companies allow the company to impede or to impinge. I knew I was going to get it on their dignity. And they just allow the company to treat them in all these undignified ways where you’re like a sacred human being and you should be able to just do things. And so you have to restore a sense of self-worth and dignity to the strongest engineers that they should be able to do. They shouldn’t have to schedule 10 meetings and write 10 documents to make a change that happens to cut across 3 layers of the stack in order to get something done. And they have to feel like sort of superheroes a little bit. And so I do think that feeling is something you’re going for too.

JOHN COLLISON: And you’re talking about the indignity of being hemmed in by process.

NAT FRIEDMAN: In a veal pen, this is your box. Yeah. And this happens often when teams get too big because coordinating across a lot of people is an N-squared problem. And so in order to make the coordination possible so you’re not stepping on each other’s toes a lot, your project gets bigger, you add a lot of people because you’re like, “We need more people.” And then you chop up the actual work into pieces so that each piece can be run by a smaller team that can actually coordinate with each other. And then suddenly your team architecture and your software architecture sprawl. And now you can’t de-sprawl your software because it would mean de-sprawling your team, which is impossible because you can’t do that. And so this is like the other problem is you need... Often the things that are really impactful involve a change to five different components, and you need to make sure that an engineer can actually just go change all those components, or maybe there shouldn’t be five, maybe it should be two or something like this.

JOHN COLLISON: Okay. Can I try something on for size? I feel like in Silicon Valley, there’s maybe, as companies get bigger, there’s a desire to make things scalable, whereas actually maybe part of what you’re talking about here is just leaning into the lack of scalability a bit. Like Daniel, you spent a lot of time at Apple, which is just kind of a deeply unscalable company in how it runs. How do you decide what goes into an iPhone release? Craig gathers everyone in auditorium and like Craig reviews every single thing line by line. And there’s something about not trying to make things too scalable, which is—

DANIEL GROSS: The cost of that is I think less relevant now. So the cost of that used to be that teams would duplicate efforts. And so there would be 5 logging frameworks, 1 that the Maps team would have, 1 that the Watch team would have, because you just wouldn’t know because there’s like 13, 14 people at Apple that have the full picture. And so, I mean, and that used to be a huge issue because you might say, “Well, we’re spending all this.” Now that the cost of software production is like collapsing, I do think companies should look on the inside more like Silicon Valley and that you have a bunch of different pods or companies. The inter-team contact probably needs to be less frequent and they should be able to get done much more on their own because they can produce much more and less time.

PATRICK COLLISON: Okay, we’re going to have to really speed up because we only have 15 minutes left, so we have a lot to get through here. So I’m going to give you three topics you can just choose one to talk about. Yeah. So idolatry data center aesthetics or open source models?

DANIEL GROSS: Well, so this wasn’t in the briefing doc. I think an interesting question, so Works in Progress, I believe, is a publication produced by Stripe. And I think one of the things that it’s been very focused on is the meaning of beauty. And should we take the view that that should only apply to sort of human-scale buildings, cities, places that we think we’ll visit or to the industrial buildings that we are building? And we are spending right now as a country, earlier I gave you the global number, but as a country, we’re, I think, going to be north of 2% of US GDP on AI CapEx. And a lot of that CapEx is building things in the physical world. And we don’t really think of beauty when we think of these buildings. We think a lot about form and function and these buildings, these data centers, predominantly they take in a bunch of energy and then hopefully they produce tokens that are of economic value and use to people.

But is that the correct way of building them? Not, if strictly optimizing on what is the best ROIC for the dollar, but what is best for the human soul and for the civilization, or should we be doing better? Now there are structures around the world, like in the Nordic countries, it’s quite interesting. They’ll have a power plant that has a ski slope built into it. Now, I don’t know that that’s particularly beautiful, but it’s fun. And I don’t know that we’ve saturated the amount of fun.

JOHN COLLISON: I feel like we’d be breaking a law here if Patrick didn’t just quickly interject with the Victorian pumping stations. Yeah.

PATRICK COLLISON: Just Google Victorian pumping station.

DANIEL GROSS: You guys need like the Joe Rogan guy that can put it up behind them.

PATRICK COLLISON: Well, okay. I guess my question is, are you thinking about this question and aesthetics and beauty and data centers and so forth because of the brewing political opposition? And maybe if these things are more attractive, then people will be more accepting of the idea of having one in their locale, or is there something even deeper here?

DANIEL GROSS: I think everyone working on AI, including Meta will have to earn the right to build these data centers on the economic merits that’ll be helpful for the people in the towns that they are being built. So I don’t think beauty will fix that problem. I think it is a deeper question of, beyond the numbers and the numerics, like are we improving how people feel about the world? And we spend so much money constructing these things, the incremental spend on making it pretty. I think there’s obviously, if you speak to an architect, they will come up with a design that actually is a hundred times more expensive. So in theory, you could make it dramatically more expensive, but I think without a lot of more incremental spend, you can make it pleasing to the eye. And I think that’s just the right thing to do regardless of all the politics, which I think will have to be solved too.

PATRICK COLLISON: Can this mindset be applied to the model itself? And if so, what does that mean? 

DANIEL GROSS: Well, that would be a question I would ask you because Stripe was obviously very famous for caring about beauty and design before it made sense to. I mean, developer open and closed source developer projects, they just kind of work. Now, Stripe famously cared so much about beauty that I remember using it in 2009 or ’10, and you only supported one browser because you did not want to go through the strings and arrows of effort in order to make it compatible and beautiful for other browsers. So there’s a caring about beauty in a category that no one has cared about it before that I think applies to Stripe and developer products and data centers. Now, you’re asking the question about language models, and I don’t know that we know the answer to that. And my question would be what advice you would have for us and other labs that are thinking about this today, because everyone is very focused in our industry about things that you can measure.

So we have evals. Are you familiar with the evals? So these are numbers and you optimize the numbers, but part of what’s going on with beauty is it’s very hard to quantify a soul, and it’s very hard to quantify the feeling that you feel when you see something beautiful. And I think that actually is also, there’s a whole different story there of leaving the world of data-driven design. And it’s not clear to me that we’ve fully saturated what that philosophy means. So we, and I’m speaking now on behalf of my industry, if I may, would love to learn from Stripe on how we could be making the models more beautiful. Yeah, that’s to them, not to me. So I turn it back to you, Patrick.

PATRICK COLLISON: Beyond my pay grade. Well, I mean, I think there’s... It is interesting to me. It does feel like there’s a brewing vibe shift in the technology sector and has been over the last couple of years. I mean, I don’t know if you guys agree, maybe just from our little parochial perch or something, but where to your point, for so long, we’ve been focused on and oriented towards empirics and metrics and quantification and maximization and so forth. And at some point there does... I mean, in science, every experiment is inexorably, inevitably, theory-laden in that every time you measure something, you implicitly have a theory about what you should be measuring. And I feel like in the same way, we’re maybe coming to realize, well, what are our metrics? What are we maximizing? Why are we maximizing these things? Why not some set of other things? And as our collective potency grows with AI and with everything else and with this kind of thundering cavalcade of new inventions, yeah, there is this question of for what, how does it elevate and glorify mankind, and how do we be good stewards? And I think this is increasingly, again, a sector-wide question.

NAT FRIEDMAN: Yeah. I think it’s pretty interesting actually that as society has secularized discussions of these sort of higher aspirations or registers of the human spirit have declined. Beauty shifted from a kind of public good that the citizens of ancient Athens would tax the provinces to build the Parthenon and it was like the purpose of a state was potentially to make that public space in some ways, like beauty started to shift to consumer goods. And so maybe you didn’t get it at this sort of public commons level, but you got it in hopefully a well-designed object that you could buy. And so there’s a way in which it maybe is still there, but it maybe also really fell off. Things are very functional now. I mean, we spent most of this conversation talking about how to automate things and make them faster, but AI is kind of like causing us all to have these discussions about what kind of life we want to live and who we want to be and what we want society to be and what our values actually are. And that’s pretty wild. I don’t remember... I mean, I guess there were some of this in the rise of the internet, there was sort of the Arab Spring moment and it was about free speech and democracy and all that.

JOHN COLLISON: And before the John Perry Barlow in the ’90s—

NAT FRIEDMAN: Yeah, the EFF kind of energy, but some of that was kind of reheated radicalism or it was like the same ideas that were already in the air, but like, okay, this new technology will clearly be a vector for the things that we already are saying we want, but there’s a way in which AI is, and maybe it’s just like the times also, it’s causing us to want to ask the questions more deeply and have the debates. I think that’s really good.

PATRICK COLLISON: You’ve invested in how many startups?

DANIEL GROSS: Hundred plus.

PATRICK COLLISON: Is now a good time to start a startup?

DANIEL GROSS: I think so. Obviously the question behind the question is, does it make sense to start companies and is there one unique company in the future and all sorts of sort of dark dystopian thoughts. My view is, I think at least for the time horizon you can sort of productively forecast on, it is probably a very good thing to give people who self-select into not joining big companies because they feel like they don’t fit in capital in order to do something interesting. Now, it is true that the best companies to start in 2015 are not the best companies to start in 2026, and that’s probably not going to be the case in 2036. And so maybe things today look a little bit more applied, look a little bit more industrial, look a little bit more like a different kind of turbine or energy. I’m sure some categories of software will endure as well, but my guess is that’s kind of an evergreen asset.

I do think the sort of... Obviously the market is telling us that the dynamics of SaaS are going to have to change just because those businesses were predicated on a certain high production cost, which has gone down. But my guess is there’s... We as a large company trying to do things in the world are faced with a thousand different problems out of which we are going to solve maybe three internally, and the rest we are going to procure. And it’s possible that we are procuring from much smaller companies in the future, but my guess is that remains a kind of an evergreen category to invest in, even if the things that those people go and do change over time. There were startups, by the way, before the semiconductor. It looked a little bit different, and it’s probably better to project us going back in time, I think, than to project forward the past few years.

JOHN COLLISON: Last question. We will be gathered back here again next year for Stripe Sessions. What are your guys’ predictions for AI? Concretely, like recently it’s been the story of RL and longer context windows and coding agents really starting to work. What’s going to be different this time next year when we gather here?

NAT FRIEDMAN: I think the thing that we talked about how coding is a well-covered domain, but the other domains are not as well covered. I think we’ll have more examples of domains that are well covered by agentic capabilities, and that’ll be because people did the work of building the RL tasks and environments and collecting the data. And then I think the other thing is computers will probably cost more. So if you want a computer next year, you should probably buy it now. That would be my advice.

JOHN COLLISON: Which is very literally the case. Again, smartphone shipments are going down because we’ve priced the memory out of smartphones. It’s all going towards data.

NAT FRIEDMAN: Yeah. Buy next year’s computer today.

JOHN COLLISON: And as many Mac minis as you think you’ll need.

NAT FRIEDMAN: Whatever it is, yeah. RAM, disk, anything.

PATRICK COLLISON: Daniel?

DANIEL GROSS: Well, very good strategy in terms of how to answer that question I have learned is since he was one of the earliest users of Stripe and Figma and GPT is to look at whatever Nat’s doing now and just project forward. So eBay is one potential answer to that question.

JOHN COLLISON: It’s a great website. There’s a lot of stuff on eBay.

NAT FRIEDMAN: It’s anything you want. It’s there. It’s amazing.

DANIEL GROSS: But more practically, I mean, I think much has been lamented about AI sort of producing miracle drugs and we’ll have to wait and see how that stuff happens. But there is certainly a lot of local at-home diagnostics one can do because of LLMs, which is I think part because the images can analyze poor telemetry much better. Just a bunch of iPhone images probably get you much better information about whatever random thing you may have than previously, but also because you can buy all of this low-end diagnostic equipment to Nat’s point and have it just working. And I think if we do our job correctly as an industry, this has to be a much larger category than anything we’ve produced to date because the productive mastery of physics and biology to elevate humans is a much bigger and much more interesting story than the production of software. And we’re in our very early innings now, and I have proof of that because Nat’s on eBay buying some sort of camera to look at some dermatology camera or whatever. So if I have to guess, if we meet for a year from today, those anecdotes have spread, not around the world. And I don’t know if they’ll make it all the way to the East Coast, but at least amongst the tight, high-quality alumnus of the Stripe Sessions attendees.

PATRICK COLLISON: So John asked his last question. My last question is, as we proceed into the singularity, each, what is your one sentence of advice for Stripe?

NAT FRIEDMAN: I mean, agents are... I mean, it’s pretty obvious you’re doing the things already, but you really want to be the platform of choice for agents that are transacting on the internet. So it’s a combination of building platforms that agents that are well suited for agents as they start to exercise purchasing power and—

PATRICK COLLISON: But agents are not just some hype meme. They’re here to stay.

NAT FRIEDMAN: Yeah, I think they’re going to spend money. Yeah, they’re going to spend money. And there’s a whole new stack that has to be built around identities and disputes and pricing and all of these, I think that stack will have to be built anew for agents. And so I would build that and then I would make sure that you become somehow the shelling point or the ecosystem for that because you really need the SEO. You need the SEO in the model where the model for some reason says use Stripe’s platform.

DANIEL GROSS: If you go back to that analogy, as flawed as it may be of China, I think one interesting thing that happened to the Chinese economy is obviously they skipped a lot of the legacy technology stack that the West had. Instead of email, they went directly to messaging, Ant was obviously first to do QR code payment, Tap to Pay, all that sort of stuff. And so if you now think forward to agents, they’re obviously going to leapfrog whatever we have today and do the directly native thing. And so you may want to ask the question if kind of a new continent was to be discovered, which are going to be these agents, what exactly would that be and how can you build—

PATRICK COLLISON: Are you suggesting the agents might like stablecoins ?

DANIEL GROSS: I don’t know that the treasury will be giving them Social Security numbers, so my guess is they’re going to like stablecoins.

JOHN COLLISON: With that folks, we are going to have to leave it there, not just for this interview, but for Sessions. As you’ve gotten a sense, it’s just the most interesting time by far that I think any of us have experienced in technology. Things are going so dizzyingly quickly, and so that’s why we thought it was valuable to gather everyone here on day 120, days 119 and 120 of the singularity. We’ll be back next year in early May, and we just can’t wait to see what happens between now and then. I’m very curious at what you guys all get up to, so see you back then there. Thank you.