The interview transcript below has been edited for length and clarity.
Farhan Thawar: Let me tell you a story. So let's say we want to build a feature, and let's say you scope out the feature. It's gonna take two or three weeks to build. Or we could spend two or three months building the infrastructure so that you could build that feature in one hour. Which one should we choose, right?
And Shopify almost always chose the latter. We'd almost always want to build the infrastructure such that the feature can be built quickly. And the reason for that is, once you build that infrastructure, many more things could be built on top of it that you never thought about, whether that's internal first-party developers at Shopify or third-party developers in our ecosystem.
In the past, that was, like, a trade-off, right? Two weeks versus two months. But now with AI, you've got these, lovingly we call them clankers, right? These AI tools to actually just go and take the idea that you have with your steering around the architecture, how we want to build it, how it should fit into our infrastructure, how we want it to appear to the merchants.
All of those things can now be embedded in a way that you can almost build the infrastructure at the same speed as the feature. And so if you are building something and you're like, "Wow, I really wish I had this API," or, "I really wish I had a way to get this type of information," it is your duty at Shopify to actually build out that infrastructure for yourself to enable that for your feature, because many more people after you can build on top of it.
The experiment is literally what I mentioned. You would build a prototype. You want to see if something's gonna resonate. You might give it out to a small segment of merchants. You might give it out to a small segment of geographies. You might give it out to a small segment of buyers 'cause you're really testing something.
But we really do pride ourselves on taking the information from the test, deleting the code behind the prototype, and actually reimagining the solution from scratch with all of the learnings that we just got from that test.
Sam Witteveen: So which of the three are you building more now because of AI? Infra?
Farhan Thawar: It's always been infra.
It's always been infra, right?
Sam Witteveen: That is always,
Farhan Thawar: it's always been infra. One of the great examples of our infrastructure focus was actually during COVID when we were able to quickly launch things for many more markets, for many more merchants because we had the infra in place versus many other places I've worked at where you have people build feature by feature.
We've always built more infra. We will continue to always build more infra.
Sam Witteveen: Today we're talking with Farhan Thawar, head of engineering at Shopify, who runs AI across one of the world's largest commerce platforms. Farhan is deeply familiar not just with the tech of AI, but with the challenges of the current token economy.
And with 29% of companies cited in the VentureBeat Pulse survey saying that it's actually token costs that are killing projects, not model failure, when do you reach for the most intelligent proprietary model you can get, and when do you distill that intelligence into a narrow open source model tuned for one specific job?
This is VentureBeat's Beyond the Pilot: Enterprise AI in Action. I'm Sam Witteveen. Today's episode is presented by OutShift by Cisco's emerging tech incubation engine and driver of agentic AI, quantum, next-gen infra, and beyond.
One of the cool things with Shopify is you have a leader with a lot of vision.
His memo about AI, which I thought was really interesting from an engineering spe- perspective, "Our task at Shopify is to make our software unquestionably the best canvas for which to dev- develop the best businesses of the future. We do this by keeping everyone cutting edge and bringing the best tools to bear so our merchants can be more successful."
So you're in charge of building a lot of those tools, right? What is the developments that you're seeing from these recent frontier models? What's been unlocked for the Shopify team? What has that been, made possible that just wasn't possible a year or two ago?
Farhan Thawar: That tweet in 2025, right?
I call it the tweet that was heard around the world, where we said that AI will be the baseline expectation of Shopify, was really a way to solidify the thinking that- You're gonna have this tool available to you that you could use to remove toil, and you can use it or not use it, but we're gonna just, gauge your impact as if you had access to this tool.
We have this line at Shopify, which is, "AI replaces tasks, not jobs." You can just really get to focus on the best parts of the- your working style, and all this toil and drudgery, maybe you have to go over to this system and grab information and convert into a format to go to that system. You can use AI to do all that toil, and now you can focus on the more higher leverage parts of your job.
We really want all of our employees to embrace that in a way such that we can also bring that to all of our merchants so that they can focus on the thing we want them to focus on, right? You mentioned us building the best tools for entrepreneurs to use Shopify as a canvas for their products.
We really think that the more entrepreneurship is better in the world, and we just believe that all of these entrepreneurs, they wanna build amazing products and talk to their customers, and all of the toil around that, we can take from them. Having to have a storefront that is up, worrying about fraud, payments, inventory, fulfillment, loyalty, discounts, taxes, like, all those things, we wanna take away from them so they can just focus on build the best product and talk to your customers.
We really do think that by having these frontier models in the development of our employees to build things at Shopify and inside the product itself will allow those folks not have... They don't have to keep up with the latest developments in ChatGPT and Opus, and they can just focus on their products, and we can bring them all of those innovations into the product directly so they, again, they can just focus on amazing experience for their customers.
Sam Witteveen: From what I understand, Shopify's always had this really good ecosystem of people building plug-ins, people building apps around it, which benefits from you guys building the infra. In this sort of day of anyone can vibe code up something, is the danger for you to just build out all those things?
Do you consciously have to think, "No, we gotta pull back a little bit and leave something for the ecosystem," or...?
Farhan Thawar: We're focused on more entrepreneurs in the world, right? Whether it's us, it's someone else on top of our infra- on top of under- other infra that we never thought about, that is all net good for Shopify, right?
We want more entrepreneurs in the world, full stop, right? Does that mean it has to be on Shopify? No. But we wanna make sure the infra is so easy to use and so seamless and frictionless for the merchants on our platform and for buyers to transact that we just believe that it's gonna be much...
it's gonna improve the ecosystem and have more entrepreneurs in general. And so when we build infra we always try to think about the problems that we wanna solve and potentially problems other could solve on top of on top of the Shopify platform.
Sam Witteveen: Okay. So how does this actually show up with your developers?
I know you've been a huge proponent of pairs programming going back quite, quite a way. Is that still valid now in the days of Claude code? What tools are in your stack for your developers now?
Farhan Thawar: We deployed GitHub Copilot in 2021, which is a year before ChatGPT. And that was very much like the pair programming vibe, right?
I don't know if you remember even the announcements around then, they're like, "It's your AI pair programmer," because you could write code and feel like you're not by yourself anymore. You have a... You type in part of a line, and it would complete the line. You would write a comment, and it would complete the whole function.
You would literally be pair programming with the AI. And at that time still, we felt like having two humans at the keyboard and pairing with the AI was going to increase the learning rate of what we were, trying to learn about our customers, and of course would increase the velocity of shipping things to our merchants and buyers.
And so net-net for two years we had this like AI pair programming. And then you hit this like December Opus 4.5, GPT-5.2 moment in 2025, where all of a sudden we started seeing that actually the agents were better at writing code than us. And I still feel like having two people at the keyboard is going to increase your learning rate because you're gonna learn different things from the LLM that's giving you feedback as you're building, and different things from the human beside you, another smart person, while you pair across all of the work that you are doing for our merchants.
And so both those things together, we think like the ideal, like AI centaur is not just human and LLM. It's like it could be two humans pairing and the LLM helping you remove the toil from your workflow. And today we still have a very strong culture at Shopify. We don't pair like 40 hours a week like in some companies, and what I've seen in my past as well, but the teams do regularly get togeth- get together and pair on certain parts of building out their infra or incidents, and they bring those tools to bear to make sure they can move forward.
On the AI tooling, so we're big fans of of using all of the frontier models, right? We actually have an LLM proxy internally, and what that allows us to do is we can buy tokens in bulk from all the providers. We can have a proxy that everyone in the company connects through, and it gives them access to the latest models, it gives us access to reporting, it gives us access to failover, right?
So you might be using a model on one provider and there's an availability issue, we can automatically seamlessly transfer you to another provider. Same thing with our in-production products, and you can just seamlessly feel like, "Oh, I'm using Opus," and it's spraying across the other providers behind the scenes, and you don't have to care that, one of them is having an availability issue.
It also allows us internally to really ask like interesting questions, like not just on token spend and toke- and numbers of tokens, but also, "Hey, who's using the most expensive tokens? Who's spending more times on reasoning? What types of models are being used, and what disciplines, and by levels?"
And so we can get a good understanding of what's going on from a token usage perspective inside the company.
Sam Witteveen: What about the actual... So are you guys a Claude Co- Claude Code, Codex? Or is it just anyone, people can choose whatever they want?
Farhan Thawar: So we're big fans of having, people having choice.
So once I buy the tokens, I don't care what harness you wanna use. We expose everyone to the different harnesses so they can get a feel for what may or may not work in their workflow. So Claude Code, Codex, Pi, Cursor, GitHub Copilot inside VS Code, like, all these things inside VS Code. People can choose however they wanna use it.
And then we also have our own agentic platform, like River, which again, switches between models, our products switch between models, and then we even have open source internally. And so there's lots of ways to get your hands on different AI from whatever agentic harness you wanna use. And many people, by the way, and maybe this is counterintuitive, use different harnesses for different things.
You might pull up Cursor for something, you might pull up Claude Code for something else, you might be in Pi, you might write an agent. All of those things are available to you.
Sam Witteveen: Yeah, that seems definitely to be a trend that we saw just recently with the OmniGent that got released, where it's a meta harness that uses multiple harnesses and stuff like that.
Farhan Thawar: I would say the more interesting thing is that we're seeing just this growth outside of R&D, right? My surprise was that having folks in not in R&D start using a terminal or start using a Cursor window or using Claude Code and then, and Codex, using the Codex app. And so I see, it's not just R&D, right?
It's people in talent, HR, finance, sales building out tools for themselves. We also just recently went public with our Qwik platform, right? I'm from the early '90s of the internet and we had GeoCities. Remember? You could put up any website anywhere, and it's our... It's really, it's very much our own internal GeoCities, which lets you drag and drop an HTML file or JavaScript, and it hosts it for you.
'Cause what would happen is you, Sam, might build a tool and be like, "Look everyone, I built this tool for me." And they're like, "Can I see it?" How do I see it?" And you're, like, sending screenshots and now you can just literally deploy it and be like, "Cool, here's sam.qwik.shopify.io," and I can share it with everybody and I can keep it updated with my current workflow and everybody can see h- this tool that I built.
Sam Witteveen: So how do you manage when everyone's trying to ship code? We did some interviews, for example, with LinkedIn, where they talked about the idea of a full stack builder. And that what they were finding was that product managers and other people, designers, everyone is now starting to contribute code. And that causes a lot of challenges, for engineering where you have to start thinking about, okay, is that code safe?
Is that code secure? Or how are you guys doing that at Shopify?
Farhan Thawar: There's different tiers of applications here, right? So when we talk about things that land on Qwik, just N of one software, I'm building something for myself or I'm building something for my team, I'm in sales. Like all that stuff there's no, like it just goes to Qwik and there's no reviews, right?
It's internal only. It's, we're not worried about it. We have a secure version of Qwik, which allows you to say, okay, I only want to have four people read this or be able to access this tool, so I want to launch it there. But when you're talking about production code that's going to our mer- our merchants, which are our customers and our brands, that's still the onus and the responsibility still falls on the engineer.
So anyone's allowed to prototype and actually we see this now in the R&D cycle where designer, PM, they're prototyping in high fidelity the experience they want to show to our customers. And it's so much better right, than showing a screenshot or a deck or something on the experience. So they show a working prototype.
It might even connect directly into our APIs or it might be fake in the backend, but it shows you a higher fidelity view of what we want to ship. Then we take that inspiration And then we try to build it for real. And I mentioned we don't just try to build a feature, we try to figure out what's the right infrastructure that feature should live on.
Does it exist? Great. Use it. Does it not exist? Maybe we should... Not maybe, we should build the infrastructure first so then we can enable it. And that, those are still owned by engineers. And we have a saying internally that you can't the bot can't be responsible for the things that go into production.
Of course, you're gonna be working with AI, of course, we might even tag it with Claude Code. I wrote this with Claude Code. Great. But you're still responsible. The human still has to be responsible of what goes to production. With the volume of AI code, we of course have checks that go through the code before they go to production, and we have these things in place, but that doesn't mean anyone can just vibe code something and try to get it through our pipeline.
It still has the responsibility layer of an engineer having to own it.
Sam Witteveen: Everyone's been talking about token maxing. How are you thinking about that now? The bills are going up.
Farhan Thawar: Yeah. There's two ways to think about it. So in 2025, again, when Toby put that tweet out AI's gonna be reflexive at Shopify, it was all about just the closing the distance between you running into a problem that has toil and you reaching for AI, right?
The sooner you reach for AI, the faster you can get unblocked, the faster you can remove the toil from your path, and then you can actually work on the thing that you wanna work on. It wasn't about using the most tokens, it was just by, about reaching for it as soon as possible. And then of course, yeah, it turned into oh my God people were using tokens, and then leader boards would appear, and people wanted to be on the leaderboard, and of course the incentives were, like, misaligned.
We don't have a leaderboard anymore. We have a usage dashboard. You renamed it, right? We renamed it but we also now try to highlight the utility behind what we wanna learn from it versus who's in the top 10, right? We also moved in 2026 away from AI reflexivity to AI leverage, meaning it's not just about now that everybody's reaching for AI all the time, wait we don't have that problem anymore.
The idea is to get people to really think about where they can get leverage from AI, right? I call it, like, how can I brag about being so lazy that I had this tool build the, build out what I need to do need to build out or remove this toil from my workflow? We also do have circuit breakers in place, so we don't have limits on our tokens, but we have circuit breakers in place which basically allow someone to get a message if something they're doing is long-running and spending a lot of tokens, and they will get pinged and said, "Hey, you are, you just spent X dollars.
Did you mean to spend this?" And sometimes the person is "Oh, absolutely." Press the button, unblock, keep going. Other times they're like, "Whoa, I didn't know that doing this task, it was running in the background. I totally forgot about it. It's gone it's been running for 10 hours. Actually, I don't think it's valuable.
I'd rather stop it now," and then we can catch things that are runaway at that point. But we don't have that we don't have people wanting to be in the top 10 or anything like that in terms of token spend.
Sam Witteveen: This series is brought to you by OutShift, Cisco's incubation engine. By creating an open interoperable infrastructure, OutShift is enabling agents and humans to share intent, context, and reasoning.
The cognitive evolution for agents is here. Explore the internet of cognition at outshift.com.
Sam Witteveen: Is it engineers that are not aware that they're spending a lot, or is it other departments? You talked about your pro- your proxy system. I wanna come back to that. I imagine that gives you very high uptime for everything, which is pretty rare in LLMs at the moment.
But also gives you really interesting insights into how the tokens are being used, and it sounds like that is like a big plus versus token maxing.
Farhan Thawar: Any person in the company can see their spend in AI. That's easy for employees to see, and I think it, it's good to know, hey, I spend, $10 a day or I spend $100 a day.
I think it's worth people understanding that idea. And it is true, you are right. Engineers spend the bulk of the tokens because, of course, their, the code writing and architecture and reading of code and understanding of code and all of the things around the actual building of the product are where a lot of the token spend happens.
And you're right, the proxy does give us a lot of insight, but the insight is, it's a learning insight, right? So if someone spends a lot of tokens, my reaction isn't to go to them and say "Oh my God, you spend so many tokens. What are you doing?" Yeah. It's "Hey, you're doing... It seems like you're doing something interesting.
What are you actually trying to do here? I wanna understand what you are, how you are using AI as a lever to improve the quality of the output here." And I find amazing things, right? Sometimes I find things I'm like, "Oh, I didn't, I wouldn't have even tried that." And people are seeing the ability to learn from others by seeing that.
But I would say the bigger unlock on the cross-team collaboration aro- ar- around AI has really been River, right? River, the, the product I mentioned earlier, which is our agentic substrate across the company. River can see Slack messages, it can query into our data warehouse, it can see our product management system.
And more interestingly- River only works in public, which means you can't go into a private channel and say, "Hey, River, help me build," or ask this question. You have to go into a public channel and ask it. And what's happened is everybody has a public channel. I have a Farhan River channel.
And in the channel I ask all these questions and people can see. And what's cool is I might ask a question and it might not... River might not h- be able to figure it out. Someone might jump into the channel and say, "Hey, River, how come you didn't check like this table in the data warehouse? That's clearly where this data lives."
River goes, "Oh, you're absolutely right." It finds the... It finds that table, but now any future query to River learns and will never have that mistake again. It will never not find that table. And in future people will be like, "Oh, I just asked a question to River, and it came back with the answer," because somebody jumped into my thread to to answer it.
So we are seeing token... To go back to your original question, the growth of tokens being spent by River is going up. The number of PRs being submitted to our our code base from River is going up because people are starting to see that as a way to kinda get their work done because it has a substrate of information across the whole company.
Sam Witteveen: A- and my guess is that you're all bene- you're benefiting from everyone else being human in the loop for it not just yourself kind of thing.
Farhan Thawar: By the way, I think it was an unintended consequence, right? Putting River in public was intended, but the fact that other people are jumping in to help River understand a problem a different way, and then it changing the now way River answers that question in the future, I think was an unintended benefit.
Sam Witteveen: Fascinating. One of the things that we've been seeing a- at VentureBeat is w- we've been doing a lot of different sort of research. When people are looking at the different sort of failures of proj- projects, people talk about both the model, but also the sort of infrastructure around it. How do they balance this open versus closed things?
Perhaps just very quickly explain to people what distillation is.
Farhan Thawar: Sure. Yeah. So distillation is this idea that you can have two models, like a teacher model and a student model, and you can train a student model usually on a narrower task, right? It doesn't have to be. Some people I think, try to distill into a generalized model, but for us, usually a narrow task And what happens is the student model ends up being very specialized for the narrow task, and you get a bunch of benefits from that.
One, you typically... You... It's a smaller model, so it tends to be faster. Two, it's a smaller model, so it tends to be cheaper. And then three, this is the, I would say, the non-intuitive part, it can actually get better at the narrow task than the teacher model, right? And let's get like a dumb example.
Let's say you wanted to have a, yeah, a model that find your favorite movie or whatever. You might wanna train a smaller model just on that one subtask of finding really good movies. And you can make it, again, smaller so it's faster, smaller so it's cheaper. But also maybe because it's just focused on that, it does better than the general model on finding those amazing movies.
Now, the downside is one, you gotta train it. So if you change your... If I said, "Oh, I want not just movies, but I also want favorite songs," you gotta retrain it because it's not really designed for that. And then two, you lose generalization. So that j- frontier general model, you can ask it everything from the weather and other things.
This smaller teacher model, because it's been fine-tuned, it only really is good at that that narrow task. And so at Shopify, we've done this a few, in a few places whereby we have... obviously, we use the frontier models in lots of places, but we have these narrow tasks, right? So for example, inside of Sidekick, Sidekick is our flagship AI product inside of our admin.
It does lots of subtasks for for merchants, so that helps them remove toil. And some of these subtasks don't need to call the frontier model. We were able to distill some of these things to be smaller smaller models. So again, they're faster, better for the user, they're cheaper. And then again not counterintuitively for us anymore better at that subtask than the generalized model.
And so we see savings in size from 2X, which is huge down, sometimes down to 30X, like 30 times cheaper, faster, and more, and of higher quality. And then of course, we still use the frontier models for a lot of the steering.
Sam Witteveen: How does someone go, "Hey, we're doing this task. Maybe we should distill this task."
This is a very specific narrow task. Maybe it's one feature o- on something. How do you actually have a pipeline that goes ahead and does it?
Farhan Thawar: So instead of just building this one time and be like, "Oh, I'm gonna take this, part of Sidekick and distill it by hand," we early on decided to build infrastructure so that anyone could build a distilled version of their subtask.
And so we call it UDP, universal distillation platform. And what it allows us to do is you give it the teacher model, you give it data, you give it the evals, and you give it the target model. You're like, "Okay, I have Opus 4-8 and I wanna distill this subtask down just for this narrow thing to Qwen,
Sam Witteveen: right?
Qwen 3.5, which is a, which is an open source model."
Farhan Thawar: And the pipeline will run through the data and the evals. Takes, I think, about a day or so to come back, and then comes back and says, "Hey, here's the output, the evals of what we were able to find and what we were able to fine-tune the weights of this open source model."
And then it gives you an evaluation. It'll say, "Okay- Here's what we got. And it can be, faster, hopefully, cheaper, hopefully and better at the sub-task sometimes, not always. And then we can then decide, we'll say, "Actually, this is a good trade-off. We wanna deploy this into production." But because we built it as infra, there's no, what's the right word? There's no approval process or anything. There's no there's-- It's literally there. We have other pieces of infra. We have a, an internal platform called Tangle, which I think we talked about, it's open source now, that allows you to visualize The actual running of this of this universal distillation pipeline as well, and makes it easy for anyone to take any sort of task that they're running now with any sort of model in the cloud, to distill it down to something they might wanna run locally in our own, on our own GPUs.
Sam Witteveen: So I, I actually saw some of your people give a talk at ICLR last year about exactly this. How often is that happening? Is that happening every day in Shopify? Is it just engineers that are doing that, or is there now, is now this making its way across to marketing saying, "Hey, I need a model that does this super well"?
Farhan Thawar: My dream, and I've talked to the team about this already, is to actually not give the distillation pipeline a target model at all. What I'd love to do is give it the data, the evals, and the teacher model and say, "Based on, your learnings over time, I want you to look at a different class of model, different sizes, different types, and for you to tell me what the right distillation target is."
'Cause maybe we'll get surprised. Maybe it'll be a such a small model it could run on a phone based on your problem, and other times maybe it comes back and says, "There isn't a way to distill this down to anything better than what we have at the frontier." I'd love it to be, to get... And we're moving in that direction so it gets even smarter.
You can kinda give it less data and it can learn from that. Your... It's a good question on the non-R&D versus R&D. So far we've only seen really R&D folks using the distillation pipeline. It's a great idea though to think about it as a marketing or other types of tools that potentially you could say, "Hey, for this use case we do all the time in this other discipline, what if we trained a specific model to help us with that task versus having to use the frontier?"
Especially if we can get the accuracy to be higher on that task. So I will write that down as a feature, but we don't have that today.
Sam Witteveen: One of the big questions we always get when we talk about things like this is how do our people know what to do for evals? Everyone just says evals" , but what does that actually mean, and how does that actually look in Shopify?
Farhan Thawar: Yeah, so actually on the... It, again, depends per product, but evals are actually a really important part of the process because if you don't know, if you don't have a repeatable way to actually judge how a new model might affect the output of what you're working on, then you're in this world of everything's a one-off, right?
You build it, and then you're not able to plug and play a different model, a different size, different reasoning levels, and the evals are the way that we do that. Typically starts with us having a golden dataset of of data that we can use per problem. We also work to generate new data, right?
So we actually work with a firm called Toloka that helps us generate new data around this stuff as well. Because what happens is you might be doing an evaluation for something that isn't necessarily well-served by a frontier model. Let me give you a shopping example. Let's say you wanted to build something that's good at recommending based on T-shirts that you buy other types of clothes that you might wanna buy.
That might not be automatically well-served by a frontier model because it doesn't have that much training data. But we could fine-tune- A model specifically good at that, and you have to have evals, that meaning a golden data set of "Hey, when somebody looks for this T-shirt, we tend to also look for those types of T-shirts."
And some of it can come from our data, and some of it can come from training data. Then what happens is we might use that evals either in a distillation or another fine-tuning approach to figure out, hey, we've now come up with a way and evals to feel like this... If it can satisfy the evals to this level, we think that it's worth actually shipping to production, right?
Using that kind of pipeline, it allows us to then figure out, hey, this is past this threshold 70%. We have different evaluation. I'm just gonna use percentages. 70% of the recommendations are good. That could be something that we want to deploy. The more interesting thing is, let's say a new frontier model comes out or a new open source model or new data.
Having the pipeline that goes through this eval framework allows us to quickly iterate through this set to come back and say, "Hey, we found the new Qwen model is actually better," or, "We found the new this new data set is better. We've got new training data. We've now updated our evals." Because if you don't have that, you're doing things one-off, but it gives us a very good way to judge the LLMs and even just classical machine learning on the things that are going to production and whether they're worth they're improved or not.
Sam Witteveen: Do you foresee a time where you'll be doing that for your customers, like for entrepreneurs at, with actual specific shops where they're like, "Hey, I want a model for my shop that rec- does perhaps like a rec system or something like that," where it's very specific to their particular kind of shop and their particular kind of use case?
Farhan Thawar: Yeah. So we have some- we have something called Sim Gym that we launched, which is like simulation gym, and what it does is it actually helps merchants who... So in the world of A/B testing, if you have lots of traffic, it's easy to A/B test, right? Because you can just be like, "Okay, 1% of the traffic's gonna see this thing, 99 sees that thing.
I have a control." We can figure those things out. If you're a small merchant, you don't have this traffic, so doing an A/B test when you only have 10 orders a week or 100 orders a week is really hard because you don't have enough traffic. So Sim Gym uses all of the amazing data we have around buyer behavior for your store- Actually simulate a change that you make in your on your store.
So you might say, "I'm gonna change the size of this add to cart button, and I wanna make sure that it's something that is gonna increase conversion," or, "I think that I wanna test this different theme, 'cause I think having a summer theme is gonna do work better than having a fall theme."
And so now Sim Gym will simulate buyers going through your store and give you a readout as if it was an A/B test such that you could say, "Wow, this did perform better," or, "Oh, no, it didn't perform better. I don't wanna launch it." And that's literally what we're doing inside of Shopify today because it lets you do that.
The other thing we're doing is we have something called Pulse, which helps you really think through from an async perspective. It'll just look at your store, and it'll say things like, "Hey, did you notice that you're making these changes and it's slowly been increasing the latency of your site? I'm not sure if you noticed, but we know that latency is a conversion killer, and you should always be as fast as possible."
And so we give you these async insights into your store in the background based on all of the amazing data we have about buyer behavior and themes and conversion and performance, et cetera. And so we are doing a lot of these things, and they are personalized directly for the store.
Sam Witteveen: What are the trade-offs of the distillation?
I- is this something that's costing you more in, now you need big G- racks of GPUs for doing inf- You're, you're really taking on the whole inference thing yourself by doing this. What are the challenges that cause?
Farhan Thawar: The one challenge is, just calling the frontier model for everything is easy, right?
Just call the API. Of course, we have the evals and everything else, but you don't have to do anything. You just do it there. The other downside is anytime you make a big change, like I mentioned, like I wanna find, great movies, now I wanna find great songs. If you make a change, you do have to retrain these models.
You brought up a good point, which is on infra. You do ha- we have a GPU cloud with our own open source models running, and we use the GPUs to train and run these distillations, and then also we have the GPUs to serve the inference for the resulting models. And so yeah, you have to build out a little bit of a, of the infrastructure to then train and serve these models as well, which by the way- Some companies don't want to have.
Maybe they just have inference in the cloud, they don't have GPUs directly or from anywhere, and they're, like, thinking, "Do I want to worry about this?" In which case they they may not do it at all. So but for us, at our scale, we do have we do have access to that type of infra-
Sam Witteveen: What sort of split do you see at the moment between in production use of the frontier LLMs versus, distilled models?
I imagine for development and stuff like that internally, it's very much the frontier stuff. But for in production stuff where you're doing stuff at scale, can you talk a little bit about the split, even if you can't say exact figures, like-
Farhan Thawar: Yeah. I don't even know if I know exact figures, 'cause it's not something I really track that way.
But you're right, so dev tokens for sure at this point cannot be distilled. It'll be interesting to see as more and more, open source or local dev models be- appear to see if they can compete on the coding models against the frontier models. 'Cause of course all of us would love to see, the cost of that come down, but also having it run locally means you could run it on your laptop, when you're on a plane, like, all those kinds of things.
As we find narrow use cases, we do then try to distill them, because again, it wasn't just about cost and latency, which are big, it was about accuracy. In some cases we're able to get the accuracy to be higher than the generalized model. So for us it's again win, win. You got to run the process.
My guess is we have, I'm gonna guess maybe half to a dozen of these distillations running in production, and we're just gonna see more and more over time as as we do a better job there.
Sam Witteveen: Are you guys keeping the traces from your, your developers with the aim of eventually being able to train your own coding model, your own...
w- what we've seen, Cursor do. A- and I imagine you've got the scale to actually be able to do it. That's the kind of interesting thing.
Farhan Thawar: I never thought it about in a way to train our own development model, 'cause I'm pretty sure every- everything that we're doing is it's different in a way.
Such... ag- if you take it all the way to its limit, right? If that were true, then in theory I could run everything in a local model, right? Because I could just train my own coding model. But I think coding is such a wide open space, and even just an innocuous one letter change of code somewhere could have rippling effects across the whole infrastructure, which is why we don't even allow our engineers to use smaller models for this.
So I think the answer would be that I don't think that's possible at this stage. It may be, again, maybe in the future we come up with techniques to have like frontier style models that are even just local, which allows me to then maybe take my internal development data. That would be interesting.
But instead, I'm more thinking about how do I get the most intelligence to my engineers such that they can be on the forefront of building as much as possible on the roadmap as fast as possible, versus trying to be like let me constrain them down to this like very cheap and fast model," but at the expense of like frontier intelligence.
Sam Witteveen: Shopify is one of the leaders with the UCP protocol. I think you guys were very key in developing it along with Google, if that's right? Yes. Yep. Yeah. Where do you see things going with agents? I- if we project out a year from now, am I literally just telling my agent, "Hey, I want a new set of hiking boots"?
I want 'em, for this kind of terrain, and I want 'em to be waterproof, and these are the brands I like. Go make it happen. And then it's got my credit card, and it goes off. How- wh- where do you see the balance of this-
Farhan Thawar: It's so funny because in 2025 when people would say 2026 is the year of agents, I would always eye roll 'cause I never understood what they meant.
What does an agent mean? And it wasn't until I installed OpenClaw in January that I'm like, 'Holy shit, this is like a whole new, like a whole new interaction model.' There's two ways to think about agents. Yeah, there's like the OpenClaw style, like personal agent, and then there's also like agentic commerce, which is like, where are buyers going, right?
So you mentioned one interaction model as a buyer. Maybe you're gonna message your agent, wherever that lives, in one of the LLMs or OpenClaw, 'I need new hiking boots.' Maybe you have to tell it, by the way, the brands you like, or maybe it already knows because it knows you, right? My OpenClaw knows me pretty well.
And so it's possible that the Claw has my my interests already embedded and says 'Hey, I know you like this brand, and I know your size already, and I know you're in in my case, in Toronto, so I'm gonna find things that are in that local area.' Versus maybe you go to ChatGPT or Claude or Gemini and you ask it, "Hey, I'm looking for new boots."
And by the way, those things have memory too, right? So they might be like, 'Hey, by the way, I know you live in Toronto. Let me tell you...' And so what we see here is that this thread is running through the whole conversation is we wanna build the infrastructure to enable anything to happen, right?
Meaning the reason we developed UCP was we are trying to get everything set up such that no matter which direction the buyer goes, the infrastructure is there for all of the amazing products to show up on these surfaces, right? So if you're in Claude and you ask, "Hey, I wanna buy a gift for my grade eight daughter because she's graduating from grade eight," you want amazing products to appear there, right?
From all over the internet, including, of course, the amazing Shopify catalog. And so we built the catalog at Shopify to enable anyone, including all the LLM providers, to have access to all of the amazing things that our entrepreneurs are building. So that's one stage, which is like it just shows up there, right?
And the old model was, oh, cool, here's you mentioned hiking boots. Maybe your daughter's into hiking. You press the button, and it just takes you to that website, right? That was one interaction model. And then with UCP, what we wanted to do is bring that closer to the experience inside the LLM.
And what UCP allows you to do is not only c- can do your transaction right inside of that surface, but bring the branding And the full checkout experience to that surface such that it looks and feels like the brand you're interacting with. So it's not just a generic buy it now button, 'cause guess what?
People care about sizes, and variants, and colors, and taxes, and duties, and loyalty, and, like, all the things that make that brand experience their brand. And you want that to appear all the way through inside of that surface because not only does it get increased conversion, because it does, but it's like, it feels like you're c- you're transacting with that brand.
And then of course, if you're like me, 'cause sometimes I still wanna be like what else you got in the store?" You can click on the store link and have the full experience there as well. Because we're building the infrastructure, we're enabling all sorts of commerce to happen, and it's up to see where...
It's up to u- us to, to watch where the buyers are gonna move to, if they're gonna go to your world, which is just tell my agent, "I need boots. Buy the same ones as last time or find me something cool." And by the way, we see that, people giving money to their agent and saying, "Up to this amount, go spend it."
Or if they're gonna do it inside of an LLM surface like a ChatGPT, a Claude, or a Gemini, or some other one, or they're gonna just do d- what I said, which is they're gonna research, and then they're gonna go into the store, or they're gonna click on the link and go to the website. We do see the early signals around a agentic commerce.
It is... The numbers are small, but the growth rate is huge.
Sam Witteveen: What other changes y- I guess, do you think that both developers, entrepreneurs, people involved in this area should be thinking about now that, agentic commerce is growing at such a large rate? It does seem in some ways that perhaps search is being replaced, SEO is being replaced.
Farhan Thawar: Yeah, I think a couple things. One, we always th- think about building infrastructure because we don't like to guess where we think people are going. We want to enable all of the different variations, right? We also think that commerce is multi-channel, right? You might see something on Instagram, you might walk by the store, you might...
Your agent might recommend something, your friend might tell you something. There's so many ways that you learn about commerce, and so we wanna make sure all of those are available to you. So I tell developers, "Hey building infra-" At this stage is probably the right thing 'cause you don't know where things are gonna go.
That's one. And then two, we wanna make sure that your products appear in those surfaces. What's amazing about Shopify is is that because we've built this catalog, once you're on Shopify and once you know the signals are in the positive direction, your products automatically will appear in those places because, we work with all those providers to make sure they do.
And that allows you to have again, like SEO, whatever it turns out to be, GEO, whatever they're calling it, really allows folks to be agnostic, right? If I'm an entrepreneur, I just wanna build amazing products. I don't wanna think in my mind, "Oh, wait a sec, I need to advertise on TikTok, Amazon maybe, eBay.
What about Instagram? Oh, wait a sec, what about ChatGPT? How do I get it in there?" You don't wanna think about that. The whole reason Shopify exists is so you don't. You just have to be like, "I'm gonna build great products," and you can just press buttons. There's literally a button inside Shopify that's like Agentic Commerce, like on/off, and then it just shows up, and then you don't have to worry about it.
Sam Witteveen: Do you see agents as becoming a key thing as a coworker and stuff like that? It really sounds like you're going down that path already with River, right? That sounds to me like it's basically perhaps not a personal agent, but a hive mind agent that helps everyone in the organization.
Farhan Thawar: River is an agent in that you can talk to it, but it doesn't talk to you, meaning it doesn't proactively message you. The thing that ... I would say the difference between the agentic definition when I think about an agent, and of course I just mentioned agentic commerce. That's also not yet messaging you.
When you look at the world and what the the really wealthy people have, and that's coming to the consumer is how you know, like, where innovation happens. Wealthy people have a buyer for their clothes, and they message them like, "Hey, remember you bought that jacket last week?
I found the perfect pair of jeans." They're messaging you, right? So right now we're not at the point where, your ChatGPT is messaging you, although they have the Daily Pulse and things like that. My OpenClaw does message me, by the way. The thing that, got my ar- the hair on my arms going up was when my, when I was traveling and my agent messaged me and said, "How's your trip?"
And I was like, "What the hell?" My Claw messaged me, and I was, like, really freaked out because one, how did you know I was on a trip? And of course it had my calendar, so then it figured it out. But just the fact that it was just checking in on me, I was like, "Okay, that's agentic." So when we get to the point where a River or you know, whether a Claw or something is messaging you and giving you "Hey, by the way, I know you like, Vessi shoes, and I don't know if but Vessi just released a brand-new color for this shoe and in your size is available and ships to you."
And then you're like, "Whoa, that's cool." And maybe in the future you go, "Hey, next time it does that just buy it." And then you'd be at home and all of a sudden Vessi shoes show up. That's ... Then I think we're in this future world where we start trusting the recommendations, the payment rails, the budgeting.
And it does change everything about the s- surface area. And so again, that's why we're building the infrastructure to enable it all. It's not too long. I think in we're talking two to three years we will have people giving budgets to their agents. We will have agents buying things on their own.
Things will just show up. I'm pretty close to it myself. I'm pretty close to just giving my bu- like, an agent a budget and be like, "Hey, 200 bucks a month, things that you think I need, like you've seen me buy." I had asked a question of my agent yesterday. I asked it something, and it said, "Because you have a Tesla..."
And I was like, Jesus, it it really... 'Cause I have a Tesla, it knows. And it said something about that and used that context to try to weave it together and say, "Oh, because you have a Tesla you should get this thing." And I was like, Jesus. Oh, you know what it was? It was a pool skimmer, and it said, "Because because you have a Tesla you like technology.
This pool skimmer is a Tesla-like pool skimmer." And I was like, okay. It kinda sold me on the product, and I did buy it.
Sam Witteveen: You are the founder of a very sort of famous or infamous WhatsApp group of CTOs and engineers in Silicon Valley of SaaS companies. Currently, Fable has been disabled for pretty much everyone, even though the intention was for, just people outside the US or people that were non-US citizens.
I'm curious, in that, that WhatsApp group what people were actually saying. I think you guys were already one of the Project Glasswing partners. What happens when that sort of intelligence is taken away if you're getting used to developing with that kind of thing?
Farhan Thawar: Yeah. So a couple things, right?
One, like the reason we have the LLM proxy is allows us to go across the frontier models and open source models, and it allows us to have access to the plethora of tools. When a model comes and then it goes, or it could be as innocuous as like an update, right? Hey OpenAI is not gonna let you use GPT 5.2 anymore, right?
Like any of those things really the proxy allows us to spray across the different providers. So yes, we, I... We saw Fable come out. Fable looks amazing. We used it of course. And then it went away and, the... Of course the workflows of people changed 'cause they're like, "Oh my God, I don't have access to it anymore," but they go back to Opus.
They go back to GPT 5.5. We go back to work. And so I think what I would tell companies is that you probably want to have some sort of solution that allows you to go across models so you're not super tied to anyone, only because who knows what's gonna happen in terms of like regulatory things or just being able to have a little bit of agnosticism or abstraction away from that so you can focus.
The other thing is, like I mentioned, like we are, we continue to like research and spend time in the open source models because we wanna see what's happening in that ecosystem as well. There are already people who are focused on understanding the frontier of open source way before they were worried about a regulatory or a company turning off access for some people or anything like that.
They were always like, "Hey, I believe in open source. I'm a very big fan that these are fast followers. I'm a fan of what they're offering, their price point, like how to host it," like all those things. And so those folks continue to, to see that being a a thing that they can follow for a long time.
And they don't even... Sometimes they don't even have to host it themselves, right? They use this through OpenRouter or something else. There's "I can use OpenRouter. I can pick a closed source model. I can pick a open source model." It's their LLM proxy, right? They don't have to think about having to host it themselves, and I don't think that has changed with what happened this week.
Like I, I get... I agree with you, it's unprecedented and a new thing, but I don't think it's changed the phil- I don't think it, it necessarily converted anyone's philosophy. What I think it did do though, is it did force people to be like, "Hey, do I only have one provider in my stack?" And I think if for those people who are like, "I only use one provider," they're probably like, "Wait a sec, what if something changes in a way such that this provider becomes inaccessible to me for whatever reason?"
I think every- you're right on that front. Everyone's probably thinking, what is my backup plan?" And by the way, we saw this on our own, in our own way two or three years ago, just on the availability side. You could have a Claude go down or an OpenAI go down. Like holy shit, I'm building products that go to my customers that rely on these systems.
How do I make sure that I can be available? And that's when the idea of having multiple providers in our proxy allowed us to spray across multiple backends so that now we don't have to think about it. So now I think a lot of companies will be thinking about being multi-provider now.
Sam Witteveen: Fascinating. On that note, Farhan, thank you very much. It's been a very interesting conversation. We've covered a bunch of things, and it's fascinating to hear what you're doing at Shopify. Thanks for joining us.
Farhan Thawar: Thanks for having me