The interview transcript below has been edited for length and clarity.
Matt Madrigal: Pinterest is a visual discovery platform. We have over 620 million monthly active users, and visual search is really the core of the experience. We've had a long history with open-source models, going back to BERT and the rollout of the Google paper back in 2017 and a lot of the recommender systems. Over the past year, we've invested quite a bit, really fine-tuning open source models, customizing them. We have this conversational AI model that we've recently rolled out called Navigator One, that's built on a Quen 3 VL.
Matt Marshall: This is VentureBeat’s Beyond the Pilot podcast. It's about enterprise AI in action. I'm Matt Marshall. Today's episode is presented by Outshift by Cisco, Cisco's emerging tech incubation engine and driver of agentic AI, quantum next gen infra and beyond.
Today we're talking with Matt Madrigal, CTO of Pinterest, a company running multimodal AI at scale across half a billion monthly users long before most organizations were figuring out their first pilot of AI.
Matt has led product and engineering teams at companies like eBay, Google Shopping, and Fanatics, and he's now at the center of a bet most enterprises are quietly trying to work out, whether open-source models fine-tuned on proprietary data can actually replace the closed models from frontier labs like OpenAI and Anthropic and Google for production AI workloads.
The argument is it's a lot cheaper if you could do it, and you have more control. So Pinterest says yes. Their numbers say 90% cheaper and 30% more accurate. So we're going to stress test this thesis today. Welcome to Beyond The Pilot, Matt.
Matt Madrigal: Matt, Sam, thanks for having me. Good to be here.
Matt Marshall: Your CEO has said publicly that open-source AI runs about 90% less expensive for Pinterest than proprietary frontier models do.
Can you walk us through what that number actually includes? Most people hear cheaper and it might be a specific component like inference or, fine-tuning costs.
Matt Madrigal: Pinterest is a visual discovery platform. We have over 620 million monthly active users, and visual search is really at the core of the experience where users wanna come to find inspiration and discovery.
And so we've had a long history with open-source models, going back to BERT and the rollout of the Google paper back in 2017 and a lot of the recommender systems. But Matt, most recently, what Bill's referring to is over the past year we've invested quite a bit in really fine-tuning open source models, customizing them with our own data and our specific use cases because of our scale, and specifically, we have this conversational AI model that we've recently rolled out called Navigator 1 that's built on Qwen 3 VL.
Sam Witteveen: You talked about your journey of open source and the fact that you're a sort of, image discovery and stuff like that. I'm guessing that means both text to image and also image to image things. How did you guys start with that? What sort of models did you start using for that and how has that progressed over the years?
Matt Madrigal: So I go back to some of the foundation of a lot of the recommender systems that we have and specifically this idea of shared embedding space specifically for image and text and then, what makes Pinterest special is- Not only be- because the way the human brain works, you can process images so much more quickly.
But that's only works on Pinterest if it's a relevant image. People come for discovery, and so by having that shared embedding space, we highly customize, CLIP that O- that OpenAI rolled out, back in, I think it was 2019. We have our version called Pin Clip, and that allows us to keep the context of all this incredible metadata about an image and then, at inference be able to embed or call a lot of the great user data that we have to deliver this really personalized experience.
Because Sam, what's interesting or what's critical about Pinterest is you can't just show we wanna show what is interesting to you maybe through a semantic search if you type in, let's say, black T-shirts or maybe like a home remodel inspiration. But we also wanna show you things that we believe that you will also want to discover laterally.
We call that lateral exploration. So that's really the concept of that because... And I call out this Pin Clip work because that is really what has allowed us to leverage Qwen3-VL in a way that's great for our users and allows us to do this at a much lower cost.
Sam Witteveen: Have you just developed that model yourself over the years and then gone to the LLMs added onto it?
Like you mentioned, Qwen3-VL there. How many models have you gone through on that path, and then how has that you to keep it all a lot cheaper than going with Google or one of the off-the-shelf hyperscalers?
Matt Madrigal: The big piece for us, just going back historically, Sam, is, even before CLIP came out back in 2019 a lot of the ways we think about this is, we do wanna now try to leverage open source because it allows us to go so much more quickly, especially with the foundational model there, where you can bring a lot of the uniqueness, through things like, our own embeddings, our own multimodal embeddings.
But we've also had to build some things, like we had this, before CLIP, we had this notion of this unified visual embeddings, which was our foundation that was, basically CLIP before CLIP came out, but specifically for Pinterest That, that allowed us to really understand the images, all the user data and the interactions there.
So we had to build that in-house right out of the gate. But obviously we've been able to evolve now with Clip and move much more quickly with more customized embeddings there. So that's one example. Now I think with these open source models, especially with these open Apache licenses where you can truly tweak a lot of these open weights and customize it for unique use cases, I think that's where, y- we've found just o- open source to be such a such a powerful model for us.
At our scale, it gets very costly to call y- a foundation model for anything that's really core to the user experience, again, for six hundred and nineteen million monthly active users, and that's why we've had to develop a lot of this technology foundationally in-house, and then we leverage more and more open source as that functionality gets better and better.
Matt Marshall: Matt, this is really helpful. I think there's a lot of listeners who are pr- practitioners such as yourself, leading their companies on their AI journey and trying to figure out, okay, to what extent can we actually lock in and bet on and dou- double down on the open source stack, right?
You hear Jensen from Nvidia talking about how important that is and how, Nvidia's pushing that as well. On the other hand, you're hearing that Anthropic and OpenAI are raising you know, hundreds of billions of dollars to keep subsidizing their models and winning mind share and grabbing market share artificially and offer- offering these APIs at a cost much lower than they otherwise would be.
I wanna br- I wanna bring it back to the lessons that our listeners can learn from you. And so one of the things I think you've established is, hey, Pinterest was in here building these kind of ol- older school recommendation, the old AI, right? Recommendation systems for years, you've been on top of this stuff.
But at the same time, you are using OpenAI and Gemini internally for things like sales and productivity. So can you maybe walk through, to, to what extent the open source stack works, and is it for your core? And where do you wanna actually use some of these closed source models?
Matt Madrigal: Matt, the way we look at AI usage across Pinterest, it's really three different dimensions from a very high level.
First is the Pinner and the user experience. Next is our advertising platform. And then third is for internal employee productivity, coding tools IDEs Codex, Windsurf Cursor, obviously Claude. And But I'll come back to those on the proprietary side. The way we use open source, comes back to this whole idea of core versus context, where If it's something that's gonna be so critical for our end users, that's gonna drive engagement, that will have to scale to, again, over 600 million monthly active users, we're gonna either probably build it or we're gonna leverage open source and customize the heck out of it.
That, that's really the mental model we have because we've proven we can get better quality, better recommendations, better, better user engagement. Because again, what Navigator 1 powers is Pinterest Assistant. W- what Pinterest Assistant is it's our shopping assistant that powers our, like really our conversational AI layer on Pinterest.
So it uses a voice modality input, but still visual first. And part of that conversational AI allows us to, a- again, use Qwen, but what we've been able to do is customize it pretty significantly in two ways. The first is we basically just ripped out the vision encoder layer and brought in our own multimodal embeddings.
And Sam touched upon this a little bit earlier, but why that's so important is what we're able to do is, i- is really not only capture the, all the metadata around like each of the pins and images, but this is really about performance, which drives price. And so what we'll do is, we'll precompute all these embeddings offline and then obviously constantly retrain with, new data, new pins that we get.
But inherently you get, you get the context of the metadata of that pin itself, of the image itself, and then at runtime and inference, it's just gonna perform so much better, as opposed to if we didn't do that, you're calling each image and basically have to encode every image that comes back at runtime and have to do that one at a time.
Just the scale wouldn't work and be a pretty bad user experience. Another example I'll give is just, some of the benchmarking we've done where y- before bringing our own embeddings, you're talking about a latency that's 20 times worse from an inference perspective. So th- these are some of the reasons why we've really focused on open source specifically for our user experience.
Sam Witteveen: Okay. So we've got Navigator 1, you touched on that. Th- this is in your sort of new model stack for voice. How did you make the decisions for that? So I'm guessing you've either gone for ASR, LLM in the middle, and then TTS, or you've gone for an omni model. So some- something... How do you make those decisions, and then how do you know when to swap them out?
It does seem to me that you guys have really s- done well with sticking with some, maybe not the latest and greatest, but it's the best because you've got the data and you've tuned it for your use case.
Matt Madrigal: You mentioned something, Sam, I think is important to highlight in this Navigator 1 context, which is the voice input.
You, you mentioned TTS and STT. We use OpenAI for that. I actually look at that as somewhat f- commoditized frankly, as far as the that piece. Obviously we always are mindful of latency because that's the piece with any kind of voice modality. But again, voice is just one entry point as we look at the future of Assistant because we're always vi-visual first.
Our journey specifically with Qwen and then even, some of the Hugging Face use cases we have around l- let's say search metadata or landing page optimization that we use, a-again, is it core for our user? If it is, either we're gonna build it or look for an open source option.
But I think more than ever it's about speed and speed to market, and that's really where we realized with this conversational AI functionality that this idea of a fit for purpose or like a smaller model that's gonna perform really well. Small is all relative, right? That's one piece. Now we're constantly benchmarking against other models.
We've evolved i- in this specific use case from, Qwen two five VL to now Qwen three. The main thing is it gonna be more cost effective or is it gonna be better for our user? If it's gonna be better for our users, what additional functionality can we get beyond, let's say, in our current use que- use case Qwen three v-VL.
And then we'll also benchmark against other third party models. We've looked at things in this use case for o4 1 Mini, models like that where really performance was, top of mind for us. But we settled on three VL right now because this combination of Qwen, again fine-tuned with, our own what we call pin clip embeddings, has just performed r- you know, really well.
Now w- a- as we add more feature functionality, we're gonna have to constantly benchmark and look at what our user outcome data is as far as like engagement, and then a lot of the infrastructure pieces around performance and, latency, et cetera.
Sam Witteveen: Can you walk through how you do that?
Like we saw this for example with LinkedIn, where they describe that, okay, they've got their gold set of evals, right? That they're running consistently on almost every model to see, okay, how do things benchmark. So do you do something like that and then do you slowly bring things into sort of a split test?
Things can go great with evals, but maybe they don't update well with users. W- how do you focus on that?
Matt Madrigal: Yeah. We actually do both. So we certainly have our gold set of evals that we leverage and, always looking at things around precision recall rates specifically at the user interest level.
Even at the pin interest or call it keyword interest models, we look at, things around certain personas for driving the personalization on the site. And then certainly we have really pushed the experimentation velocity up, and I think this is what's interesting now about we've got engineers, product folks, designers submitting PRs And so what that puts pressure on downstream is, how quickly can you get these tests in front of users?
And it's something that we, we're continuing to evaluate. But just going back to the evals piece we've got our golden set. We also look at things around product-level eval specifically for our taste graph powered experiences around interest targeting, a- and just different personalization tracks there.
So all that together allows us to see how's the current model performing? Is it, again, driving the outcomes we're looking for our users through, better engagement, saved pins, board creations, clicks to merchants and advertisers? So those are the outcomes that we're trying to drive for, but still long, and we've got a pretty awesome roadmap ahead even in the next next few weeks.
Matt Marshall: The recommender model's 30% more accurate than off-the-shelf alternatives, right? I think there's some thinking in the marketplace that if you're talking about open, you may be giving up something on quality, reliability data integrity, but you've got 30% more accuracy y- and may- maybe because of the fine-tuning.
How much of that gap closes over time as potentially some of these more fa- the- these foundational models improve?
Matt Madrigal: What allows us to drive these gains from a user perspective, take the technology out of it for a second. If you've got really unique data that you can then fine-tune an open source model with, basically data quality will then, frankly, outweigh or overcome, call it model size i- in this specific use case.
And I think that's the opportunity where if there's other use cases like Pinterest, you don't have to have the, hundreds of billions of parameter frontier model. A- and that's where I think some of these fit for purpose models that specifically that can be used as a foundation like open source, like the Qwen models, like what in Hugging Face, could be actually something that we that could be really useful and frankly drive a better consumer experience for different companies.
Sam Witteveen: In many ways your thinking sounds much more like the Chinese companies in that they have just focused on the user, on the product. Then, whereas perhaps here in San Francisco, a lot of people are talking about agents and AGI and this and that. Whereas you guys are really just locking in on, okay, which models give us the best result for, almost bang for the buck, as you could say.
Matt Madrigal: We always think that stuff is really cool too, Sam, like with Assistant and what you can do with agentic shopping, and that's really- ... the use case that we have right now through Navigator 1 and just going back to the user where, why do people come to Pinterest? Half of our users come to us to shop.
And just looking through that lens, it really drives a lot of the, frankly the tech innovation that, that motivates a lot of our, our engineers and our product folks in the company today. Obviously, we have really good relationships with the, the frontier model companies, we partner with them closely.
You have a, like the menu of things we can leverage, whether it's like a proprietary model and in our IDE adoption of gen AI, man, we got the whole if you're a Java person, you want Windsurf, we got it, Claude, Codex, y- and just allow our engineers to be, s- highly productive.
And those are things that, credit to all the proprietary model companies are doing an amazing job there. But w- we try to tune our technology choices based on the outcomes and, like you said, the what's best for the consumer.
Sam Witteveen: So okay, as Navigator 1 starts to take off, do you get to a point where you suddenly go, "Hang on a minute.
We now know that there are these, speech-to-to-text models. There are these TTS models. We could actually have a go at doing this ourselves and we can then put in whatever sort of image models in you want into it." Is that like the plan?
Matt Madrigal: That's certainly been the evolution for Navigator 1, Sam.
Even before we went to open source, we were using like o4-mini just to start, right? Because it was something that easily accessible. It performed it, it got to the level of performance that, that we thought we needed. And then we fact- quickly realized there's a lot more work we needed to do specifically on, again, like the visual similarity the recommendations that, that we needed to really drive a great user experience, and that's when we pivoted pretty quickly to to open source.
So a lot of times the proprietary models will give us that ability to test and move more quickly from the outset. And then again, just going back to something that, that's core versus context. If it's something that's gonna drive usage across more than 600 million monthly active users, the- then the question is the whole cost-benefit analysis of if it's gonna be great for our user, that's one thing but frankly, is it gonna be cost effective for us to scale?
Matt Marshall: Want to move on to the taste graph, and you're developing a g- a graph about me personally. You're able to personalize and for your advertisers able to reach me, right? There's a lot of information. You've had this in the marketplace. Is that receding in importance with gen AI taking over as a personalization tool, which is h- which is widely recognized as a s- a significant personalization tool?
Is it adding to what LLMs provide, right? And b- providing even more differentiation. Walk us through what the taste graph means for you.
Matt Madrigal: You go to Google if you know exactly what you want But you come to Pinterest when you don't exact- l- like when you're still, your intent is still forming, right? And you're looking for that discovery, and so very simply the way you should look at the Taste Graph is like it's our understanding of what people actually like, not just what they click on.
So for example, it's this representation of billions of people's evolving tastes. So specifically, like if you look at our biggest categories around beauty, fashion, home furnishings and really like how all of these tastes, relate to each other. It's not a social graph. It's much more of like a preference graph that like what's gonna inspire you?
What are you trying to do next? Again, it is the differentiator for the experience on the platform today.
Matt Marshall: You're post-training on the Taste Graph. So concretely, what does that change about how the model behaves?
Matt Madrigal: Best way to think about the Taste Graph again is just think of this as really like a large scale knowledge graph of people's interests their content all the visual content and the video on the platform today, and then specifically, like intent.
Specifically, you have user embeddings that, that capture a person's like evolved tastes. Sam could be a mid-century modern guy. Matt, you could be a Nantucket aesthetic there. So in the user embeddings, we'll capture that. And then as far as the actual content that you see within the Taste Graph itself, these are all things around like specific products.
Again, half of our users come in to shop. There are ideas and then creators who really have that deep semantic understanding of something like like mid-century modern that maybe Sam likes, which would then match to, let's say, a Scandinavian-type sofa. If you go from upper funnel inspiration discovery all the way through lower funnel intent.
And so I can go from seeing this, my ideal home office redesign, and for me it's gonna be like I, I wanna see like a giant TV or whatever, 80-inch TV in the background. And then I could pivot from there to say I really like this aesthetic. I'm gonna apply this to my, let's say my living room." Under the hood it's this mix of a graph structure and really representation learning that we're, constantly updating embeddings for the people and the content and the new signals that we s- constantly are bringing in.
Sam Witteveen: You kinda got representation learning on two sides, right? You've got the content side, but you've also got the user side, where we've got representations of those. Have you guys looked at doing things like reinforcement learning and, all these sort of new things? And one of the reasons why I ask this, and maybe you can talk to this as well, is that you guys have got the data, right?
I don't know how many images and stuff you have, but I'm guessing it's well into the billions. You've got the data, you've got the 600 million users. You're really in an interesting sorta spot that a lot of c- other companies just don't have that, that data moat to be able to experiment with these things.
Matt Madrigal: I think you just touched upon what makes the Taste Graph so unique, Sam. And because, look, big picture w- we think about all these associations across literally hundreds of billions of pins organized into 15 billion different boards, and then all the associations around whether it's a text query things around like semantic relevance, for example, or the visual searches themselves.
Your taste is visual, right? So I can't tell you, it's very hard to describe through text, even through a natural language conversation with a ChatGPT or Gemini to say "Hey, d- describe my style. I'm I live in the Bay Area. I work in tech." Whereas if, as a user, what's captured in the Taste Graph is if you see exactly what you're looking for you just know it.
And I think one of the biggest compliments we get, and this is powered by the Taste Graph and some of the multimodal signals that we get, is that, we hear from our users like, Pinterest just gets me." A- and I think that's the experience that we wanna deliver where as people save pins as they create boards, like I'll use, for example, boards as wish lists, for shopping for my wife and my kids.
These are the things that, that help with some of the unique context and then shopping behavior, which is great not just for our users- but for our advertisers themselves too. Being able to try to anticipate what a user wants from a personalized experience perspective, but then what are they actually gonna want next?
The example I'll give is, springtime coming, I'm looking for a fire pit, right? I'm looking for a fire pit in my backyard. Maybe it's a DIY or maybe I buy it from one of our great retail advertisers. Once I buy that, what do you start to see in the, in your home feed? You'll start seeing things around s'mores recipes or backyard parties or th- things like that really try to anticipate.
And these are some of the things that our retail partners are really looking for, again, that full funnel from inspiration to performance.
Sam Witteveen: You've gotta be in some kinda adversarial war with Gen AI on the AI slop front, right? Like you, you are curating all these images. You're basically getting really good representations of them and the users.
Everyone else though at the moment is using Gen AI to make images, which I imagine is probably not what your users necessarily wanna see if they're looking for something that they can buy and all they're seeing is sort of stuff that's made up. W- what's the c- what's the situation with the sort of AI slop war?
Matt Madrigal: I think it's always important to think about this in the broader context of content quality as it relates to Pinterest platform here, which is, we don't look at Gen AI content as necessarily are inherently bad. We really look at it in the lens of we wanna deliver the most inspirational and personalized experience for our users.
Now, at the same time, if the Gen AI is irrelevant, if it's spammy, et cetera, we're gonna down rank it and take it off the platform. We're really focused on two things beyond just content quality, which is transparency with Gen AI labels as well as user controls. Those are the two things that we're focused on there.
Sam Witteveen: So you allow people to say, "Yes, I wanna see Gen AI," versus, "No, I don't"? That is correct. So then how... So that alone must be a big sort of detection issue, right? Like- Yeah ... it's nice that Google's basically putting a watermark in every image that comes out of NanoBanana, but I don't think a lot of the other models necessarily do that.
Matt Madrigal: You nailed the technical problem there, Sam, too. Look, one thing we've invested a lot of engineering resource on, something we take pride in, which is we create our own vision models where we've got like a visual classifier that, that looks for frankly like Gen AI perceived content. And specifically, we use some of the scoring there for things like content distribution.
If you look at your home feed or your search results how are we distributing the most relevant content? How much gen AI, for example, i- is on your feed that, that's viewed as high quality? That's one. And then also, like on some of the classifiers where it's trained on how people judge images, and so not just how they're created.
And so that's a key piece there, 'cause now we're able to label four times more AI images than the prior model. And so what we've done is that model's calibrated against user judgment and feedback. And, based on how Pinners, our users, perceive an image, those labels continue to get smarter, where we're c- constantly, retrained on Pinner feedback, where we ask, users if they wanna see, for example, less gen AI or, we have an appeals process if somebody says that this is actually either is gen AI or not gen AI.
So we have that continuous loop. Our users are able to go into their preferences and determine within their diff- different categories like beauty, home, fashion, et cetera, and toggle on or off if they wanna see gen AI content.
Sam Witteveen: So you talked about things like Claude Code and all the things that are just accelerating the velocity, right?
This is something we're hearing from everyone. Everyone is saying that there was this inflection point at sort of November, December last year, and then it's really taken off this year. Both what's the good and bad of that for someone in your position? I imagine yeah, okay, it's nice to have people lighting up, writing a lot of code quite quickly, but I imagine that's also a lot of headaches when you've got someone who's not a...
who's maybe a product manager who doesn't normally do pull requests or something suddenly submitting 20 pull requests a day.
Matt Madrigal: This just goes back to the old days of how many lines of code... L- like these are all like bogus Input metrics, what I always look for are things around experimentation velocity, impact to our users or advertisers with, better, more features, et cetera.
And absolutely, you see the cause and effect with a lot of these incredible IDEs that have been that have been rolled out. And so all that does, though, is it brings a lot of the pressure even further downstream as far as you have all this great input now. How are we testing these? Like, how are we compartmentalizing the test license, et cetera?
But even before we ship, Sam, there's questions around really well defining your sandbox environments where you enable a lot of these coding agents to, drive incredible productivity and then have that right orchestration layer if you're a senior engineer. What kind of access do the agents have outside of your sandbox and your secure environment?
So these are things. It's been fascinating. I think that on the software or the feature side for users, but I don't know if people talk enough about the culture impact, which is all of us are rewriting all of our job specs, right? What does it take to be a software engineer versus a designer versus a PN?
These skills are completely different now, and I think that's actually super-- That, that's the really exciting time. But it is managing a lot of change as far as incredible velocity. I'm always looking at it as token usage. I'm like, oh my goodness, you get not linear growth. It's like exponential growth.
And so then you start thinking about, on the one hand, you wanna give your employees all the tools to be incredibly effective, et cetera. But then you just gotta manage it to the outcomes, right? Which is, okay almost having a traffic router. It's do you really need for example, Opus 4.7 sort of reasoning model for, a more simple task that you could use, maybe Sonnet or something else, right?
So I think these are all the things where, impact to cost ratio. My hairline's the way it is for a reason for things like this, but
Sam Witteveen: I take it you're not giving the, taking Jensen's advice to give all your engineers $250,000 a year of tokens at the moment?
Matt Madrigal: I was doing the math when he said that in the OPEX, and wow, okay.
Go Jensen.
Sam Witteveen: How do you decide what tools? From what I see you've got the hardcore sort of Claude Code people, then you've got the people who love Cursor, and then you've got people who love Windsurf or Devon or whatever it is. A-as a CTO, this is quite a big decision to say okay, we're gonna...
These tools are allowed, and these tools are not," or how do you do that?
Matt Madrigal: Yeah, the approach we've taken i-is really like a default yes approach. Obviously, we have to balance that with, security compliance and all the big pieces of just having a massive enterprise code base and, the distributed systems we support at Pinterest.
We really allow, for example our engineers to, to choose the different tools they like. And I actually think it's beneficial where Y- you're obviously not also locked into, one versus the other. We've got expedited review processes for new tools we want, and then we have both domain-specific tools with each of our functional units, and then more enterprise-wide tools, whether it's like Gemini or different productivity tools that way.
Matt Marshall: It's admirable, right? You're allowing your developers to have, access to all the best in breed. Does potentially, le- you know, the distribution in this day and age can be a liability. Each additional AI layer you have, o- opens the attack surface. You have these, this need for security compliance on this massive code base that has, potentially all these now avenues to be exploited.
We talk about re- recursive improvement by Anthropic on the that's good for the developers, but it's bad for the security group, and I'm just wondering, with Mythos being released and, or actually not released 'cause it was so dangerous, and the White House es- e- essentially having meetings and es- essentially freaking out about what the implications are, and OpenAI saying, "Oops, we're gonna stop our latest model too.
These things are too powerful." And Pinterest has this visual content, has, potentially har- you know, the harmful stuff that happens, the body image issues, self-harm issues, let alone the vulnerabilities in your stack. Just curious how you see the security side to that.
Matt Madrigal: We've got multiple sandbox environments. Y- and specifically around maybe for your more traditional application developer that doesn't need access to the taste graph, right? And then in that sandbox environment, you'll have more access to, let's say, external systems. Whereas having a different sandbox maybe for, f- specifically for our ML engineers that have access and do a lot of the model training with, with the taste graph, and obviously that sandbox is not gonna have as much access externally outside the environment.
So these are things that how we think about just trying to find this balance of both velocity, productivity, and then safety. A lot of this just comes back to this whole brilliant the basics, which is like how awesome are your CICD pipelines, what, as it relates to patching, right? It sounds so basic, and so we've actually invested quite a bit just as far as deployment across our fleet around just being able to deploy patches much more quickly because I think as you engineer and build new products, it just, security has to be built from the get-go.
It sounds cliché, but it's true. Better, controls. Those are all things that we have to all be pushing to work together.
Matt Marshall: Matt, thank you so much for walking us through that journey you have. It's quite extensive. Very impressive what you've built there and the results you've been able to share.
Thank you very much for joining us.
Matt Madrigal: I appreciate your time, guys.
Sam Witteveen: Thanks, Matt. This series is brought to you by OutShift, Cisco's incubation engine. By creating an open interoperable infrastructure, OutShift is enabling agents and humans to share intent, context, and reasoning. The cognitive evolution for agents is here, explore the internet of cognition at outshift.com. For more stories about the AI revolution, like and subscribe to the podcast, and check out venturebeat.com to sign up for our newsletters.