🎙️Anneka Gupta: Designing for Uncertainty

Is security still about patching after the crash? Or do we need to rethink everything when AI can cause failures on its own?

Anneka Gupta, Chief Product Officer at Rubrik, argues we're now living in the world before the crash – where autonomous systems can create their own failures.

In this episode of Inference, we explore: 

• Why AI agents are "the human problem on steroids" 

• The three pillars of AI resilience: visibility, governance, and reversibility 

• How to log everything an agent does (and why that's harder than it sounds) 

• The mental shift from deterministic code to outcome-driven experimentation 

• Why most large enterprises are stuck in AI prototyping (70-90% never reach production)

• The tension between letting agents act and keeping them safe 

• What an "undo button" for AGI would actually look like 

• How AGI will accelerate the cat-and-mouse game between attackers and defenders

We also discuss why teleportation beats all other sci-fi tech, why Asimov's philosophical approach to robots shaped her thinking, and how the fastest path to AI intuition is just... using it every day.

This is a conversation about designing for uncertainty, building guardrails without paralyzing innovation, and what security means when the system can outsmart its own rules. It’s packed with insights and practical tips that works. Watch it now →

This transcript is edited – ta-dam here is a big change – by Genspark Super Agent. It is very decent, but as always – please watch the full video and leave your feedback. It helps us grow on YouTube and bring you more insights from the builders) ⬇️

Ksenia Se: Hello everyone. Welcome to Inference by Turing Post. Joining me today is Anneka Gupta, Chief Product Officer at Rubrik. Her team is rethinking what security means in the age of autonomous agentic systems. Hi, Anneka, and welcome.

Anneka Gupta: Thank you. Thanks for having me.

The World Before the Crash

Ksenia: When I was thinking about our conversation today, I wanted to set a theme: designing for uncertainty. So let me form it into a question. We used to think of security as something that happens after a crash, right? Restore, patch, move on. Now we're in a different situation – we're in the world before a crash, where AI systems can cause one on their own. In this world, what does failure even mean when the system can create it by itself?

Anneka: I'm not sure there's anything that creates more uncertainty than AI today. There's uncertainty about what the future of AI looks like from a cybersecurity perspective, questions about how AI is already being used by cyber attackers, and what that means. But when we think about organizations and their adoption of AI, the fundamental challenge is that AI is non-deterministic.

The challenge with AI agents specifically is that they're programmed to execute an outcome without regard to the changes they'll actually make to achieve that outcome. So when we talk about negative consequences and risks we need to think about proactively as we deploy AI agents at scale, one key question is: How do I make sure my AI agents don't create unintended downtime for my applications?

We were already worried about humans creating unintended downtime. AI agents put that problem on steroids. There are going to be many more of them, taking actions much faster, in non-deterministic and potentially unpredictable ways. That can lead to application downtime if they do something like drop a database table or make changes that are really difficult to reverse.

That's the unpredictability that's amplified with AI agents – taking the human problem and putting it on steroids.

Three Pillars: Visibility, Governance, Reversibility

Ksenia: I want to understand your thinking process when you approach this uncertainty and these failures. How do you define control, error, and recovery in this new era?

Anneka: When we're talking about solving these problems, the first step is visibility. How can you monitor and get visibility into what's happening? What AI agents do you have running? What actions are they taking? What applications do they have access to? That visibility is step one.

The next step is governance – how do you govern what these AI agents are doing? What are the guardrails you're putting on them to control both the actions they can take and what they can access, as well as guardrails around the outcomes and quality of those outcomes?

The third piece is reversibility. With anything, you have to anticipate that something will go wrong. You're not going to set the guardrails perfectly and anticipate every situation ahead of time. So what do you do when an AI agent inevitably makes a mistake, and how do you reverse those mistakes?

That's the thinking we're seeing from our customers, and our own thinking is evolving around what you really need to solve for – the jobs to be done – to have more resilience for your organization while accelerating AI adoption.

Five Rules for Deploying Agents

Ksenia: Let me make this really practical for our readers. Can you give a simple set of rules – three or five – that every organization needs to think about when they deploy agents?

Anneka:

Rule #1: Define the outcome. First, figure out what you're trying to solve with your agents. What's the use case? What's the ROI you're trying to drive? What's the productivity gain?

Rule #2: Choose your tech stack. How are you going to set up that agent? What tools and infrastructure will you use? There are many options, and it depends on whether you have an engineer building an agent or a non-technical user. Understand how that fits into your organization's overall technology strategy.

Rule #3: Build observability. This is where proactive work begins to make sure your AI agents are set up properly. How are you going to build visibility into what AI agents are running, what they're doing in the system, what they have access to, and who within your organization is using them? That monitoring piece is critical.

Rule #4: Manage and update continuously. What are you going to do to manage your agents on an ongoing basis? How will you look at what they're actually doing and create new rules? You'll have initial rules around sanctioned and unsanctioned behaviors, but you need to look at those continuously and update them based on what you're seeing. This way you improve how your AI agents run and prevent unwanted actions in the future.

Rule #5: Prepare for recovery. What's your technology playbook and process when your AI agents inevitably make a mistake? How are you going to recover from that?

The Logging Challenge

Ksenia: You talk a lot about observability, visibility, and interpretability. In machine learning, interpretability was one of the hardest problems. Now with this uncertainty, what are the main bottlenecks? How do you deal with them?

Anneka: There's an even greater amount of unpredictability than in the traditional machine learning world with AI agents. Very practically, what you can do is log every single thing your AI agent is doing.

You're not going to be perfect on day one with your rules and setup. But if you're logging everything, you can go back and assess for quality challenges – is this producing the results I think it should? You can look at the security posture and overall posture of your AI agents and determine whether these are activities you want to allow.

At the end of the day, building these guardrails on a continuous basis – what's allowed and what's not – is critical. That's important both for quality and getting the ROI you're looking for, and for monitoring in case your AI agents do something malicious because an attacker gets into your organization and uses them, or if there are unintended consequences that increase your data exposure risk or other security risks.

Ksenia: Is it some sort of additional agent that does this logging? How do you deal with that?

Anneka: Think about what's happening: there's a call between the agent and the model itself. You're making a call to a model, the model comes back with next steps, then that agent typically hits some kind of MCP server that takes actions. Understanding what's actually happening behind the scenes is super important.

This can happen in cycles – you could have multiple rounds, agents talking to agents. But if you know all those model calls that are happening – both the input prompt and the result coming back – as well as what's happening with the MCP server when a specific call is made, and you can string all of that together, that gives you a solid picture of what's happening.

Now, this is easier said than done. Often this isn't one system – it's multiple systems. Your MCP server might be connected to your email, your Slack, your CRM. All these systems are doing various things behind the scenes.

You have to create a holistic view that understands what's happening at every layer. That requires multiple entry points for monitoring, multiple calls to different APIs that have that logging. Then you need to construct from all these various logs what actually happened in sequence – build that timeline.

It's not an easy problem to solve. Over time, third-party tools will continue to evolve to provide that visibility. Then it's incumbent upon vendors to string that together in a way that's actually actionable for business or technical users. We can't expect people to wade through 50 different logs and try to correlate these things on their own.

That's where we've failed quite a bit in security historically – too many signals that you can't correlate together into an attack timeline. It's similar with agents. You need to understand the agent action timeline.

Ksenia: But you can build an agent for that.

Anneka: You can build an agent for that, but you have to get access to the underlying data. I totally believe that building the timeline is easier today than ever before because of agentic AI, but getting access to the data and being able to interpret the logs well – there's still work that needs to be done. It's not like if you just put an out-of-the-box agent on top of logs, it's going to magically give you an answer. All of this is iterative and takes a fundamental understanding of the underlying data to pull that together.

The Mental Shift Required

Ksenia: Your experience is very varied, from the SaaS era to now the autonomous era. What mental shift does it require from engineers and product teams to think about and solve these new AI-related questions?

Anneka: I love this question, because the biggest barrier is often the mental shift that everyone needs to make in this era. AI is technology, but it operates very differently.

We're very used to writing code that's deterministic – we know what's going to happen. Or we program automation into our systems where these five steps will happen every single time, very clearly.

With AI, it's about defining the outcome you're looking for and determining your evals. Then it's about speed of experimentation, because no one can tell you just by looking at something that adding this data source will improve quality, or that connecting these tools together will change the outcome you can drive. You just have to try things and experiment quickly.

The faster you can experiment – within having clearly defined outcomes and clearly defined evals – that's what makes or breaks your ability to get more use cases out there and deliver them with high enough quality and efficacy.

That is so, so different than how so many of us operate day to day. It's especially different when we think about building technology – building technology first with an outcome and evals, versus thinking about technology as a set of steps, workflows, or tools.

Ksenia: How do you even approach root-cause analysis when it's so probabilistic?

Anneka: It's hard. I don't think anyone has a perfect answer for this. Root cause analysis – what are you even doing the root cause on? There aren't traditional bugs. It's some output coming out, and is that a bad output or a good output?

You look at it as a human, and if you're a knowledgeable expert, you can say yes, this is good or bad. But how do you explain that in a much more detailed way? It's very difficult to do. It's a really tough problem.

If it were easy, we'd all be seeing massive AI adoption, especially in larger enterprises where things are much more complex. Right now, all of these things are still being figured out. We're trying to learn, and no one has a perfect answer.

No one has perfect clarity around: I have this use case – is this something truly solved that I can solve well today, or do I need to wait for the technology to advance? And of course, technology and model advancements are happening so fast that something that doesn't work today could work great six months from now. It's just hard to predict.

Ksenia Se: How do you change this mental shift with your team?

Anneka: I tell my team to just experiment all the time. And I try to practice that visibly as well. Even in my personal life – every single thing, can I try AI first?

Some of the time, for some use cases, it doesn't work well at all. For other use cases, it works great. What I've found is that this changes month to month. It might change based on the model I'm using as well.

Taking that mindset both in your personal life – where it's easier because you can use more tools – and in your work environment where you might be limited in the set of tools, just try things. Think about: I have a problem I need to solve. Is there a way AI could help me? Let me try it and see where I land.

It's that experimentation, that hands-on usage every day, that gives you the intuition about what works and what doesn't. Intuition is very important in developing products and in developing AI agents to solve problems within your organization or in your product portfolio for customers. But that intuition can't be built without very hands-on experience – just building the reps and doing that.

The Enterprise Bottleneck

Ksenia: The majority of times I hear from people that they're afraid of AI is when they've never used it. Building this intuition helps understand and think about security as well. But there's a tension – if you do it personally, it's much easier and you can control many things. If you do it for an enterprise or company, there's always tension between letting agents act and keeping them safe. How do you think about design boundaries that don't paralyze innovation and don't stop companies from experimenting?

Anneka: This is a great question. Most large enterprises have some kind of AI governance committee – maybe it's not called governance, but they play a governance role in addition to hopefully helping with acceleration of AI adoption.

One of the bottlenecks that often comes up is: I'm thinking about this AI use case. It might be an external vendor I want to bring in, or something I want to build using agent builders internally, or a combination of both. A lot of the bottleneck occurs because there's huge concern around: What are these AI agents going to do? What do these technologies and AI agents have access to? What could they potentially do with this data?

That could be things like: Are they going to insert vulnerabilities into my code that I need to be thoughtful of? Anything that might be a real concern to the security posture of my company or customer trust. It could be: I'm worried about unintended data exposure.

In large enterprises, you usually need real segmentation of data between customers. You don't want one customer seeing someone else's data. You start opening things up with AI agents and the boundaries get blurred. How do you create those guardrails?

Internally with employees: I might be sitting in the product organization – I shouldn't have access to HR data, especially for another organization. Those are the concerns that come up in these meetings when AI committees are evaluating risk.

Then there's the question about efficacy as well. A lot of AI projects get stuck in prototyping and never make it to production – somewhere between 70 and 90%, depending on whose data you look at. A lot of that is because the efficacy isn't there too.

So you have to have both efficacy and address the risk you're taking. If you're able to deliver efficacy with greater guardrails that protect around data access and governance – what do these AI agents have access to and what kind of actions can they take – then you can unlock faster innovation. But it's both things: How do you get that efficacy? And how do you have the governance guardrails that allow you to move faster?

Planning in Constant Evolution

Ksenia: How often do you change your company plans, depending on this uncertainty – uncertainty with company behavior and this super-fast race of models and development?

Anneka: For me in product development, where we're developing products for external customers, the plans around AI are changing constantly. We're anchored on the problem at Rubrik: How do we accelerate AI adoption for large enterprises, highly regulated enterprises often as well?

We know what the problem space is. The solution space – the possibilities of these solutions – is changing so fast because the technology is changing, because people haven't even centralized on a single stack yet for what their layers are going to look like for different kinds of AI agents.

We have to be really responsive to what we're seeing when we talk to our customers, when we put betas out there. What's working? What's not? How is the landscape changing?

The high-level strategy isn't changing every week because we know what problems we're trying to solve. But how we're thinking about solving those problems is changing quite a bit.

The best thing for every organization building AI products for external customers is: Can you dogfood your solutions first? We are doing that. You learn so much from just trying it. You may not be your ideal customer profile, but that's okay. You'll learn a lot by trying it yourself and seeing how it works in the way your company operates and for the personas within your organization that might be using these products.

Being really strong with your point of view on the problem, but flexible on the solution, then iterating quickly using real scenarios internally and externally with customers – that's the best way to continue evolving with so much uncertainty in this space.

Ksenia: Can you name the list of problems you're solving?

Anneka: At Rubrik, we have a product we launched called Rubrik Agent Cloud. The problems we're trying to solve are:

• How do we help large enterprises get visibility into all the agents running in their environment?

• Help them understand what these agents have access to and what they're actually accessing and doing within various systems

• Enable them to block unintended or unwanted actions through predefined rules, then evolve those on an ongoing basis

• Help organizations when an AI agent makes an unintended or unwanted change in any of their environment's data or applications – being able to quickly reverse that change

That's the set of problems we think are really important to solve to accelerate adoption of AI within large enterprises.

AGI and the Undo Button

Ksenia: I usually ask builders what they think about AGI. For you, I have a double question: How do you feel about AGI, and related to what you were just saying – what would a real undo button, a reverse button for AGI, look like?

Anneka: What do I feel about AGI? My sense is it's an inevitability. It's a matter of when, not if it's going to happen. Many things are going to change.

From a security landscape perspective at Rubrik, AGI is going to be yet another tool that attackers use. Already, attackers use AI to have highly personalized and realistic social engineering that's leading to more compromised identities. Certainly AGI will help that. But I think it's going to completely change how quickly attackers are able to compromise an organization and their data once they break in.

All of this means the timeline for what happens, how quickly people get breached, and the impact of that is much higher. It kind of goes back to: even whether it's a cyber attack or unwanted AI agent actions, how do we think about reversibility?

Reversibility – AGI only makes the problem more acute. The solution to the problem is probably not that different, which is: I talked earlier about understanding the whole timeline of what an agent actually did and having those guardrails in place.

Yes, you're going to miss some stuff in the guardrails. But if you understand them, you have the guardrails, you can look at all the actions and rank them by their risk. AGI will help with that – making the understanding of that risk a lot better, with fewer false positives and false negatives, really elevating the things that are the highest risk actions.

Then it's about: How do you have the integrations into all the different technologies that allow you to quickly understand what changed and go into that system and reverse that change? That technology is our bread and butter – what we do every day at Rubrik. Helping people recover from cyber events, operational failures, and now unwanted AI agents.

It's definitely not an easy problem to solve because AI agents can be touching hundreds of different systems and devices, maybe even thousands. Being able to monitor and track all of those, pick out the signal, and then actually do the reversal really quickly – it's a really exciting and challenging opportunity.

Ksenia: Do you think AGI systems will act on their own for whatever decision, or will it still be hackers deciding?

Anneka: That's a great question. I don't know the answer. I think some version of AGI will be used by humans giving a very broad outcome. Whereas today with AI agents, you have to give a very specific outcome. With AGI, you could give a very broad outcome.

From a national security perspective, you could have a foreign government trying to wage war on another country, giving very broad outcomes for what it's looking for, and AGI will just go figure out how to execute on those outcomes.

I think it's an accelerant of what's already happening. There'll probably be new vectors that AGI discovers – interesting vectors of attack. On the defense side as well, interesting vectors of things we can defend against, both proactively and reactively. It's just accelerating the cat-and-mouse game that already exists, from my perspective.

Will we have AGI where it's not even human-directed? Now you're starting to talk about total science fiction world, which is certainly possible. But then we might have a lot bigger problems on our hands than what we're talking about here today.

Science Fiction and Building the Future

Ksenia: It's funny that builders just don't talk a lot about sci-fi. Everything is in the trenches, hands-on, building. That's why I like to ask this question – are we thinking about sentient systems, or is this just technology we need to deal with on a different scale?

Anneka: I love sci-fi. I read sci-fi all the time. So it's stuff I like to think about. But as a builder, it's hard to predict that kind of future. That would involve a fair amount of social upheaval and a lot of different things that would monumentally affect our day-to-day life.

Ksenia: Well, then my last question would be about books. What is the book that formed you or influenced you? It can be a recent book or something from childhood.

Anneka: Like I said, I'm a big science fiction reader. There's not just one, but related to AI, Isaac Asimov's I, Robot series was so influential to me growing up.

The thing I love about sci-fi is that it helps you imagine so many alternative realities and what they could mean. What I specifically love about Asimov's books – not just I, Robot, but the rest of his books – is that they're very philosophical as well. There's so much that makes you think about ethics and how you would handle very different kinds of situations. Situations that are hard to predict, very different from things you're dealing with today.

Honestly, my love for sci-fi is what led me into being a builder, because I love thinking about the future. Some futures are probably too far ahead for what I need to think about today, but I find that so exciting – trying to think about and predict what the future could look like, even in small increments. Then thinking about how we can innovate and solve those newer kinds of problems that are going to emerge in new ways using the latest and greatest technology.

Ksenia: Thank you. That's fascinating. Is there any sci-fi book that you would love to be true in the future?

Anneka: Most sci-fi books are quite dystopian. The great technology advancements come with a lot of negatives as well.

I guess if I could have one sci-fi technology, I would love teleportation. I think that would be the best thing – to be able to just travel anywhere without having to spend the time traveling there. Getting to places like other planets, other solar systems, all of that – that would be incredible.

Ksenia Se: Maybe that's something we'll come to pretty soon. Thank you so much for this conversation. That was very insightful.