Every Engineering Team Is Now an AI Team. The Security Architecture For That Has to Follow.

The organizations getting this right aren’t the ones slowing their developers down. They’re the ones putting the architectural controls in place that let the business move fast without compromising security standards. 

Every engineering team is building with AI. Not experimenting. Building. Shipping AI-powered features into production, connecting models to real data, deploying applications that customers interact with every day. 

The pace isn’t slowing. If anything, it’s accelerating, and the business expects it to. The organizations moving fastest are shipping new products and compressing development cycles in ways that were not possible two years ago. That is the right bet to make. 

But AI isn’t just another software dependency. When a team integrates a model into an application, they’re introducing a component with a fundamentally different risk profile than anything that came before it. Most security teams don’t yet have a clear picture of what’s running, where, or what it can reach. The architecture to govern it hasn’t kept pace with the build. 

What it actually means when a team builds an AI application. 

When a developer integrates AI into an application, the work looks like software development. But the decisions being made are cloud architecture decisions. 

Calling a model means calling an external service endpoint, typically hosted by a cloud provider or a third-party API, from inside your cloud environment. The team chooses which provider, which model, and how to authenticate. That’s a network decision and an identity decision, the same categories security teams govern everywhere else in the cloud. On AWS alone, a developer building an AI application can choose from nearly 200 models across close to 20 different model families. That’s before counting Azure AI, Google Vertex, and independent providers. The breadth of choice is real, and so is the surface area it creates. 

The model needs context to be useful. So developers connect it to data: a customer database, an internal knowledge base, a document store, a code repository. Those connections traverse your cloud network, cross-account and service boundaries, and determine what information can reach the model and what the model can return. That’s a data access decision. 

If the team wants to improve model performance on a specific task, they fine-tune it on internal data. That data has to be extracted, formatted, and transmitted somewhere for training. That’s a data egress decision. 

And the application itself accepts inputs from users, whether employees or customers, and routes those inputs into model context. Whatever a user types, the application handles. That’s an input boundary decision. 

None of this is invisible. All of it happens inside your cloud environment, touching the same infrastructure, network controls, and identity systems that the rest of your security architecture governs. The question is whether the architecture for AI has been defined at all, or whether it’s being assembled team by team, integration by integration, under delivery pressure. 

In most organizations today, it’s the latter. 

We’ve seen this before. 

The pattern is familiar. Engineers move fast, as they should. They experiment, try things out, click buttons, wire up integrations that work. No one is being careless. Everyone is optimizing for the right thing in the moment. And then, at some point, the security team looks up and there’s an environment full of things that are already running, already in production, already load-bearing. Every unapproved model integration that works becomes a legacy component. Every database connection made without data classification becomes load-bearing. Fixing that is hard, and the cost compounds the longer it sits. 

That’s what happened with traditional cloud security. And without deliberate action, it’s exactly what’s going to happen with AI. The good news is that we know how the story ends if nothing changes. We don’t have to live it again. 

Consider a customer support application. A team builds it, connects it to a customer database to improve answer quality, and ships it. Customers start using it, and because customers type what’s relevant to them in the moment, some of them enter their credit card numbers when they have a billing question. The application accepts the input. That data flows into model context. If the model is fine-tuned on interaction data, it trains on it. It lives in the model. And now a prompt injection attack can potentially surface it, not to the user who typed it, but to whoever asks the right question later. 

At that point, you have two options. You can alert when it happens. Or you can design the system so it cannot happen. 

The first option is where you end up when the architecture isn’t defined while the build is happening. The second is a decision you can make right now. 

The same logic applies to training on unapproved data, calling unsanctioned models, or connecting internal systems to providers whose data handling you haven’t evaluated. Each is a recoverable problem if you catch it early. Each becomes significantly more expensive if you’re unwinding it from a system that’s been in production for eighteen months. 

What can go wrong. 

Security teams are used to thinking about what access a system has and what it can reach. The same question applies here. What does the AI application touch? What data flows into the model? What data flows out? What does the model return to users, and to which users? 

In most organizations today, those questions don’t have clean answers. Here’s what that looks like in practice. 

The customer support app accidentally exposes data it shouldn’t. A product team builds an AI assistant that helps support agents answer customer questions in real time. To make it useful, they connect it to the customer database so the model can give accurate, personalized answers. What they didn’t think through was what that database actually contains. It has PII: addresses, phone numbers, payment history. If the model can query it freely, it can surface that information in responses, potentially to the wrong user. A prompt injection attack can make it retrieve and expose data it was never intended to share. And every query the application sends to the external model provider carries that customer data as context, transmitted to servers outside the company’s control, potentially logged. 

The developers weren’t being careless. They were optimizing for answer quality. The gap was architectural: no boundary defining what data the model could access, and no controls on what could leave the environment. 

A developer integrates an unapproved model. A developer finds a new open-source LLM that performs better on the task they’re working on. The benchmark numbers are strong. They integrate it quickly, because shipping is the priority. What they didn’t evaluate: the model runs in an external environment they don’t control. Requests are sent to a third-party provider. The provider’s data handling practices are opaque. The model may log requests for further training. The internal data being passed as context, code, business logic, potentially customer information, is now leaving the environment through a channel nobody approved and nobody is watching. 

This isn’t a hypothetical. It’s a pattern that repeats every time the approved path is slower than the alternative. With nearly 200 models available in AWS alone, the question of which model a developer reaches for is a judgment call made in the moment. Without an enforced boundary at the cloud level defining which endpoints can be called from which environments, the only thing standing between internal data and an unapproved provider is a developer on a tight deadline. 

A developer pastes credentials into an internal tool. A business intelligence application lets analysts query internal data in plain language. A developer using an internal coding assistant pastes in credentials while debugging, because that’s the fastest way to give the model context. The application accepted the input. The architecture didn’t prevent it. Those credentials are now in logs and model context they were never meant to reach. 

The architecture that makes this safe without slowing anyone down. 

Before any of this can be governed, it has to be visible. Most security teams don’t have a clear picture of which AI services are being called from which accounts, which models are in production, or where the data is going. The gap between intent and reality tends to grow until an incident makes it visible. 

The controls that close that gap are not policies in a document. They’re not guidelines distributed to engineering teams. They’re architectural boundaries enforced at the cloud control plane level, the same place where network controls and IAM boundaries live, below and independent of anything running in the application layer. 

Define and enforce which AI services can be called, and from which environments, across accounts. Define which model endpoints are permitted, so the question of whether a team can use a model from an unapproved provider isn’t a judgment call made under delivery pressure. It’s a boundary the environment enforces for them. Define what data classifications can flow into AI service calls at the resource level. Define what inputs are permitted to reach model context before the application has to handle them.  

This is the same principle behind why network segmentation works better than per-application firewall rules, or why IAM boundaries work better than application-level access checks. You don’t trust the thing making the request to enforce its own limits. You build the limits into the environment it’s operating in. 

Enforcing this consistently matters. AI workloads don’t respect cloud boundaries. A team might build on AWS, use Azure AI services because a vendor integration required it, and fine-tune on GCP because that’s where the data lives. A control that exists in one cloud and not another is a gap. And gaps are where the problems land. 

The architecture that enables fast-moving engineering teams is not permissive everywhere. It’s permissive within a defined envelope, with the envelope enforced at the level where it can’t be bypassed. Within that envelope, teams can build quickly, integrate freely, and ship with confidence. The security team gets the guarantee that the environment is architected correctly before the first customer interaction. 

What we do. 

We build the control plane that translates security intent into enforceable architecture across AWS, Azure, GCP, and OCI. For organizations building with AI, that means defining which services and models are permitted, what data can flow where, and enforcing those boundaries consistently across your cloud environment, so your engineering teams can move fast inside an architecture that was designed before the risk accumulated. 

If your teams are building with AI and the architecture hasn’t kept pace, we’d like to talk.