
Intelligence
6 mins read
What a Modern SOC Really Does All Day

Assessment
6 mins read
Vulnerability Manage Without Noise

Control
8 mins read

Written by
Published

AI has arrived on both sides of the fight. It writes more convincing phishing at greater scale, and it triages alerts faster than any analyst. But the newer, less discussed problem is that the AI systems your own company is deploying have become an attack surface of their own.
A model connected to your customer database and given the ability to take actions is, from a security perspective, a new kind of privileged user — one that can be talked into things. Traditional application security assumes code follows instructions; here, the instructions arrive in the data.
The distinctive risks have names now: prompt injection hidden inside documents a model reads, training data poisoning, model extraction, and agents with credentials broader than anyone intended.
Prompt injection is the one that keeps engineers awake. If a model summarizes a web page or an email, and that content contains instructions, the model may follow them — it has no reliable way to distinguish the data it was asked to process from the commands it was asked to obey. There is no patch for this; there is only architecture that limits the damage.
Treat every model as untrusted and every model output as user input. Give AI systems the narrowest credentials that let them function, and never wire an agent directly to an action — deleting records, sending payments, changing permissions — without a human confirming the specific request. Keep an inventory of where AI is deployed, including the tools staff adopted without telling anyone. Log prompts and responses, classify what data may reach a model at all, and decide explicitly whether a third-party provider may retain it.

Guardrail layers that filter inputs and outputs help, and so do gateways that sit between your applications and a model provider, giving you a single place to enforce policy and see usage. None of it is watertight. The controls that actually hold are the old ones applied in a new place: least privilege on the agent's credentials, human approval before consequential actions, and blast-radius limits that assume the model will one day be manipulated successfully.
Blanket bans on AI tools do not stop adoption; they move it onto personal accounts where you cannot see it. Offer a sanctioned option that is genuinely good, publish plainly what may and may not be pasted into it, and explain the reasoning rather than issuing a rule. Staff comply with restrictions they understand and route around ones that appear arbitrary.

Red team the model the way you would red team an application. Hide instructions in a document and see whether the assistant obeys them. Ask it, in a hundred creative ways, to reveal its system prompt or reach data it should not touch. Test the agent's permissions directly rather than trusting its refusals — a model that politely declines a request while holding credentials that would allow it has not been secured, only persuaded.
Most AI risk in organizations today is not adversarial at all — it is an employee pasting a confidential contract into a consumer chatbot to get a summary before a meeting. That behavior comes from genuine time pressure, and it will not be trained away by a policy nobody reads. Give people a safe tool that is faster than the unsafe one, make it clear that asking whether something is allowed is always welcome, and treat the first person who admits to pasting something sensitive as a source of intelligence rather than a disciplinary matter.
Type
Intelligence
Reading time
8 mins read
Share
Join our newsletter and stay updatedon the latest trends in digital design

Intelligence
6 mins read

Assessment
6 mins read

Control
8 mins read
Automate tasks, centralize projects, and collaborate in real time — all from one sleek
FREE 14-DAY TRIAL · NO CREDIT CARD