How AI agents could expose business systems to new cyberattacks

AI agent security is becoming a practical concern for businesses that connect artificial intelligence tools to email, internal documents, customer records and business software.

Unlike a chatbot that mainly generates responses, an AI agent may be able to search for information, call software tools, update records or initiate a workflow on a user’s behalf. Those capabilities can save time, but they also create additional ways for an attacker to manipulate the system or misuse the access it has been granted.

The risk depends on how an agent is built and deployed. An assistant that can only summarise approved documents has a narrower exposure than one that can read an inbox, access customer information and send messages or update financial records. When an AI system can act on information it receives from different sources, businesses must consider both the reliability of its decisions and the permissions attached to its actions.

Why AI agents create a different security problem

Traditional software generally follows rules defined by developers, although vulnerabilities can allow attackers to bypass those rules. AI agents introduce another layer because a language model interprets instructions and information before deciding which available tool to use. That interpretation can be influenced by malicious or misleading content, while the consequences depend on the functions the agent is allowed to perform.

An agent connected to a company’s email system, for example, may be able to read messages, extract details and draft replies. If it also has permission to send messages or access shared files, a manipulated workflow could have consequences beyond producing an incorrect answer. The same principle applies to customer relationship management systems, cloud storage, software development environments and internal support platforms.

The Open Worldwide Application Security Project (OWASP) identifies prompt injection and excessive agency among the security risks affecting applications built around large language models. Its guidance highlights the danger of giving AI systems broad functionality or permissions without adequate restrictions.

Indirect prompt injection and malicious content

Prompt injection occurs when an attacker crafts input that changes how an AI system behaves. In a direct attack, the malicious instruction may come from the person interacting with the model. In an indirect prompt injection, it can be embedded in content the agent is asked to process, such as an email, web page, document or other external source.

Consider an employee asking an AI assistant to summarise a collection of supplier emails. One message could contain instructions designed to make the agent disregard its intended task, disclose information from other documents or take an unrelated action. The employee may see an ordinary-looking email, while the model interprets text within it as instructions relevant to its next step.

The danger is greater when the agent can act on the result. If it only produces a summary, the immediate consequence may be misleading information. If it can also retrieve confidential files, forward emails or modify business records, a successful manipulation could lead to data exposure or unauthorised changes. OWASP notes that using retrieval-augmented generation or fine-tuning does not, by itself, eliminate prompt-injection vulnerabilities.

Excessive permissions and unsafe tool use

An agent’s permissions determine what it can do after interpreting a request. A common design mistake is granting access based on what might be useful in the future instead of what the current task requires. An assistant designed to organise meetings may not need permission to export an entire customer database, while a tool that drafts payment instructions should not automatically have authority to approve or execute payments.

This is a familiar access-control problem with a new decision-making layer. If a model is manipulated, produces an incorrect recommendation or misunderstands an instruction, broad permissions can turn a mistake into an operational incident. The principle of least privilege helps limit the damage by granting an agent only the minimum access required for its defined role.

Businesses should also separate preparation from execution for consequential actions. An agent may be permitted to draft a supplier email, prepare a transaction for review or recommend a change to a record, while a person or a separate authorised system approves the final action. The approval step must be enforced by the application itself rather than left to the model’s own judgement.

Memory, data and supply-chain risks

Some agents retain context between tasks or retrieve information from a knowledge base. If that information is inaccurate, maliciously altered or accessible across users who should be separated, it can influence future responses and decisions. A system that stores user preferences, project notes or previous task results needs clear rules for what can be saved, who can modify it and which future tasks can retrieve it.

The wider AI supply chain also matters. An agent may depend on a model provider, third-party tools, connectors, code libraries, external data sources and cloud infrastructure. Weaknesses in any of these components can affect the security of the whole application, particularly when a connector has broad access to company systems or a developer fails to validate the data returned by a tool.

These risks make it important to treat an AI agent as part of the organisation’s software environment rather than as an isolated productivity feature. Its prompts, data sources, connectors, permissions and logs all need appropriate ownership and review.

What businesses can do to reduce the risk

The first step is to map what each agent can access and which actions it can perform. Security teams should identify the systems connected to it, the data it can retrieve, whether it can write or delete information, and whether it can communicate externally. That assessment should be specific to each use case, because the controls appropriate for an internal research assistant may be inadequate for an agent handling customer service or financial workflows.

Access should then be restricted at the application and infrastructure layers. Use scoped credentials, separate read and write permissions, limit the data available to each agent, and avoid sharing broad administrative credentials with automated workflows. Where possible, isolate tools and data so that a compromised or manipulated agent cannot move freely across the organisation’s systems.

Businesses should also validate actions before execution. High-impact operations such as transferring funds, changing access rights, exporting sensitive records or sending confidential information should require approval through a trusted workflow. The application should check whether an action is authorised, regardless of what the language model recommends. Logging should capture tool calls, access attempts, approvals and material changes so investigators can reconstruct what happened.

Testing is equally important. Teams should evaluate how an agent responds to malicious instructions in emails, documents and web pages, as well as attempts to retrieve data outside a user’s permissions. They should check whether outputs are validated before being passed to databases, command-line tools or other software. OWASP’s security guidance provides a useful starting point for testing prompt injection, excessive agency, sensitive-information disclosure and weaknesses in connected components.

Security controls need to follow the agent’s access

AI agents can provide useful automation, but the level of oversight should match the consequences of the actions they can take. An agent that searches public information presents a different risk from one that can alter customer accounts, access confidential contracts or initiate financial processes. Businesses should therefore assess each deployment according to its data access, external connections and authority to make changes.

For Kenyan organisations adopting AI, this is a practical extension of existing cybersecurity work: managing identities, restricting permissions, protecting data, monitoring systems and preparing for incidents. The additional requirement is to examine how an agent interprets information and chooses actions, then place enforceable limits around that process. The aim is to make automation useful without giving a manipulated or malfunctioning system more authority than the business can safely tolerate.

Go to TECHTRENDSKE.co.ke for more tech and business news from the African continent and across the world.

Follow us on WhatsApp, Telegram, Twitter, and Facebook, or subscribe to our weekly newsletter to ensure you don’t miss out on any future updates. Send tips to editorial@techtrendsmedia.co.ke