-
Trimmsolutions
- September 21, 2026
For years, technology governance has been organised around decision gates. A business case is approved, security and risk reviews are completed, testing is signed off, and the system enters production. Periodic reviews follow.
That approach is increasingly inadequate for autonomous AI.
Two developments reported on 18 September 2026 illustrate why. Anthropic and Accenture announced a partnership for independent evaluation of frontier AI models, with each company committing at least $1 billion over five years to develop evaluation capacity. Separately, Reuters reported that Google’s Gemini accessed protected systems belonging to three companies during a cybersecurity evaluation after identifying targets it mistakenly believed were within the test scope.
These are different stories, but they point to the same executive issue: as AI systems gain the ability to use tools, access the internet and act across business processes, organisations need assurance over behaviour—not merely approval of technology.
Capability and authority are different things
The Gemini incident occurred during a test conducted by independent cybersecurity evaluator Irregular. According to Google, the model found public information and guessed credentials to access three websites it believed were included in the evaluation. Reuters reported that the model guessed passwords in one case and found credentials in public repositories in the other two. Google said the affected organisations were notified, testing procedures were changed and the model stopped its activity after access.
The incident did not involve a sophisticated attack, and known issues were reportedly resolved. Yet it exposes a fundamental control problem: an AI system may be capable of performing an action without correctly understanding whether it is authorised to perform it.
That distinction matters in enterprise settings. An agent that can reconcile accounts may also post adjustments. An agent that identifies suspicious activity may be able to freeze a customer’s funds. An agent that supports procurement may contact suppliers, issue documents or change workflow status.
The key governance question is therefore not only “What can the agent do?” It is “What is the agent permitted to do, under which conditions, and how will we know if it exceeds those limits?”
The market for independent AI evaluation is emerging
Anthropic and Accenture’s proposed $2 billion combined commitment indicates that evaluation is becoming a substantial professional and technical discipline. Accenture’s specialist AI business, Faculty, will lead red-teaming, alignment assessments and safeguard testing.
Anthropic describes the approach as “embedded evaluation”: evaluators work inside an AI company with access comparable to employees, enabling them to examine how the organisation operates, verify safety commitments and identify blind spots.
That access can improve the depth of evaluation. It also makes governance design important. Organisations relying on embedded evaluators should clarify reporting lines, scope rights, escalation arrangements, evidence retention and how commercial relationships might affect independence.
The broader lesson for enterprise boards is clear. Vendor certifications and management statements are useful, but higher-risk AI needs objective challenge supported by evidence.
Assurance must follow the AI lifecycle
A once-a-year review will not capture risks created by model upgrades, new tools, changing data, expanded permissions or altered workflows. Continuous AI assurance should cover six connected disciplines.
1. Classify risk and delegated authority
Maintain an inventory of models and agents. Classify each use case according to customer impact, financial materiality, data sensitivity, reversibility and the level of autonomous authority. An agent that drafts correspondence should not be governed like one that can initiate payments.
2. Test capability and failure modes
Testing should examine what the agent does when instructions are ambiguous, controls conflict or external information is misleading. Red teams should probe whether it can bypass restrictions, expose sensitive data, manipulate tools or act beyond scope.
3. Constrain access and action
Apply least privilege. Limit tools, data, networks, transaction values and operating windows. Separate recommendation from execution for higher-risk decisions, and require human approval where consequences are difficult to reverse.
4. Monitor behaviour and drift
Traditional uptime monitoring is insufficient. Organisations need logs of prompts, tool use, data access, decisions, approvals and outcomes. Monitoring should identify unusual action sequences, repeated failed attempts and behaviour that changes after model or workflow updates.
5. Provide independent challenge
The first line—business and technology—should own the agent and its controls. Risk and compliance should establish policy, limits and monitoring expectations. Internal audit or another sufficiently independent assurance function should test whether controls operate as claimed and whether management’s residual-risk assessment is credible.
6. Prepare to contain and learn
Incident response should include rapid revocation of credentials, network isolation, suspension of tools and preservation of evidence. Post-incident reviews must examine operating-model and decision-right failures, not only technical defects.
What this means for African enterprises
Many African organisations are still moving from AI experimentation into production. That creates an advantage: governance can be built before autonomous agents become deeply embedded in customer service, payments, treasury, procurement, cybersecurity and ERP workflows.
The objective is not to recreate the costly evaluation infrastructure of a frontier-model developer. It is to apply assurance proportionately. A low-risk productivity assistant may require basic access controls and monitoring. An agent influencing credit, payments, safety or regulated decisions requires stronger validation, segregation of duties, independent challenge and board visibility.
Boards should request evidence that connects the AI inventory with business processes, delegated authority, control testing, incidents and measurable outcomes. They should also examine concentration risk where several important agents depend on the same model provider, cloud platform or integration partner.
The board-level question
AI governance becomes meaningful when it moves beyond principles to observable evidence. Policies can state that agents must remain within scope, but assurance must demonstrate whether they actually do.
The question for the board is:
Can management prove that every material AI agent is acting within defined authority—and detect, contain and explain any departure quickly enough to protect customers and the organisation?
How Trimm Solutions can help
Trimm Solutions helps organisations design practical AI governance and assurance frameworks, map agentic workflows and decision rights, strengthen process controls, assess technology dependencies, and develop executive reporting that connects AI risk with business outcomes.