AI tools in software development range from simple API calls to fully autonomous agent systems. Understanding the categories, their risk profiles, and governance requirements is essential for any organisation adopting AI-assisted development.
AI Technology Categories
The risk levels below (Low / Medium / High) are a qualitative, illustrative assessment to help you triage categories for review—they are not derived from an external benchmark or standard. Calibrate them to your own threat model.
LLM APIs
Large Language Model API providers (OpenAI, Anthropic, Google AI, Cohere, Mistral).
- Risk Level: Medium
- Governance: May process sensitive data via API calls. Requires data classification review.
AI Coding Assistants
AI-powered code generation and assistance tools (GitHub Copilot, Cursor, Codeium, Tabnine, Amazon Q Developer).
- Risk Level: Medium
- Governance: May expose code context to external services. Requires security review.
AI Frameworks
AI/ML application frameworks and orchestration (LangChain, LlamaIndex, Haystack).
- Risk Level: Low
- Governance: Standard dependency management applies.
Vector Databases
Vector storage for embeddings and RAG applications (Pinecone, Weaviate, Qdrant, ChromaDB, pgvector).
- Risk Level: Low
- Governance: Document embeddings may contain sensitive content.
AI Agents
Autonomous AI agent frameworks with tool use capabilities (LangChain Agents, CrewAI, AutoGPT, Claude Agent SDK).
- Risk Level: High
- Governance: Agents can execute code, access files, and make network requests autonomously. Requires strict resource limits.
Agent Orchestration
Multi-agent coordination and workflow systems (LangGraph, CrewAI Crews, AutoGen GroupChat).
- Risk Level: High
- Governance: Multi-agent systems multiply attack surfaces. Require strict resource limits and audit logging.
Usage Patterns and Maturity
The two tables below are an illustrative, proposed model—not an industry standard or a published benchmark. Use them as a starting point for an internal conversation, not as a normative framework.
| Pattern | Maturity | Key Indicators |
|---|---|---|
| Simple API Call | Experimental | Single API call, basic error handling |
| Tool/Function Calling | Emerging | tools= parameter, function definitions |
| Basic RAG | Emerging | Vector store queries, context assembly |
| Advanced RAG | Standardised | Multiple retrievers, reranking |
| Single Agent | Standardised | Agent executor, tool chain |
| Multi-Agent System | Optimised | Agent orchestration, role specialisation |
| Autonomous System | Strategic | Continuous execution, goal decomposition |
Maturity Levels
| Level | Name | Description |
|---|---|---|
| 1 | Experimental | Individual developers testing AI APIs. Hardcoded keys, single-file usage. |
| 2 | Emerging | Team-level AI adoption. Environment variables, basic retry logic. |
| 3 | Standardised | Organisation-wide AI standards. Shared libraries, proxy/gateway usage. |
| 4 | Optimised | AI Centre of Excellence. Custom abstractions, caching layers, usage analytics. |
| 5 | Strategic | AI-native architecture. Multi-model routing, agentic workflows, RAG infrastructure. |
Governance Framework
Categories Requiring Approval
- LLM APIs (
ai-ml/apis) - AI Agents (
ai-ml/agents) - Agent Orchestration (
ai-ml/patterns/orchestration)
Categories Requiring Security Review
- LLM APIs — data classification and API key management
- AI Agents — autonomous execution capabilities
- Tool/Function Calling — input validation requirements
- AI Coding Assistants — code context exposure
Data Classification
| Category | Risk |
|---|---|
| LLM APIs | May process sensitive data via API calls |
| AI Agents | Autonomous execution with external access |
| RAG | Document embeddings may contain sensitive content |
| Coding Assistants | May expose proprietary code to external services |
Risk Mitigation
For AI Agents
- Implement least-privilege tool access — Only grant the permissions each agent needs
- Add rate limits and circuit breakers — Prevent runaway execution
- Log all agent actions for audit — Every tool call, every decision, every output
- Implement kill switches — Ability to terminate autonomous agents immediately
For Tool/Function Calling
- Validate all tool inputs — Never trust LLM-generated arguments without validation
- Avoid eval/exec with tool arguments — Treat tool inputs as untrusted user input
- Implement rate limiting — Prevent excessive tool calls
For Agent Orchestration
- Set iteration and time limits — Prevent infinite loops
- Authenticate agent-to-agent communication — Don't assume internal messages are trustworthy
- Monitor resource consumption — CPU, memory, network, and API call budgets
Tracking AI Tool Adoption
Most organisations have no visibility into which AI tools developers are using. Build inspection and desktop monitoring can identify:
- Which AI coding assistants are in use (and which are unsanctioned)
- How much code is AI-generated vs human-written
- Which teams are early adopters and which need support
- Whether AI-generated code follows the same review standards as human code
This visibility is the prerequisite for effective governance. You cannot govern what you cannot see.