NEW DELHI: The Indian Computer Emergency Response Team (CERT-In) has set out specific security and auditing requirements for artificial intelligence (AI) systems that can independently plan and carry out tasks, use external tools, and delegate work to other AI systems.
The Artificial Intelligence (AI) Security Auditing Guidelines: Framework for Secure, Ethical, and Auditable Artificial Intelligence Systems identify features such as autonomous task execution, persistent memory, and the ability to delegate work as factors that can increase the risks associated with these systems.
The framework calls for testing whether an AI agent can exceed its authorised permissions, use tools without approval, bypass approval mechanisms, or misuse credentials. It also identifies unsafe delegation of tasks and the effectiveness of emergency-stop mechanisms as areas that auditors should examine.
The guidelines also address the risks associated with an agent’s memory and the information it retains from earlier interactions or tasks. They call for testing whether this memory can be manipulated, whether information belonging to one user can leak to another, and whether sensitive information is retained or reused without authorisation.
For multi-agent systems, the framework calls for records that allow auditors to reconstruct what happened — including requests, delegated tasks, permissions, decisions, tools used and resulting external actions. Each agent must have a distinct identity and defined access rights, and communications between agents must be authenticated and recorded.
Controls must also detect and restrict repeated delegation, uncontrolled task creation and loops in which agents repeatedly pass work between themselves.
For high-risk systems, the framework also specifies the ability to suspend an individual agent or an entire workflow and revoke its access credentials. It also calls for an inventory covering production models, training data, software and external tools.
The guidelines outline four risk tiers for AI security audits and aim to ensure the governance, security, privacy, fairness, transparency, resilience and auditability of AI systems.
