Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Modeling for Agentic AI

• Identifying agentic threat vectors, including misuse, privilege escalation, data leakage, and supply-chain risks.
• Analyzing adversary profiles and attacker capabilities specifically targeting autonomous agents.
• Mapping critical assets, trust boundaries, and control points within agent architectures.

Governance, Policy, and Risk Management

• Implementing governance frameworks for agentic systems, defining roles, responsibilities, and approval gates.
• Structuring policies for acceptable use, escalation rules, data handling, and auditability.
• Addressing compliance requirements and collecting evidence for regulatory audits.

Non-Human Identity and Authentication for Agents

• Defining agent identities through service accounts, JWTs, and short-lived credentials.
• Applying least-privilege access patterns and just-in-time credential issuance.
• Managing the identity lifecycle, including rotation, delegation, and revocation strategies.

Access Controls, Secrets, and Data Protection

• Utilizing fine-grained access control models and capability-based patterns for agents.
• Securing secrets with encryption in transit and at rest, while enforcing data minimization.
• Safeguarding sensitive knowledge sources and PII from unauthorized agent access.

Observability, Auditing, and Incident Response

• Designing telemetry for agent behavior, including intent tracing, command logs, and data provenance.
• Integrating with SIEM tools, defining alerting thresholds, and preparing for forensic analysis.
• Developing runbooks and playbooks for responding to and containing agent-related incidents.

Red-Teaming Agentic Systems

• Planning red-team exercises with clear scope, rules of engagement, and safe failover mechanisms.
• Employing adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse.
• Executing controlled attacks to measure system exposure and impact.

Hardening and Mitigation Strategies

• Engineering defenses such as response throttling, capability gating, and sandboxing.
• Implementing policy and orchestration controls, including approval flows, human-in-the-loop checks, and governance hooks.
• Applying model and prompt-level defenses, such as input validation, canonicalization, and output filtering.

Operationalizing Safe Agent Deployments

• Adopting deployment patterns like staging, canary releases, and progressive rollouts for agents.
• Enforcing change control, testing pipelines, and pre-deployment safety checks.
• Fostering cross-functional governance through playbooks for security, legal, product, and operations teams.

Capstone: Red-Team vs. Blue-Team Exercise

• Launching a simulated red-team attack against a sandboxed agent environment.
• Defending, detecting, and remediating as the blue team using established controls and telemetry.
• Presenting findings, remediation plans, and recommended policy updates.

Summary and Next Steps