AI Cybersecurity & Governance

Throughout the course, you will connect lessons from major cybersecurity breaches to the emerging risks posed by generative and agentic AI. You will examine how these technologies are changing the threat landscape and explore the governance frameworks, regulatory requirements, and safeguards organizations can use to manage AI risk.

By the end of the course, you will be able to:

• Identify recurring managerial and technical causes of major cybersecurity breaches and apply those patterns to AI-specific vulnerabilities.
• Assess the expanded attack surface created by generative and agentic AI, including prompt injection, model extraction, deepfake fraud, and Shadow AI.
• Evaluate AI governance frameworks and connect them to operational controls across the AI lifecycle.
• Assess organizational exposure to applicable AI regulations and identify where personal and corporate liability may arise.
• Apply governance approaches including AI Bills of Materials, human-in-the-loop safeguards, and regulatory mapping to support informed AI adoption decisions.

Module 1: Cybersecurity Primer

Review the evolution of cybersecurity and the recurring patterns behind major data breaches.

• Describe how cybersecurity threats and defenses have evolved over time.
• Explain how supply chain vulnerabilities and weaknesses in multi-factor authentication can contribute to major data breaches.
• Compare major breaches involving Target, JPMorgan Chase, and the Office of Personnel Management to identify recurring patterns.

Module 2: AI Primer

Build foundational context by following the evolution of artificial intelligence from early rule-based systems to modern generative AI.

• Trace major developments in AI from early rule-based systems to modern generative AI.
• Differentiate machine learning from traditional rule-based approaches based on learning patterns from data.
• Associate developments such as neural networks, transformer architecture, and landmark AI applications with the advancement of modern AI.

Module 3: Multimodal GenAI Risks

Examine how generative AI creates new risks across text, audio, video, and other AI-enabled systems.

• Illustrate how generative AI can create deceptive content across text, audio, and video.
• Interpret the causes of AI hallucinations, their potential use in misinformation, and mitigations such as retrieval-augmented generation.
• Outline how adversarial attacks can fool AI systems and how human-in-the-loop review, adversarial training, and ensemble methods can mitigate these risks.

Module 4: Agentic AI Risks

Focus on the security risks associated with increasingly autonomous AI agents and approaches for managing them.

• Classify key risks including Shadow AI, authorization and control hijacking, supply chain and dependency attacks, and alignment faking.
• Justify why increasingly autonomous AI agents require organizational visibility, management, and governance.
• Describe how AI inventories and AI Bills of Materials can help organizations manage Shadow AI and AI supply chain risks.

Module 5: Breaches on and with AI

Examine AI as a dual-use technology by looking at both attacks on AI systems and the use of AI to attack other systems.

• Distinguish between attacks on AI systems and attacks in which AI systems are used to attack other systems.
• Explain how model extraction attacks can replicate an AI model's functionality and compromise the intellectual property invested in its development.
• Discuss how model distillation can use the outputs of a larger teacher model to create a smaller model with similar capabilities at substantially lower cost.

Module 6: AI Governance and Regulation

Learn how governance frameworks and regulations can support the responsible and trustworthy use of AI systems.

• Justify why AI governance frameworks can help organizations manage risks associated with the development, deployment, and use of AI systems.
• Compare governance frameworks, including ISO 42001, the Secure AI Framework (SAIF), and the AI risks they address.
• Recognize how AI laws and regulations establish requirements, guidelines, and potential penalties for the use of AI.

Faculty Spotlight: An Interview with Professor Junfeng Yang

Hear from Professor Junfeng Yang about research spanning machine unlearning, AI security, and AI-generated content detection.

• Explain machine unlearning and its relevance to privacy compliance and the removal of poisoned training data.
• Illustrate how backdoors can be introduced into AI systems during model loading or compilation, independent of the training data itself.
• Evaluate how prompt diversification can improve automated jailbreak discovery and its implications for red-teaming and guardrail design.