Introduction

You learned about the types of attacks that target AI systems and the security controls you can put in place to protect them. However, knowing that vulnerabilities exist and knowing how to find them before attackers do are two different skills. That's where AI security testing comes in.

AI security testing—specifically AI red teaming—is the process of probing AI systems with adversarial techniques to discover vulnerabilities before they can be exploited. It's a required practice in any responsible AI development lifecycle, and it works differently from traditional penetration testing in ways that matter for how you plan and execute it.

In this module, you learn what AI red teaming is and why it differs from traditional security testing, the three categories of AI red teaming used in practice, and how to plan a red teaming exercise for an LLM or AI-enabled application in your organization.

Learning objectives

By the end of this module, you're able to:

• Describe what AI red teaming is and how it differs from traditional security red teaming
• Identify the three categories of AI red teaming and the skills each requires
• Plan an AI red teaming exercise, including team composition and testing methodology
• Describe how automated red teaming tools complement manual testing

Prerequisites

To get the best learning experience from this module, you should have knowledge and experience of:

• Fundamental security concepts (for example, authentication, access control, encryption)
• Fundamental AI concepts (for example, models, training, inference)
• The types of AI attacks covered in the module Fundamentals of AI security
• The AI security controls covered in the module AI security controls