We covered comments from Google's red team targeting AI back in episode 248. Here's a similar article from Microsoft.
A few things stand out from "normal" red team exercises. One, this testing is more expensive. Two, it requires repeated attempts. Three, and more interesting to me, is testing how such systems might generate harmful content for average users. In other words, the testing is just figuring out what a malicious actor might do, but what the system might do to a benign user.