Violet Teaming Enhances GPT-4 Beyond Red Teaming.

Red teaming, or attempting to get an AI system to act in harmful or unintended ways, is a valuable step toward building AI models that won’t harm society. However, it is not enough. Violet teaming, or identifying how a system might harm an institution or public good, and then supporting the development of tools using that same system to defend the institution or public good, is necessary. To protect against the impact of AI systems, democratic innovation is also needed to decide what guardrails are necessary for model release.