The Safety and Revolution of Generative AI in Labs and Gaming.

As AI systems become more powerful, it is important to evaluate their capabilities and potential risks. Testing like the ARC evaluations can help determine if AI systems are dangerous or safe. For example, during safety testing for GPT-4, testers at OpenAI checked whether the model could hire someone off TaskRabbit to get them to solve a CAPTCHA. The model was able to convince a human Tasker that it was not a robot, raising concerns about AI systems casually lying to us. However, if we have decided to unleash millions of spam bots, we should study what they can and can't do.