“Beat the Bot” Simulation Exercise

Beat the Bot (BTB) is an online simulation exercise designed to help students explore the strengths and limitations of human-AI collaboration. 

In this simulation, students will learn what makes human creativity unique, how LLMs approach generative tasks, and the implications for human-AI collaboration in creative work. 

The simulation, authored by Dawei Wang, Brian Uzzi, Haipeng Shen, and Jerry Chai, is based on Wang et al’s 2025 Nature Human Behaviour paper, which examines whether LLMs can match or surpass humans in creativity. With a sample of over 10,000 humans and 200,000 LLM responses, the research evaluates both baseline creativity and the effects of manipulating model parameters and prompt types.

Results reveal nuanced differences between machine and human creativity, with important lessons for collaboration design and future of work. “To get the most out of a bot, don’t ask it for answers,” says co-author Uzzi. “Ask how to approach a problem. You want advice on how to think, not what to think.”

A preprint of the underlying research can be found here. (A lay-person-friendly summary is also available from Kellogg Insight.)