Yiming Zhang didn't grow up playing chess. Like many other people, the Carnegie Mellon University Ph.D. student discovered the Netflix series "The Queen's Gambit" during the pandemic and began playing online. However, he quickly realized how unnatural it felt playing against chess bots.
"After I learned the rules, I was in the bottom 10%, maybe 20% of players online," said Zhang, who is part of the Language Technologies Institute (LTI) in CMU's School of Computer Science. "For beginners, it's not interesting or instructive to play against chess bots because the moves they make are often bizarre and incomprehensible to humans."
Zhang's frustration led him to develop Allie, a chess bot powered by artificial intelligence that demonstrates the benefits of AI tools that think like humans. He believes training future AI systems to ponder and deliberate on complex problems could create better agents for use in therapy, education and medicine.
"There's been an obsession with building superhuman AI that's better at math or other reasoning tasks than most humans," said Daphne Ippolito, Zhang's adviser and an assistant professor in the LTI. "But there are a lot of opportunities for us to train AI models to act like humans, and I think that's worth exploring."
Allie plays similarly to a human and can adapt to various strengths, from beginner to expert. It was trained similarly to the language models that underpin modern chatbots, such as ChatGPT. But instead of feeding Allie text from the internet, the team trained it on 91 million transcripts from the popular chess platform Lichess. Exposing Allie to transcripts from chess games played by humans taught it how to make moves a human player would make, take time to contemplate critical positions, and resign when the game is unwinnable.
"I'm excited about how the adaptive methods we used combine classic AI search procedures with modeling of human behavior, and how that combination is better than either method on its own," said Daniel Fried, an assistant professor in the LTI who worked on the project. "Methods like the ones we used have already been applied in complex games such as Diplomacy, and I'm excited to see them used in other tasks where AI needs to act strategically, but in human-compatible ways."
Most chess engines are built with one goal: to win. They simulate countless future moves, pitting variations of themselves against each other in a self-improving loop with the absence of human data. This approach results in systems with nearly unbeatable strength, such as AlphaZero or Stockfish, making them unenjoyable opponents for casual players and beginners.
"Prior to Allie, a chess engine didn't exist that modeled how people think," Zhang said. "Chess bots instantly made moves in complex positions where humans would need time to consider various options, or would continue playing in completely lost positions where humans would normally resign. This made the chess AIs that existed seem unnatural."
When asked about future plans, the team explained that Allie is completely open source, and has amassed nearly 10,000 games since its deployment on Lichess.
"Our project is meaningful because it assesses how people interact with AI that attempts to be humanlike," Ippolito said. "We also deliberately built an open-source platform that people can build from."
Allie was presented at the 2025 International Conference on Learning Representations in Singapore, one of the premier venues for machine learning research. Zhang, Ippolito and Fried collaborated on the project with Athul Paul Jacob, a Ph.D. student at the Massachusetts Institute of Technology; and Vivian Lai, a researcher at Visa.