XiJia Zhou (Symbolic Systems) | Building Pedagogical Agents For Social Reinforcement Learning (Human and Machine Cognition Lab, Technische Universitat Darmstadt)

Image

I spent three incredible months as a visiting researcher at the Human and Machine Cognition Lab at the Technical University of Darmstadt, led by Professor Charley Wu. Supported by the GRIP fellowship, my visit focused on building computational models of teaching agents: artificial agents that learn from one another in simulated environments, as a window into how knowledge passes between individuals—between teachers and students, caregivers and children, and perhaps across generations and cultures. 

My dissertation research at Stanford uses reinforcement learning (RL) to model how infants form attachments with their caregivers and how these early relationships shape learning. Professor Wu's lab studies the next steps of this question—how people generalize what they learn from initial experience and how knowledge spreads socially—which made it an ideal host for extending my work. For my GRIP project, I built on a previous study from the lab investigating how a naive "learner" agent can acquire useful skills simply by observing an experienced "teacher" agent navigating a grid world to find rewards. My extension asked: what changes when the teacher is genuinely pedagogical? Instead of acting as a passive expert who simply performs the task well, my teacher agents adjust their own behavior to make things easier for the learner to understand. 

I implemented and tested these teaching agents in code, then compared how different naive learners picked up knowledge from pedagogical versus non-pedagogical teachers. Next, we want to test how well that knowledge transferred when the environment changed. The results speak to a question that bridges machine learning and psychology: can simple, helpful teaching behaviors transmit a surprisingly rich understanding of the world? For me, this is ultimately a step toward understanding the benefits of caregiving relationships, and it opens a path for my dissertation to connect attachment theory with the lab's frameworks for social learning and cultural transmission. 

Beyond the experiment itself, one of the most valuable parts of the visit was engaging with other lab members and the broader cognitive science community in Darmstadt and beyond. Regular meetings with Professor Wu, lab meetings, department journal clubs, and countless informal conversations left me with a deeper appreciation of the field, a much clearer sense of my own research identity, and invaluable memories and friendships.

Life outside of research was just as memorable. I fell for the local food scene—Handkäse mit Musik, the Hessian specialty of sour-milk cheese with pickled onions on rye bread, along with plenty of grilled fish and excellent kebabs. I spent weekends walking in nearby parks under beautiful trees, and evenings at the climbing gym right outside the office with colleagues; the lab even went on a hike together to Frankenstein Castle, where early scientific "experiments" were once conducted. During my stay, I also caught Darmstadt's lively music and sports scenes, attending a music festival and watching a marathon of runners of all ages making their way through the city (with music from local bands!). Darmstadt had a wonderful mix of calmness and energy, and it made everyday life outside the lab a great joy. On weekends, I would also hop on a train and visit nearby Frankfurt.

I returned to Stanford with new models, new collaborators, and a sharpened dissertation direction. Professor Wu and I plan to continue our collaboration toward publication, and I hope this visit marks the beginning of a lasting connection between Stanford and the computational cognitive science community in Germany. I am deeply grateful to the Europe Center, the Stanford Club of Germany, and the GRIP program for making it possible. TEST