Teaching robots to do just about anything, from assembling components in an industrial setting to cooking a meal in one’s home, is very challenging. And if those robots have to move and act in a natural-looking way in the process, it is a far more difficult job yet. That is not always necessary — an industrial robot, for instance, need not worry about appearances. But any robot that has direct interactions with humans has to get its act together or it will be perceived as something between awkward and frightening.
The robots of the Walt Disney theme parks cannot go around scaring guests away, so the engineers at Disney Research have been working on a method that makes natural-feeling interactions more practical for real-world deployment. Their approach, called AMOR (Adaptive Character Control through Multi-Objective Reinforcement Learning), builds on the common practice of reinforcement learning. But where reinforcement learning algorithms are typically very computationally-intensive and fiddly, AMOR is optimized to significantly reduce time spent in processing and manual tweaking.
An overview of the approach (📷: L. Alegre et al.)
Conventional reinforcement learning systems use a carefully weighted sum of reward functions to guide a robot’s behavior. These rewards often conflict — for example, minimizing energy usage while maximizing movement precision — making it difficult to strike the right balance. Engineers have traditionally had to spend hours tuning these weightings by trial and error before training even begins. Worse yet, if the result is not quite right, they have to go back and start over.
AMOR upends this approach by introducing a multi-objective framework that conditions a single policy on a wide range of reward weights. Instead of committing to one balance of rewards from the outset, AMOR allows those weights to be selected after training. This flexibility lets engineers quickly iterate, adapting the robot’s behavior in real time without needing to retrain from scratch.
These characteristics make this approach especially useful in robotics, where a policy trained in simulation often performs poorly in the real world due to the sim-to-real gap. Subtle differences in physical dynamics, sensor accuracy, or motor responsiveness can make previously optimized policies fail. AMOR’s adaptability makes it much easier to bridge that gap, allowing real-world adjustments without expensive retraining cycles.
It has also been demonstrated that AMOR can be embedded in a hierarchical control system. In this setup, a high-level policy dynamically adjusts the reward weights of the low-level motion controller based on the current task. For example, during a fast movement, the controller might emphasize speed over smoothness. During a delicate gesture, the balance might shift in the opposite direction. This not only improves performance but also adds a degree of interpretability to the system’s internal decision-making.
The result is a controller that can execute a wide range of motions — from high-speed jumps to precise, emotive gestures — with lifelike fluidity and responsiveness. AMOR not only improves how robots behave, but also how quickly and flexibly they can be taught to do so. For a place like Disney, where realism, reliability, and rapid development are all crucial, AMOR could prove to be very helpful in bringing animated characters to life with far less friction.