MSc
Biosketch
Rafael Zimmer is a computer scientist with a strong background in high-performance computing and quantitative finance. He earned his Bachelor’s degree from USP, Brazil, and is currently pursuing a Master’s at the Technical University of Munich. His previous work includes developing deep reinforcement learning architectures for high-frequency trading, building limit order book simulators, and working with on-policy algorithms for decision making in market microstructure. Professionally, he has designed quantitative models and trading frameworks at Clave Capital, and RoboBanker, and worked as a researcher for CNPq and FAPESP, in Brazil.
relAI Research
Equilibrium Policies: Leveraging Deep Equilibrium Models and Dynamical Systems Theory for Safe and Reliable Reinforcement Learning
Deep Reinforcement Learning policies are parameterized operators, which applied repeatedly to dynamical systems, generate trajectories in state space. Policies trained using DRL are, however, intrinsically brittle, sensitive to distributional shifts, and commonly have unpredictable failure modes. Reframing the task of DRL using dynamical systems theory, specifically through the lenses of Deep Equilibrium Models could generate safer paths and additional interpretability in provably safe policies. DEQs use the implicit theorem to transform neural network forward passes into a fixed-point solve. This formulation offers a structured prior that naturally aigns with recursive self-consistency of the Bellman equations. The research question at hand is to analyze policies parameterized using DEQ policy architectures and how the built-in property of convergence and stability guarantee from monotone operator theory and contraction analysis could add safety to DRL tasks.