Research Areas

The research program combines mathematical and algorithmic foundations of reliable AI along with domain knowledge in three core application domains (as visualised in this figure): medicine & healthcare, robotics & interacting systems, and algorithmic decision-making.

Mathematical and Algorithmic Foundations

Reliability of AI with all its facets can only be achieved through a profound understanding of its foundations. In fact, the current gap between theory and practice of AI methodologies is one of the key obstacles for deriving comprehensive guarantees as required by critical applications. Supporting our goal of reliable AI, the general research challenges we aim to address are twofold.
Firstly, we aim to establish theoretical guarantees for AI. This includes expressivity of AI models, analysis of learning algorithms, generalization capabilities of trained AI systems, and aspects such as robustness, aiming predominantly at concrete error bounds and certification. A particular challenge are novel and highly complex architectures such as graph neural networks or transformers. Secondly, to support reliability, we research algorithmic foundations of AI on relevant topics, such as IT security, federated learning, distributed systems, and causal modeling, thereby ensuring a tight link to the application domains and their practical realization.

Fellows in Foundations


  • Stefan Bauer

    Data Science and Intelligent Systems (TUM, Helmholtz AI)

    Mathematical & Algorithmic Foundations Medicine & Healthcare Algorithmic Decision-Making
  • Pramod Bhatotia

    Decentralized Systems Engineering (TUM)

    Mathematical & Algorithmic Foundations Robotics & Interacting Systems
  • Bernd Bischl

    Statistical Learning & Data Science (LMU)

    Mathematical & Algorithmic Foundations
  • Daniel Cremers

    Computer Vision and Artificial Intelligence (TUM)

    Mathematical & Algorithmic Foundations Medicine & Healthcare Robotics & Interacting Systems
  • Mathias Drton

    Mathematical Statistics (TUM)

    Mathematical & Algorithmic Foundations
  • Vincent Fortuin

    Efficient Learning and Probabilistic Inference for Science (Helmholtz AI)

    Mathematical & Algorithmic Foundations
  • Debarghya Ghoshdastidar

    Theoretical Foundations of AI (TUM)

    Mathematical & Algorithmic Foundations
  • Stephan Günnemann

    Data Analytics & Machine Learning (TUM)

    Mathematical & Algorithmic Foundations
  • Reinhard Heckel

    Machine Learning (TUM)

    Mathematical & Algorithmic Foundations
  • Eyke Hüllermeier

    Artificial Intelligence & Machine Learning (LMU)

    Mathematical & Algorithmic Foundations Algorithmic Decision-Making
  • Georg Kaissis

    AI in Medicine (TUM)

    Mathematical & Algorithmic Foundations Medicine & Healthcare
  • Christoph Kern

    Social Data Science and Statistical Learning (LMU)

    Mathematical & Algorithmic Foundations
  • Niki Kilbertus

    Ethics in Systems Design & Machine Learning (TUM, Helmholtz)

    Mathematical & Algorithmic Foundations Medicine & Healthcare
  • Gitta Kutyniok

    Mathematical Foundations of AI (LMU)

    Mathematical & Algorithmic Foundations
  • Johannes Maly

    Mathematical Data Science (LMU)

    Mathematical & Algorithmic Foundations

Medicine and Healthcare

In relAI, we combine the expertise in AI healthcare and medicine to successfully tackle these challenges. AI has the potential to fundamentally transform the future of medicine and healthcare by enabling earlier and more accurate diagnosis and better treatment, leading to improved outcomes for patients and increased efficiency in healthcare. The emergence of AI for medicine and healthcare also offers a number of transformative opportunities for economic growth. Examples cover prevention and early detection, e.g. AI for wearable devices as well as AI for screening (e.g. mammography).

A key requirement for the successful deployment of AI in clinical environments is the development of safe, secure, and trustworthy ML techniques. In particular, advances are required in robust and data efficient learning, privacy preservation, and interpretable deep learning.

Fellows in Medicine & Healthcare

Robotics and Interacting Systems

Engineers and computer scientists are currently developing autonomous systems with AI techniques as a core component. This provides endless possibilities but also comes with enormous challenges regarding safety, security, and privacy. For example, how to guarantee safety of an autonomous agent (e.g., a robot in a human environment) under all circumstances, given that a designer cannot foresee all situations the agent will face in the future? How to balance the advantages of AI cloud computing with the increased risk of security violations? How to leverage data to adapt to the needs of a human user while bearing privacy concerns in mind? To answer such questions, relAI will focus on safe, secure, and privacy-preserving AI in the context of autonomous agents and interacting systems.

Fellows in Robotics & Interacting Systems

Algorithmic Decision-Making

Ever more applications in AI consider prescriptive modeling in the sense of learning a model that stipulates appropriate decisions or actions to be taken in real-world scenarios: Which medical therapy should be applied? Should this person be hired for the job? Decisions of that kind are increasingly automated and made by algorithms instead of humans, often relying on AI methods. Our ambition is to develop AI-based methodologies for reliable algorithmic decision-making (ADM).

This comes with the need to address specific technical issues such as the lack of an objective “ground truth” underlying every prediction, and learning from partial training information, comprising feedback about the decision made, while lacking information about counterfactuals. Methodological research on ADM will be complemented by more application-oriented research on reliable decisions in business and management.

Fellows in Decision-Making

  • Stefan Bauer

    Data Science and Intelligent Systems (TUM, Helmholtz AI)

    Mathematical & Algorithmic Foundations Medicine & Healthcare Algorithmic Decision-Making
  • Stefan Feuerriegel

    AI in Management (LMU)

    Algorithmic Decision-Making
  • Eyke Hüllermeier

    Artificial Intelligence & Machine Learning (LMU)

    Mathematical & Algorithmic Foundations Algorithmic Decision-Making
  • Göran Kauermann

    Statistics in Economics, Business Administration and Social Sciences (LMU)

    Algorithmic Decision-Making
  • Frauke Kreuter

    Statistics & Data Science in Social Sciences & the Humanities (LMU)

    Algorithmic Decision-Making
  • Volker Tresp

    Database Systems & Data Mining (LMU, Siemens)

    Algorithmic Decision-Making