Multi-Head Attention has become ubiquitous in modern machine learning architectures, but how much efficiency can still be gained? This question was the focus of Dr. Maximilian Baust’s talk, "Beyond Transformers: Why Beating Multi-Head Attention is Hard."
In his presentation, Dr. Baust explored potential solutions for improving efficiency, ranging from implementation strategies and algorithmic modifications to new architectures, including spiking neural networks.
Dr. Maximilian Baust serves as Director of Solution Architecture Industries EMEA at NVIDIA and is also an industry mentor for one of relAI’s PhD students.
We extend our gratitude to Dr. Baust for sharing his insights and to our director, Gitta Kutyniok, for inviting him to relAI.
/ mc
We are proud to announce that relAI Fellow Prof. Solveig Vieluf won the 2024 Young Investigator Award of the American Epilepsy Society (AES). Her abstract, titled “Seizure Monitoring with Combined Diary and Wearable Data - a Multicenter, Longitudinal, Observational Study”, was selected from over 1,500 submissions for this honor. This award recognizes 20 young investigators conducting basic, translational, or clinical epilepsy research.
She presented her work at the AES Annual Meeting in early December 2024.
Congratulations!
/ mc
The recent relAI Collab Accelerator Workshop brought together researchers to share their work, explore new ideas, and identify potential collaborations. Here's a brief overview of the event:
The day began with the participants pitching their research topic from 9:00 to 11:00, followed by a coffee break until 11:15. After the break, participants engaged in one-to-one sessions until 14:00, followed by lunch, discussion, and feedback.
Participants contributed diverse topics in the field of Machine Learning and Artificial Intelligence. Mohamed Amine Ketata discussed Generative AI for Graphs, Max Beier presented on Learning Operator of Dynamical Systems, Richard Schwank explored Robust Aggregation through the Geometric Median, and Yurou Liang delved into Differentiable Learning for Causal Discovery.
The workshop was fertile ground for generating new research ideas and possible collaborations. During the one-to-one discussions, participants identified several projects for cooperation, such as principled modifications of loss functions to enhance robustness against outlier data rows.
Participants gained new insights into their research during the event. For example, one participant discovered a probabilistic approach to their forecasting issue without relying on a model. Another learned about structure learning as it applies to tabular data, which provided a temporal interpretation of the data. One researcher was challenged about the convexity of their problem. Discussions highlighted intriguing applications of median aggregation techniques to abstract spaces, connecting concentration inequalities with uncertainty quantification.
The relAI Collab Accelerator Workshop was an enriching experience, offering a platform for researchers to connect, share insights, and pave the way for future collaborations. The feedback during this first iteration will help refine the format and make it even more engaging. We are looking forward to the next iteration!
relAI thanks Max Beier and Richard Schwank for their initiative and the organization of the event.
/ mc
Congratulations!
The recent work of relAI PhD student Lukas Gosch has won the Best Paper Award at the 3rd AdvML-Frontiers workshop at the 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024). The workshop and paper presentation took place at the Vancouver Convention Center in Canada on December 14th, 2024.
Lukas is a PhD student at relAI, advised by the relAI Co-Director Prof. Dr. Stephan Günnemann. His research focuses on robust and reliable machine learning, as well as machine learning on graphs.
The award-winning paper „Provable Robustness of (Graph) Neural Networks Against Data Poisoning and Backdoor Attacks“, that Lukas authored together with Mahalakshmi Sabanayagam and relAI Fellows Debarghya Ghoshdastidar and Stephan Günnemann, develops the first architecture-aware certification technique for common neural networks against poisoning and backdoor attacks.
Our sincerest congratulations to Lukas and his co-authors on this achievement!
/ mc
relAI is proud to announce that relAI fellow Prof. Daniel Rückert has been awarded the Gottfried Wilhelm Leibniz Prize 2025. This prestigious award, regarded as the most important German research prize, is endowed with 2.5 million euros by the German Research Foundation (DFG).
The professor of Artificial Intelligence (AI) in Medicine and Healthcare at the Technical University of Munich (TUM) has been honored for his research on AI-assisted medical imaging. He has developed pioneering methods with which AI algorithms can generate particularly informative images from computer tomography or magnetic resonance imaging, analyze them, and interpret them for improved medical diagnostics.
At relAI, he focuses on the safe and privacy aspects of the Medicine & Healthcare research area. His research encompasses several key topics, including reliable machine learning for medical imaging and sensing, privacy-preserving AI, trustworthy medical foundation models, and safe and responsible clinical AI. As a member of the relAI Steering Committee, he represents the Medicine & Healthcare area, playing a crucial role in shaping the organization's goals and future directions of the school.
We are happy to invite you to the upcoming Munich AI Lecturefeaturing two distinguished researchers Prof. Holger Hoos from RWTH Aachen University and Prof. Franca Hoffmann from California Institute of Technology. The lecture is organized by the Chair of Mathematics of Information Processing with support by MCML.
In the first talk, “Dynamics of Strategic Agents and Algorithms as PDEs“, Prof. Hoffmann will speak about dynamics of interactions between algorithms and a population.
In the second talk, “Learning, reasoning and optimisation: Adversarial robustness of neural networks”, Prof. Hoos will discuss robustness of neural networks and its verification.
Event Details:
Speakers: Prof. Dr. Holger Hoos and Prof. Dr. Franca Hoffmann
Date and Time: December 17, 2024, 16:00 CET (16:00-17:00 talk by Prof. Hoffmann and 17:30-18:30 talk by Prof. Hoos – We will have a small break with coffee/tea and snacks in between)
Franca Hoffmann obtained her master’s in mathematics from Imperial College London (UK) and holds a PhD from the Cambridge Centre for Analysis at University of Cambridge (UK). She held the position of von Kármán instructor at Caltech from 2017 to 2020, then joined University of Bonn (Germany) as Bonn Junior Professor and Quantum Leap Africa in Kigali, Rwanda (African Institute for Mathematical Sciences) as AIMS-Carnegie ResearchChair in Data Science, before arriving at the California Institute of Technology as Assistant Professor in 2022.
Bio Holger Hoos
Holger H. Hoos holds an Alexander von Humboldt professorship in AI at RWTH Aachen University (Germany), where he also leads the AI Center, as well as a professorship in machine learning at Universiteit Leiden (the Netherlands) and an adjunct professorship in computer science at the University of British Columbia (Canada). He is a Fellow of the Association of Computing Machinery (ACM), the Association for the Advancement of Artificial Intelligence (AAAI) and the European AI Association (EurAI), past president of the Canadian Association for Artificial Intelligence, former editor-in-chief of the Journal of Artificial Intelligence Research (JAIR) and chair of the board of CLAIRE, an organization that seeks to strengthen European excellence in AI research and innovation (claire-ai.org).
relAI is a co-organiser of the Munich AI lectures. Find more info on these other upcoming events on the Munich AI lectures home page.
Enkelejda Kasneci is a Distinguished Professor (“Liesel Beckmann Distinguished Professorship”) for Human-Centered Technologies for Learning at the TUM School of Social Sciences & Technology. Her research focuses on Human-Computer Interaction and developing AI systems that sense and infer the user’s cognitive state, expertise, actions, and intentions based on multimodal data.
Prof. Dr. Enkelejda Kasneci together with her colleague Prof. Dr Tina Seidel, both directors of the TUM EdTech Center, received the Heinz Maier-Leibnitz Medal 2024 in recognition of their “outstanding research in the field of digital teaching and learning as well as their commitment to establishing the TUM Center for Educational Technologies.”
Congratulations on this achievement!
/ mc
We are excited to announce that the call for applications to the PhD program 2025 of our Konrad Zuse School of Excellence in Reliable AI (relAI) is now open!
The novel, innovative PhD relAI program offers a cross-sectional training for successful education in AI including scientific knowledge, professional development courses and industrial exposure, providing a coherent, yet flexible and personalised training.
Funded applicants will receive a full salary for three years, including social benefits (TV-L E13 of the German public sector). They are further supported by travel grants, e.g. for conference attendance, research stays or home travel. Doctoral students are hosted by a relAI Fellow who helps them to define their research project. Depending on the affiliation of this hosting fellow they enrol at TUM or LMU.
We highly encourage you to apply if you have:
an excellent master’s degree (or equivalent) in computer science, mathematics, engineering, natural sciences or other data science/machine learning/AI related disciplines;
a genuine interest to work on a topic of reliable AI covering aspects such as safety, security, privacy and responsibility in one relAI’s research areas Mathematical & Algorithmic foundations, Algorithmic Decision-Making, Medicine & Healthcare or Robotics & Interacting Systems;
With the Deutscher Zukunftspreis, the German Federal President acknowledges outstanding achievements in technology, engineering, and science, as well as software and algorithm-based innovations that significantly expand the international state of research and technology and are already being used in practice. The prestigious award is endowed with 250,000 euros.
Björn Ommer and his team have developed a groundbreaking Stable Diffusion Model for image generation, which serves as a foundational technology for many AI models, including those from Google and OpenAI.
We’re proud to have Prof. Ommer in our relAI family.
The report opens with remarks from Markus Blume, Bavarian Minister for Science and Arts, alongside our relAI directors Stephan Günnemann (TUM) and Gitta Kutyniok (LMU). It provides an insightful overview of the key components of our relAI school, including our research, educational initiatives, and organizational structure. Most importantly, it highlights the vibrant “relAI family” — our students, fellows, and our partners from both industry and academia.
Additionally, the report offers a comprehensive overview of the activities carried out by relAI during its first two years. It outlines our relAI events, public engagement initiatives, sustainability efforts, and notable achievements within this timeframe. A dedicated chapter features perspectives from relAI members, showcasing insights from relAI fellows, industry partner SAP, and our students, each sharing their unique viewpoints on the relAI experience.
Enjoy reading!
Link to the report: https://zuseschoolrelai.de/home/reports/
