A gentle introduction to uncertainty quantification

SVG Icon Editor

Author: Lisa Wimmer

Success stories about artificial intelligence (AI) focus on its remarkable predictive power. Take, for instance, your smartphone’s ability to recognize your face on photos and collect them into a “Selfies” folder ready to supply snaps for social media.

When it comes to more safety-critical tasks, like using facial recognition for security at a high-stakes research lab, simple label predictions ("that's you!") won't cut it. There, a mistaken identity carries far bigger risks than confusing you with a friend. If we are to actually deploy predictive models—usually in conjunction with humans—we need some measure of prediction quality so we can step in when the current output can’t be trusted. With every prediction we’d like some sort of confidence assignment, along the lines of “I think it’s most likely A but it could also be B or C”. Such statements are the result of a machine learning subtask called Uncertainty Quantification (UQ).

What we would like to see

Ideally, UQ happens alongside prediction and takes into account different sources of uncertainty. Most researchers in the field agree on the existence of two distinct types: aleatoric and epistemic uncertainty (e.g., Hüllermeier & Waegeman, 2021).

Aleatoric uncertainty is taken to be a property of the data that no amount of skilfull modeling can alleviate. It arises from the fact that the observed features X are not enough to predict the target Y beyond doubt, be it due to measurement errors, random processes in the physical world, or simply because variables other than X are causative for Y. Epistemic uncertainty, on the other hand, relates to a lack of knowledge about the relationship between X and Y. In simple terms, epistemic uncertainty tells us how confident we are that the trained model is the right one. As such, it is reducible (in theory) by observing more data.

However, as Gruber et al. (2023) would argue, other sources of uncertainty are no less important. We might start by challenging the frequently held belief that our data, i.e., given in Latin, reflect an infallible truth. For example, labels in the training data, which are usually assigned by human annotators, can be ambiguous (think of images that contain multiple objects, or natural language with various conceivable interpretations). It might also happen that poorly designed data collection processes introduce a systematic gap between the quantity of interest and what’s actually measured, or that specific types of observations never make it into the dataset (e.g., young people less likely to own a landline will be underrepresented in telephone surveys). Further down the road, most learning processes come with algorithmic uncertainty: finite machine precision, noise introduced by minibatching huge amounts of data, or computations that need to be distributed across several processing units.

What we actually see

You might think that for these powerful new models, which can chat with you about modern Shakespeare adaptations, expressing confidence should be a piece of cake. Indeed, reasoning about uncertainty is second nature to classical statisticians who’ve been designing models for decades. They deal in a currency called probability distributions, and distributions exist to express uncertainty.

However, today's massive AI models often struggle with a few key issues:

  1. a fundamental inability to express (all types of) uncertainty,
  2. a lack of resources to compute proper uncertainty estimates, and
  3. a tendency toward overconfidence.

What we face

Let’s look at each of these challenges in turn.

Expressing uncertainty

Start with aleatoric uncertainty (the observed features X are not sufficient to predict the label Y beyond doubt). Such a scenario is easy to imagine in medical imaging, where inferring patients’ conditions from MRI scans can be hard even for human experts. Enabling a model to express aleatoric uncertainty typically requires training it to output distributional or set-valued predictions. Rather than merely classifying an image as either tumor A, tumor B or tumor-free, it will return a probability distribution (”70% tumor A, 5% tumor B, 25% tumor-free”) or a set (”with ≥ 95% probability it’s one of: tumor A, tumor-free”). Both variants carry information that humans-in-the-loop can ponder in subsequent decisions (send patient home, subject them to further tests, operate immediately?).

Epistemic uncertainty is a tougher nut to crack, or at least a more expensive one. Recall that we want to quantify our confidence about having found the right model. We can’t judge that without considering alternative models that would have been plausible under the given data. If many different models are similarly conceivable, we have high epistemic uncertainty. Typically, this means reasoning about an entire distribution over infinitely many admissible models, adding a layer of (computational) complexity to the problem.

It turns out that dealing with aleatoric and epistemic uncertainty is already so difficult that few researchers have ventured into the more exotic types. Ideas to handle label ambiguity, missing data, algorithmic instability & Co. will hopefully become more numerous in the future.

Calculating uncertainty

You might have noticed by now that much in UQ is about considering multiple alternative hypotheses, and that translates pretty much immediately to higher modeling costs. Extending analyses to further sources of uncertainty only exacerbates the matter. Limited resources, therefore, often force practitioners to resort to crude approximations in UQ. You can picture this endeavor as trying to measure the expansion of the Alps when your (time) budget is just barely enough to walk around the foot of the Matterhorn.

Overconfidence

Lastly, overconfidence is an issue mainly with the fancy large-scale models that have received much of the recent hype. It’s somewhat ironic that the very properties responsible for their success should have a negative effect on UQ capabilities. Loosely speaking, these models succeed by warping the input data, in which the relationship between X and Y is highly complicated, into a new space where it’s suddenly very simple.

Classifying cats vs dogs in images is hard if all you have is RGB values per pixel, but if you manage to sort the images along dimensions like “has whiskers” or “has floppy ears”, you task becomes much easier (humans do this naturally and subconsciously; for a mathematical model emulate that requires a lot of complex design). Cycling back to UQ, the model will get very confident in its predictions when it has found this warped space (or data representation) that separates dogs from cats. This can become problematic, however, if it loses the ability to say “I don’t know”. Imagine feeding the model the image of a bunny: it will happily apply its rules about whiskers and pointy ears to label the bunny as a cat with high certainty. We have ways to address overconfidence, but they are all imperfect because of one fundamental problem: there is no ground-truth uncertainty. All we know is that it’s a bunny, but can we really say if it’s more appropriate to predict 50/50 probability for cat/dog (because a bunny is neither) or perhaps 60/40 (because bunnies look a little more like cats)?

All is not lost

UQ is definitely a thorny problem, yet the recent achievements in AI suggest that we will see innovative and smart solutions in the near future. As technology advances, we can also anticipate the resolution of many computational barriers. It is the desire to incorporate AI in real-world decision-making processes, though, that will drive innovation by sheer need. The path may be shrouded in uncertainty (pun intended), but there will be many brilliant minds to walk it.

RELATED

  • / Valentyn Melnychuk

    Can We Trust the Uncertainty of Causal Foundation Models?

    Causal foundation models promise fast, flexible treatment-effect estimation from observational data, but can their uncertainty be trusted? This post examines prior-data fitted networks (PFNs) for causal inference, revealing a hidden prior-induced confounding bias, and introduces a one-step posterior correction that restores reliable, frequentist-consistent uncertainty for average treatment effect estimation. Full post
  • / Molly Kennedy

    Exploring XAI Methods for Interpretability of Large Language Models

    Large Language Models can summarise text, answer questions, and classify sentiment with impressive accuracy, but their reasoning often feels like a black box. Why did the model land on that prediction? This post walks through four complementary approaches from Explainable AI (XAI) that help open up the box. Together, these methods make LLMs less opaque and more trustworthy. Full post
  • / Tzu-Yuan Huang

    Robotic Decision Making via Diffusion Models

    Robots are increasingly expected to operate in complex, real-world environments, but the machine learning methods that drive their decision-making, like reinforcement and imitation learning, often struggle with training stability and capturing the multimodal nature of behavior. This post introduces diffusion models as a powerful alternative for robotic decision making, explaining their core denoising mechanism and surveying their applications in several contexts. Full post
  • / Lukas Gosch

    A Beginner’s Guide to Certifiable Robustness

    Machine learning models, such as ChatGPT and those used in autonomous driving, are becoming essential tools in our daily lives. However, the existence of "adversarial examples" demonstrates that these systems are not free from vulnerabilities. The post, introduces the concept of adversarial examples and discusses Certifiable Robustness, a methodology designed to combat them. Full post
  • / Balian He

    Responsible Textual Generative Models (Part I): Generating Truthful Content

    This post introduces hallucination in large language models—why fluent outputs can still be wrong, what distinguishes intrinsic from extrinsic errors, and how training data, model design, inference, and alignment contribute. It surveys detection and mitigation approaches and sets up later posts on toxicity, bias, and inclusivity in responsible textual generative AI. Full post
  • / Aswathi

    Random Convolutions: A Simple Way to Boost Generalization

    Random Convolutions (RC) are a simple and effective data augmentation technique designed to improve the robustness of deep learning models, particularly in medical imaging. Instead of relying on fixed or learned filters, RC applies a small stack of randomly generated convolutional filters to input images during training. This introduces strong but controlled variations in image texture while preserving the underlying anatomical structure. By exposing models to a wide range of plausible appearance changes, Random Convolutions reduce overfitting to dataset-specific characteristics such as scanner type or acquisition protocol. In practice, RC acts as an implicit form of domain randomization, encouraging models to focus on stable, shape-based and contextual features rather than fragile texture cues. Despite its simplicity and low computational cost, RC has been shown to substantially improve generalization across institutions and imaging conditions in challenging tasks such as multiple sclerosis lesion segmentation. Full post
  • / Ahmed Abdelrahman

    Neuromorphic Computing: A Brain-inspired Approach to Robot Intelligence

    This post offers a high-level walkthrough of neuromorphic computing, why it matters for energy-efficient robot intelligence, and how SNNs, event cameras, and neuromorphic chips fit together Full post
  • / Shengqiang Zhang

    Introduction to Embodied Instruction Following

    This work explores how AI systems bridge the gap between understanding human instructions and performing real actions by learning through direct interaction with their environment. Full post
  • / Maximilian Fleissner

    From Unlucky Strikers to Statistical Learning Theory

    Using the analogy of football striker, this post explains how statistical learning theory helps us understand when we can trust model predictions and empirical averages, highlighting the challenges of generalization in modern machine learning, especially with complex models like deep neural networks. Full post
  • / Unai Fischer Abaigar

    Performative Prediction

    Machine learning systems are increasingly used to support decision-making processes. Yet, these systems do not merely reflect the world—they also reshape it. Once deployed, predictions can influence behaviors, alter policies, and redirect resources, creating feedback loops that change the very data-generating processes they aim to model. Full post
  • / Moritz Knolle

    What even is differential privacy?

    A concise introduction to differential privacy which offers provably privacy guarantees for training machine learning models. Full post
  • / Sameer Ambekar

    Mitigating Domain shifts

    Adapting a deep neural network to unseen data and tasks is imperative these days, however access to target data is often available. Common target adaptation techniques including domain adaptation and generalization train for meaningful representations during source training. Recent paradigms such as Test-time training/adaptation focus on optimizing the source model on unseen data. To do so, they finetune the model on the streaming unsupervised data which is useful for practical scenarios. Moreover, these techniques can be applied to variety of tasks such as regression, classification and segmentation. Full post
  • / Blog Editorial Team

    Welcome to the relAI Blog

    Welcome to the relAI blog of the Konrad Zuse School of Excellence in Reliable AI (relAI). This blog will serve as a platform to share cutting-edge research and developments from our school, highlighting the significant strides we are making towards making AI systems safer, more trustworthy, and privacy-preserving. Full post
LOAD MORE