Uncertainty
===========

``mixle`` has several uncertainty surfaces, all with the same design bias:
uncertainty should change behavior. It should decide whether to answer,
escalate, collect more data, or report which evidence source mattered.

This page covers:

* uncertainty over LLM answers;
* claim-level reliability for generated text;
* epistemic versus aleatoric decomposition;
* the ``uq`` dispatcher for fitted models, point predictors, ensembles, and
  LLM-like callables;
* conformal answer-or-abstain behavior;
* cross-modal latent evidence fusion;
* calibrated task cascades.

Unified ``uq`` Dispatcher
-------------------------

``mixle.inference.uq`` provides one front door when the caller owns a
heterogeneous predictor and wants Mixle to choose an uncertainty route from the
object's capabilities.

.. code-block:: python

   from mixle.inference import uq

   fitted_uq = uq(model, training_rows)
   interval_uq = uq(point_predictor, (x_cal, y_cal), alpha=0.1)
   llm_uq = uq(generate, example_prompts, alpha=0.1)

``UQResult`` exposes method-specific accessors:

* ``sample_models`` and ``credible_interval`` for fitted Mixle models through a
  Laplace parameter posterior;
* ``interval`` and ``epistemic_std`` for point predictors or ensembles through
  split conformal calibration;
* ``semantic_entropy`` and ``confident`` for LLM-style generation callables.

Use the specialized APIs below when you already know the exact uncertainty
method. Use ``uq`` when the application boundary should accept several kinds
of predictor behind one call.

LLMUncertainty
--------------

``LLMUncertainty`` wraps any stochastic callable:

.. code-block:: text

   generate(prompt) -> answer

It samples multiple answers, clusters them by meaning, and returns the majority
meaning plus a confidence and semantic entropy.

.. code-block:: python

   from mixle.reason import LLMUncertainty

   def equivalent(a, b):
       return str(a).strip().lower() == str(b).strip().lower()

   uq = LLMUncertainty(generate, equivalent=equivalent, n=20)
   assessment = uq.assess("Which city is the Eiffel Tower in?")

   print(assessment.answer)
   print(assessment.confidence)
   print(assessment.semantic_entropy)
   print(assessment.clusters)

High confidence means most samples fell into the same meaning cluster. High
semantic entropy means the model is disagreeing with itself about the answer,
not just rephrasing it.

Equivalence Matters
-------------------

The default equivalence relation is exact equality. That is fine for labels or
normalized short answers. For prose, pass a domain-specific relation:

.. code-block:: python

   import re

   def normalize_city(text):
       return re.sub(r"[^a-z]", "", text.lower())

   uq = LLMUncertainty(
       generate,
       equivalent=lambda a, b: normalize_city(a) == normalize_city(b),
       n=20,
   )

In production this relation might use canonicalization, embeddings, entailment,
or a task-specific parser.

Epistemic and Aleatoric Split
-----------------------------

Use several prompts as ensemble members, for example paraphrases of the same
question. ``decompose`` separates disagreement across members from spread
within each member.

.. code-block:: python

   dec = uq.decompose(
       [
           "Who discovered penicillin?",
           "Name the scientist credited with discovering penicillin.",
           "Penicillin was discovered by whom?",
       ],
       n=10,
   )

   print(dec.epistemic)
   print(dec.aleatoric)
   print(dec.total)

Epistemic uncertainty is reducible uncertainty: model or prompt sensitivity.
Aleatoric uncertainty is within-member ambiguity: the question or output space
itself remains variable.

Calibrated Answer-or-Abstain
----------------------------

Sampling confidence is still only a signal until calibrated. ``calibrate`` uses
labeled examples to choose the lowest confidence threshold whose answered set
has empirical error at most ``alpha``.

.. code-block:: python

   examples = [
       ("Capital of France?", "Paris"),
       ("2 + 2?", "4"),
   ]

   uq.calibrate(examples, alpha=0.1)

   answer = uq.answer("Capital of Japan?")
   if answer is None:
       escalate_to_human_or_frontier_model()
   else:
       print(answer.answer)

After calibration, ``answer`` returns ``None`` below the threshold. This is the
important behavioral change: the LLM can abstain instead of hallucinating.

Claim-Level Reliability
-----------------------

A response can have a stable headline answer and still contain one fabricated
detail. ``assess_claims`` takes one sampled response, extracts claims, and
checks whether independent samples corroborate each claim.

.. code-block:: python

   info = uq.assess_claims(
       "Summarize the contract renewal and include dates and parties.",
       threshold=0.6,
   )

   print(info.reliability)
   for claim in info.fabricated:
       print(claim.claim, claim.support)

Defaults:

* claim extraction is sentence-like splitting;
* corroboration uses information-weighted content overlap across samples.

For serious text, pass your own extractor or entailment-based corroborator:

.. code-block:: python

   info = uq.assess_claims(
       prompt,
       extract=my_claim_extractor,
       corroborates=my_entailment_check,
   )

Uncertainty Helpers
-------------------

The underlying decomposition functions are available from
``mixle.inference.uncertainty``:

.. code-block:: python

   from mixle.inference.uncertainty import (
       Clustering,
       UncertaintyDecomposition,
       cluster_samples,
       decompose_entropy,
       decompose_uncertainty,
       decompose_variance,
       marginalize_meaning,
       posterior_ensemble,
       predictive_distribution,
       semantic_entropy,
   )

Use them when you already have samples, probability vectors, or prediction
ensembles and do not need the LLM wrapper.

``UncertaintyDecomposition`` is the shared result object for epistemic,
aleatoric, and total uncertainty summaries. ``predictive_distribution`` and
``posterior_ensemble`` build ensemble-style predictive objects from fitted or
posterior models, ``decompose_variance`` performs the variance analogue of the
entropy split, and ``marginalize_meaning`` aggregates probabilities over
semantic clusters represented by ``Clustering``.

Cross-Modal Reasoning
---------------------

``mixle.reason.reason`` fuses evidence into a shared latent belief. Each
evidence source is a linear-Gaussian observation:

.. code-block:: text

   y = H z + noise,  noise ~ N(0, R)

Example:

.. code-block:: python

   import numpy as np
   from mixle.reason import Evidence, Latent, reason

   prior = Latent.vector(2, mean=0.0, var=10.0)
   evidence = [
       Evidence(np.array([[1.0, 0.0]]), np.array([2.0]), 0.2, name="sensor-a"),
       Evidence(np.array([[0.0, 1.0]]), np.array([-1.0]), 0.5, name="sensor-b"),
   ]

   ans = reason(prior, evidence)

   print(ans.mean)
   print(ans.interval(level=0.9))
   print(ans.information_gain())
   print(ans.attribution(normalize=True))

``ReasonedAnswer`` exposes:

* posterior mean and covariance;
* credible intervals;
* total information gain;
* per-modality attribution;
* prediction-level epistemic/aleatoric variance split.

Mechanistic Latents
-------------------

``Latent.mechanistic`` builds a Gaussian prior over a trajectory constrained by
a linear dynamical law. Evidence at one time step updates all time steps through
the dynamics.

.. code-block:: python

   A = np.array([[1.0, 0.1], [0.0, 1.0]])
   prior = Latent.mechanistic(A, steps=20, process_cov=0.01 * np.eye(2))

Use ``block_selector`` from ``mixle.reason.core`` to observe a specific time
block of the stacked trajectory.

Task Calibration
----------------

For local task models, uncertainty becomes an answer/escalate decision through
``CalibratedTaskModel`` and ``Cascade``:

.. code-block:: python

   from mixle.task import CalibratedTaskModel, Cascade

   model = CalibratedTaskModel(student, alpha=0.1).calibrate(cal_x, cal_y)
   cascade = Cascade(model, teacher)
   y = cascade("new request")

See :doc:`task-distillation` for the full serving workflow.

Related Reasoning Workflows
---------------------------

This page focuses on uncertainty behavior. See :doc:`reasoning-systems` for the
broader ``mixle.reason`` stack: finite-hypothesis reasoning, cross-modal
retrieval with raw-data fallback, graph-producing LLMs, evidence acquisition,
amortized modality encoders, and trainable cross-modal latent models.

Choosing the Right Tool
-----------------------

.. list-table::
   :header-rows: 1

   * - Need
     - Use
   * - Does the LLM know the answer?
     - ``LLMUncertainty.assess`` and semantic entropy
   * - I have a fitted model or predictor and want Mixle to pick a UQ route
     - ``mixle.inference.uq``
   * - Should the LLM answer or abstain?
     - ``LLMUncertainty.calibrate`` and ``answer``
   * - Which claim in this answer is suspect?
     - ``LLMUncertainty.assess_claims``
   * - Is uncertainty due to prompt/model sensitivity?
     - ``LLMUncertainty.decompose``
   * - How do multiple modalities update a latent?
     - ``reason``, ``Evidence``, ``Latent``
   * - Should a local task model escalate?
     - ``CalibratedTaskModel`` and ``Cascade``