mixle.reason package

mixle.reason – the cross-modal scientific-reasoning front door.

A scientific question is a query on a joint posterior over a shared latent that every modality is evidence about. This package wires that idea into one call:

answer = reason(prior, [evidence_from_modality_1, evidence_from_modality_2, …]) answer.mean, answer.interval(0.9) # a posterior, with honest error bars answer.attribution() # which modality sharpened the belief (nats) answer.predict(H, R).epistemic # split a prediction’s uncertainty (epi vs aleatoric)

The exact core here is linear-Gaussian: each modality contributes a linear-Gaussian observation y = H z + noise(R) and the beliefs fuse by exact Kalman assimilation (a product of experts). Non-linear / learned encoders (Phase 3+, and application-specific forward models in the sibling mixle_pde package) plug in by producing such evidence – a linearized (H, y, R) or a Gaussian expert – so the front door is stable while the encoders grow underneath it.

Design: notes/mixle-cross-modal-reasoning-design.md. Built on mixle.inference.belief (the belief state) and mixle.inference.uncertainty (the epistemic/aleatoric split).

class Ontology(classes=<factory>, relations=<factory>, axioms=<factory>, disjoint=<factory>)

Bases: object

A typed schema over knowledge: class hierarchy + relation signatures + axioms + disjointness.

Parameters:

• classes (dict[str, str | None])

• relations (dict[str, tuple[str, str]])

• axioms (dict[str, set[str]])

• disjoint (list[tuple[str, str]])

classes: dict[str, str | None]

relations: dict[str, tuple[str, str]]

axioms: dict[str, set[str]]

disjoint: list[tuple[str, str]]

add_class(name, parent=None)

Parameters:

• name (str)

• parent (str | None)

Return type:

Ontology

add_relation(name, domain, range_, *axioms)

Parameters:

• name (str)

• domain (str)

• range_ (str)

• axioms (str)

Return type:

Ontology

add_disjoint(a, b)

Parameters:

• a (str)

• b (str)

Return type:

Ontology

is_a(cls, ancestor)

Whether cls is ancestor or a descendant of it (walks the parent chain).

Parameters:

• cls (str)

• ancestor (str)

Return type:

bool

check_triple(h, r, t, types)

Every named violation of (h, r, t) given entity types ({} means unconstrained).

Parameters:

• h (str)

• r (str)

• t (str)

• types (dict[str, str])

Return type:

list[str]

check_graph(triples, types)

Audit a triple set: per-triple violations plus the cross-triple axioms (functional/asymmetric).

Returns {consistent, n_triples, violations: [{triple, problems}]} – every problem named.

Parameters:

• triples (Any)

• types (dict[str, str])

Return type:

dict[str, Any]

filter_triples(triples, types)

Split triples into (kept, rejected-with-reasons) by per-triple consistency – the decode mask.

Parameters:

• triples (Any)

• types (dict[str, str])

Return type:

tuple[list[tuple], list[dict[str, Any]]]

class OntologyConstrainedKG(kg, ontology, *, entities, relations, types)

Bases: object

A fitted KG embedding, typed by an ontology: probability mass only on schema-consistent triples.

Wraps a KnowledgeGraphDistribution (entities and relations as integer indices) together with the symbolic ontology and the index<->name maps. The tail posterior is masked to entities whose class conforms to the relation’s range and renormalized, so completion can never propose an ontology-violating tail – Graph(ontology) as a distribution.

Parameters:

• kg (Any)

• ontology (Ontology)

• entities (list[str])

• relations (list[str])

• types (dict[str, str])

tail_posterior(head, relation)

p(tail | head, relation) over ONLY the range-conforming entities (renormalized).

Parameters:

• head (str)

• relation (str)

Return type:

dict[str, float]

complete(head, relation)

The most probable ontology-consistent tail (or None when the range admits no entity).

Parameters:

• head (str)

• relation (str)

Return type:

tuple[str, float] | None

constrained_decode(llm, prompt, ontology, types, *, n=None, floor=0.5, calibrator=None)

Ontology-constrained decoding (D2): an LLM may only assert schema-consistent, confident facts.

Samples llm (a GraphLLM) n times, masks every sampled graph through Ontology.filter_triples() (violating triples are REJECTED with named reasons – the typed-grammar mask applied post-parse), then marginalizes the constrained graphs into a GraphDistribution and keeps only facts whose edge marginal clears floor – the calibrated confidence floor (pass a fitted calibrator from fit_fact_calibrator() to apply the floor on CALIBRATED truth probability rather than the raw marginal). Consistent-but-underconfident facts are reported as withheld, never silently dropped: the decode says what it refused to assert and why.

Parameters:

• llm (Any)

• prompt (str)

• ontology (Ontology)

• types (dict[str, str])

• n (int | None)

• floor (float)

• calibrator (Any)

Return type:

ConstrainedDecode

class ConstrainedDecode(facts, rejected, below_floor, n_samples)

Bases: object

The result of ontology-constrained LLM decoding: what survived, what the schema rejected, and why.

Parameters:

• facts (list[tuple[Any, float]])

• rejected (list[dict[str, Any]])

• below_floor (list[tuple[Any, float]])

• n_samples (int)

facts: list[tuple[Any, float]]

rejected: list[dict[str, Any]]

below_floor: list[tuple[Any, float]]

n_samples: int

asserted()

Return type:

list[Any]

as_dict()

Return type:

dict[str, Any]

class NonlinearEvidence(h, y, R, jacobian=None, iterations=2, name='')

Bases: object

One modality’s evidence through a NONLINEAR forward model: y = h(z) + noise.

Assimilated by (iterated) extended-Kalman linearization: at the current belief mean m the forward is replaced by its tangent h(z) ~ h(m) + J(m)(z - m) and the exact linear update runs on that tangent; with iterations > 1 the linearization point is refined at the updated mean and the update repeats FROM THE PRE-UPDATE BELIEF (the iterated EKF), which matters when the prior mean is far from the truth. jacobian is analytic when you have it; otherwise a central finite difference is used. Honest caveat: this is a Gaussian approximation around the linearization point – for strongly multimodal posteriors it reports one mode’s belief, not the mixture.

Parameters:

• h (Any)

• y (Any)

• R (Any)

• jacobian (Any)

• iterations (int)

• name (str)

h: Any

y: Any

R: Any

jacobian: Any = None

iterations: int = 2

name: str = ''

class DiscreteAnswer(belief, attribution=<factory>)

Bases: object

The posterior over hypotheses plus per-source attribution (nats of entropy removed).

Parameters:

• belief (CategoricalBelief)

• attribution (list[tuple[str, float]])

belief: CategoricalBelief

attribution: list[tuple[str, float]]

property probs: ndarray

map()

Return type:

Any

top(k=3)

Parameters:

k (int)

Return type:

list[tuple[Any, float]]

summary()

Return type:

str

decide(loss, actions=None, *, abstain_cost=None)

The Bayes-optimal action under this posterior — EXACT over the finite hypothesis set.

Parameters:

• loss (Any) – an (A, K) matrix (loss[a, k] = cost of action a when hypothesis k is true) or a callable loss(action, hypothesis) -> float.

• actions (Any) – action labels (defaults to the hypothesis labels — the “declare k” actions).

• abstain_cost (float | None) – when given, an extra "abstain" action with this flat cost — chosen whenever every committal action’s expected loss exceeds it (the escalate-don’t-guess decision, priced explicitly).

Returns:

{action, expected_loss, alternatives} with the exact expected loss of every candidate.

Return type:

dict[str, Any]

model_evidence(name, models, x)

Evidence from fitted mixle models: hypothesis k <-> models[k], scored on observation x.

Returns (name, log_lik) with log_lik[k] = models[k].log_density(x).

Parameters:

• name (str)

• models (Any)

• x (Any)

Return type:

tuple[str, ndarray]

reason_discrete(prior, evidence)

Fold evidence into a categorical belief and return the posterior with per-source attribution.

Parameters:

• prior (Any) – a CategoricalBelief, an int K (uniform over K), or a list of hypothesis labels (uniform over them).

• evidence (Any) – a sequence of (name, log_lik_vector) pairs — e.g. from model_evidence() — assimilated in order by exact Bayes.

Return type:

DiscreteAnswer

reason(prior, evidence, *, query=None)

Fuse evidence into prior by exact Kalman assimilation; return the queried posterior.

Parameters:

• prior (Any) – the latent’s prior belief (GaussianBelief; build one with Latent).

• evidence (Any) – a sequence of LinearGaussianEvidence and/or NonlinearEvidence – one per modality / observation. Nonlinear items assimilate by iterated-EKF linearization (a Gaussian approximation; see NonlinearEvidence). They are folded in one at a time (order does not affect the result), and the nats each removes are recorded for ReasonedAnswer.attribution().

• query (Any) – optional latent coordinate indices to restrict the answer to.

Returns:

A ReasonedAnswer – the posterior belief plus attribution and prediction UQ.

Return type:

ReasonedAnswer

class Latent

Bases: object

Factories for the shared latent’s prior belief (the starting point of assimilation).

static gaussian(mean, cov)

A Gaussian prior N(mean, cov) over the latent.

Parameters:

• mean (Any)

• cov (Any)

Return type:

GaussianBelief

static vector(dim, *, mean=0.0, var=1.0)

An isotropic Gaussian prior over a dim-vector latent: N(mean*1, var*I).

Parameters:

• dim (int)

• mean (float)

• var (float)

Return type:

GaussianBelief

static mechanistic(A, steps, *, x0_mean=None, x0_cov=None, process_cov=None)

A physics-constrained prior over a latent trajectory z_0 .. z_{steps-1}.

The trajectory follows a linear dynamical law z_{t+1} = A z_t + w_t, w_t ~ N(0, Q) – a discretized linear ODE/PDE (A is the state-transition operator; take it from a mixle_pde DynamicsOperator for real physics). The returned belief is the exact joint Gaussian over the stacked trajectory (steps * d,) (block t is z_t); its block-tridiagonal precision is the mechanistic prior. Because the states are coupled, evidence at any one time informs all times through the dynamics – fusing observations via reason() is then exact Kalman smoothing, so a sparsely-observed field is filled in by the physics, not by a generic smoothness assumption.

Parameters:

• A (Any) – (d, d) linear state-transition operator (one discrete step).

• steps (int) – number of time steps T in the trajectory.

• x0_mean (Any) – mean of z_0 (default zeros).

• x0_cov (Any) – covariance of z_0 (default identity).

• process_cov (Any) – process-noise covariance Q (default zeros – deterministic dynamics).

Return type:

GaussianBelief

Evidence

alias of LinearGaussianEvidence

class LinearGaussianEvidence(H, y, R, name='')

Bases: object

One modality’s evidence about the latent z: y = H z + noise, noise ~ N(0, R).

H is the (possibly linearized) forward operator mapping the latent to this modality’s measurement space, y the observed data, R its noise covariance (matrix, diagonal, or scalar). Application forward models (e.g. mixle_pde geophysics operators) produce these.

Parameters:

• H (Any)

• y (Any)

• R (Any)

• name (str)

H: Any

y: Any

R: Any

name: str = ''

block_selector(step, n_blocks, block_dim, within=None)

An observation matrix that reads time-block step of a stacked trajectory latent.

For a latent built by Latent.mechanistic() (shape (n_blocks * block_dim,)), returns the H selecting block step – use it to build LinearGaussianEvidence for an observation at that time. within optionally reads only part of the block (a (k, block_dim) local readout); by default the whole block is read (identity).

Parameters:

• step (int)

• n_blocks (int)

• block_dim (int)

• within (Any)

Return type:

ndarray

class ReasonedAnswer(belief, prior_entropy, contributions)

Bases: object

A posterior belief about a query, with the UQ a scientific answer needs.

Beyond mean / interval / entropy (delegated to the belief), it exposes attribution() – the nats of uncertainty each modality removed – and predict(), which splits a prediction’s uncertainty into epistemic (from latent uncertainty) and aleatoric (observation noise) via the law of total variance.

Parameters:

• belief (BeliefState)

• prior_entropy (float)

• contributions (dict[str, float])

property mean: ndarray

cov()

Return type:

ndarray

sd()

Return type:

ndarray

entropy()

Return type:

float

interval(level=0.9)

Per-coordinate central credible interval at level (an (d, 2) array of [lo, hi]).

Parameters:

level (float)

Return type:

ndarray

information_gain()

Total nats of uncertainty the evidence removed from the prior (H[prior] - H[posterior]).

Return type:

float

attribution(*, normalize=False)

Per-modality information gain in nats – which modality sharpened the belief, and by how much.

With normalize=True, values are the fraction of the total gain (they then sum to ~1).

Parameters:

normalize (bool)

Return type:

dict[str, float]

predict(H, R=0.0)

Split the uncertainty of a new prediction y* = H z + noise(R) (law of total variance).

epistemic = diag(H P Hᵀ) (from the latent’s remaining uncertainty, reducible by more data) and aleatoric = diag(R) (irreducible observation noise). Exact for the Gaussian belief.

Parameters:

• H (Any)

• R (Any)

Return type:

UncertaintyDecomposition

marginal(indices)

Restrict the answer to a subset of latent coordinates (query a specific variable).

Parameters:

indices (Any)

Return type:

ReasonedAnswer

class GaussianBelief(mean, cov)

Bases: BeliefState

A multivariate-Gaussian belief N(mean, cov) over a continuous latent.

Evidence is a linear-Gaussian observation y = H z + noise, noise ~ N(0, R); update() applies the exact Kalman measurement update (Joseph form, so the covariance stays symmetric positive-definite). fuse() combines two beliefs about the same latent as a product of Gaussian experts. condition() does noiseless Gaussian conditioning on a coordinate subset.

Parameters:

• mean (Any)

• cov (Any)

property dim: int

mean()

The posterior mean of the latent.

Return type:

ndarray

cov()

The posterior covariance (not defined for every realization).

Return type:

ndarray

entropy()

The differential/Shannon entropy H[q] (nats) – watch it shrink as evidence arrives.

Return type:

float

interval(level=0.9)

Per-coordinate central credible interval at level – an (d, 2) array of [lo, hi].

Parameters:

level (float)

Return type:

ndarray

sample(n=1, rng=None)

Draw n latent samples from the belief.

Parameters:

• n (int)

• rng (Any)

Return type:

ndarray

update(H, y, R)

Kalman measurement update: condition on y = H z + noise, noise ~ N(0, R).

Parameters:

• H (Any) – (k, d) observation matrix (or (d,) / scalar for a single linear readout).

• y (Any) – (k,) observed value (or scalar).

• R (Any) – (k, k) observation-noise covariance (or (k,) diagonal / scalar).

Return type:

GaussianBelief

fuse(other)

Product-of-experts fusion of two beliefs about the same latent (cross-modal fusion).

Equivalent to conditioning self on other treated as a direct Gaussian observation (H = I, R = other.cov), so it reuses the exact Kalman update.

Parameters:

other (GaussianBelief)

Return type:

GaussianBelief

condition(indices, values)

Noiseless Gaussian conditioning: fix latent coordinates indices to values.

Returns the belief over the remaining coordinates. This is the exact R -> 0 limit of an observation that reads off those coordinates.

Parameters:

• indices (Any)

• values (Any)

Return type:

GaussianBelief

marginal(indices)

The belief restricted to a subset of latent coordinates.

Parameters:

indices (Any)

Return type:

GaussianBelief

class BeliefState

Bases: ABC

A distribution over a latent, exposing a uniform query + update interface.

Realizations answer where they are defined: mean(), cov(), var(), sd(), entropy(), interval(), sample(), marginal(), and – the point of a belief state – update(), which returns a new belief conditioned on fresh evidence.

abstractmethod mean()

The posterior mean of the latent.

Return type:

ndarray

abstractmethod entropy()

The differential/Shannon entropy H[q] (nats) – watch it shrink as evidence arrives.

Return type:

float

abstractmethod sample(n=1, rng=None)

Draw n latent samples from the belief.

Parameters:

• n (int)

• rng (Any)

Return type:

ndarray

abstractmethod update(*args, **kwargs)

Return a new belief conditioned on fresh evidence (the assimilation step).

Parameters:

• args (Any)

• kwargs (Any)

Return type:

BeliefState

cov()

The posterior covariance (not defined for every realization).

Return type:

ndarray

var()

Per-coordinate posterior variance.

Return type:

ndarray

sd()

Per-coordinate posterior standard deviation.

Return type:

ndarray

interval(level=0.9)

Per-coordinate central credible interval at level – an (d, 2) array of [lo, hi].

Parameters:

level (float)

Return type:

ndarray

marginal(indices)

The belief restricted to a subset of latent coordinates.

Parameters:

indices (Any)

Return type:

BeliefState

class AcquisitionPlan(items=<factory>, total_cost=0.0, total_gain=0.0, belief=None)

Bases: object

A budgeted evidence-acquisition plan: the chosen items, the nats they bought, and the final belief.

Parameters:

• items (list[tuple[int, str, float, float]])

• total_cost (float)

• total_gain (float)

• belief (Any)

items: list[tuple[int, str, float, float]]

total_cost: float = 0.0

total_gain: float = 0.0

belief: Any = None

property indices: list[int]

select_evidence_batch(store, belief, *, budget, query=None, fine_cost=1.0, coarse_cost=0.2, fidelities=('coarse', 'fine'), candidates=None, max_items=None, min_gain=1e-9)

Greedily acquire the most-informative-per-cost (item, fidelity) evidence under a total budget.

At each step every remaining candidate is scored – at each allowed fidelity – by the entropy it would remove from the query given the belief so far, divided by its cost; the best affordable one is folded in. Adaptive re-scoring means a batch never double-counts overlapping evidence. Stops when nothing affordable helps.

Parameters:

• store (CrossModalStore)

• belief (Any)

• budget (float)

• query (Any)

• fine_cost (float)

• coarse_cost (float)

• fidelities (Sequence[str])

• candidates (Sequence[int] | None)

• max_items (int | None)

• min_gain (float)

Return type:

AcquisitionPlan

as_belief(obj, node=None)

Adapt any object exposing mean/cov (a FieldPosterior node, a fitted Gaussian, a ParameterPosterior) into a GaussianBelief.

node is forwarded when the source is node-addressable (e.g. FieldPosterior.mean(node) / .cov(node)); otherwise mean/cov are called with no argument.

Parameters:

• obj (Any)

• node (Any)

Return type:

GaussianBelief

class CrossModalStore(keys, payloads, *, coarse, fine, metric='euclidean')

Bases: object

A corpus indexed by embedding keys, with raw payloads conditioned on when embeddings fall short.

Parameters:

• keys (Any) – (N, d_key) embedding vectors – the retrieval index (routers, not answers).

• payloads (Sequence[Any]) – length-N sequence of raw items (arbitrary; passed to coarse/fine).

• coarse (Callable[[Any], LinearGaussianEvidence]) – payload -> LinearGaussianEvidence at embedding fidelity (cheap, lossy).

• fine (Callable[[Any], LinearGaussianEvidence]) – payload -> LinearGaussianEvidence at raw fidelity (precise, “expensive”).

• metric (str) – "euclidean" (default) or "cosine" for retrieval.

retrieve(query_key, k=8)

Indices of the k corpus items whose embedding keys are nearest query_key.

Parameters:

• query_key (Any)

• k (int)

Return type:

list[int]

assimilate(belief, query_key, *, k=8, query=None, epsilon=0.0)

Retrieve k neighbors and fold each into belief, fetching raw payloads when the embedding is too lossy for the query.

For each retrieved item the sufficiency test compares how much the raw evidence would reduce the query entropy versus the embedding evidence; if the surplus exceeds epsilon the raw payload is used, else the cheap embedding is. Returns the updated belief and a per-item provenance trail.

Parameters:

• belief (GaussianBelief)

• query_key (Any)

• k (int)

• query (Any)

• epsilon (float)

Return type:

tuple[GaussianBelief, list[RetrievalStep]]

next_evidence(belief, *, query=None, candidates=None, fidelity='fine')

Active retrieval: the corpus item whose evidence most reduces the query entropy (EIG).

Returns (index, expected_gain_nats). fidelity selects the fine (raw) or coarse (embedding) evidence builder for the look-ahead.

Parameters:

• belief (GaussianBelief)

• query (Any)

• candidates (Sequence[int] | None)

• fidelity (str)

Return type:

tuple[int, float]

class RetrievalStep(index, fidelity, gain)

Bases: object

Provenance for one assimilated item: which corpus index, at what fidelity, and the nats it removed.

Parameters:

• index (int)

• fidelity (str)

• gain (float)

index: int

fidelity: str

gain: float

class LLMUncertainty(generate, *, equivalent=None, n=10)

Bases: object

Calibrated uncertainty and selective prediction for any generate(prompt) -> str LLM.

Parameters:

• generate (Callable[[str], Any]) – callable(prompt) -> str – one stochastic sample from the model.

• equivalent (Callable[[Any, Any], bool] | None) – callable(a, b) -> bool deciding whether two answers mean the same thing (default exact match; pass a normalizer / embedding / entailment check for real text).

• n (int) – default number of samples per prompt.

sample(prompt, n=None)

Draw n stochastic responses to prompt.

generate may return a plain string, or a (text, logprob) pair – the sequence log-probability log P(s). When logprobs are provided they are used to marginalize the string distribution over meaning classes exactly (mixle.inference.marginalize_meaning()) rather than by sample counting.

Parameters:

• prompt (str)

• n (int | None)

Return type:

list[Any]

assess(prompt, n=None)

Sample, marginalize the string distribution over meaning classes, and report the answer.

The reported confidence is the marginal probability of the top meaning (summed over its equivalence class of strings), and semantic_entropy the entropy of that meaning marginal – not a per-string token probability.

Parameters:

• prompt (str)

• n (int | None)

Return type:

LLMAssessment

decompose(prompts, n=None)

Epistemic/aleatoric split across member prompts (paraphrases of one question).

Each prompt is a member; all members’ samples are pooled to define shared meaning-clusters, then each member’s distribution over those clusters feeds decompose_entropy(). Epistemic = disagreement across paraphrasings (prompt-sensitivity / model uncertainty); aleatoric = within-member spread.

Parameters:

• prompts (Sequence[str])

• n (int | None)

Return type:

UncertaintyDecomposition

assess_claims(prompt, *, extract=None, corroborates=None, n=None, threshold=0.5)

Score the reliability of each claim in the response by cross-sample corroboration.

Finer-grained than assess(): a response can be internally consistent (low semantic entropy) yet contain one fabricated fact. This decomposes the response into claims and checks each unit of information separately – a claim the model knows recurs across independent resamples; a hallucinated one appears once. This is UQ on the information in what is said, not just on the answer as a whole.

Parameters:

• prompt (str) – the query.

• extract (Callable[[str], Sequence[str]] | None) – response -> [claim, ...] (default sentence_claims()).

• corroborates (Callable[[str, str], bool] | None) – (other_sample, claim) -> bool – does a resample support the claim? (default content_overlap(); pass an entailment/NLI check for real text).

• n (int | None) – number of samples (the first is the response scored; the rest corroborate).

• threshold (float) – support below which a claim is flagged as unreliable/fabricated.

Return type:

InformationAssessment

calibrate(examples, *, correct=None, alpha=0.1, n=None)

Calibrate a confidence threshold for selective risk <= alpha on labeled (prompt, gold).

For each example, the model’s answer (majority meaning-cluster) and its confidence are computed; correct(answer, gold) (default the equivalent relation) marks it right or wrong. The threshold is the lowest confidence at which the selective error rate on the calibration set is <= alpha – so answer() abstains below it and, when it answers, is right with probability about 1 - alpha.

Parameters:

• examples (Sequence[tuple[str, Any]])

• correct (Callable[[Any, Any], bool] | None)

• alpha (float)

• n (int | None)

Return type:

LLMUncertainty

answer(prompt, n=None)

Answer prompt if confident enough, else None (abstain).

Requires a prior calibrate(). Returns the LLMAssessment when confidence >= threshold (so the answer meets the selective-risk guarantee), else None.

Parameters:

• prompt (str)

• n (int | None)

Return type:

LLMAssessment | None

fit_factuality(examples, *, signal=None, correct=None, method='isotonic', n=None)

Learn a calibrated P(answer is correct) from a raw signal, on labeled (prompt, gold).

The model’s raw confidence (its self-consistency, or a token likelihood) is not a probability that the information is correct – it can be systematically over/under-confident, or unrelated to truth. This fits a ProbabilityCalibrator mapping the signal to the empirical correctness rate, so the output is a probability of the information being right. discrimination (AUC of signal vs correctness) reports how much the raw signal knew at all – ~0.5 means it was unrelated to truth, calibration or not.

Parameters:

• examples (Sequence[tuple[str, Any]]) – labeled (prompt, gold_answer) pairs.

• signal (Callable[[str], float] | None) – prompt -> float raw score (default: the self-consistency confidence from assess()).

• correct (Callable[[Any, Any], bool] | None) – (answer, gold) -> bool (default the equivalent relation).

• method (str) – calibration map – "isotonic" or "platt".

• n (int | None) – samples per prompt.

Return type:

FactualityModel

class LLMAssessment(answer, confidence, semantic_entropy, clusters, samples)

Bases: object

One prompt’s assessed answer with uncertainty.

answer is the majority meaning-cluster’s representative; confidence its cluster share in [0, 1]; semantic_entropy the nats of meaning-uncertainty; clusters the [(representative, probability), ...] distribution over meanings; samples the raw draws.

Parameters:

• answer (Any)

• confidence (float)

• semantic_entropy (float)

• clusters (list[tuple[Any, float]])

• samples (list[Any])

answer: Any

confidence: float

semantic_entropy: float

clusters: list[tuple[Any, float]]

samples: list[Any]

class ClaimAssessment(claim, support, reliable)

Bases: object

Reliability of one claim inside a response, by cross-sample corroboration.

support is the fraction of independent resamples that corroborate the claim (in [0, 1]); reliable is support >= threshold. A claim the model actually knows recurs across samples (high support); a fabricated one appears once and vanishes (low support).

Parameters:

• claim (str)

• support (float)

• reliable (bool)

claim: str

support: float

reliable: bool

class InformationAssessment(claims, reliability)

Bases: object

UQ over the information content of a response: every claim scored, plus a summary.

claims is the per-claim reliability; reliability the mean support (how trustworthy the response’s information is overall); fabricated the claims below threshold (likely hallucinated).

Parameters:

• claims (list[ClaimAssessment])

• reliability (float)

claims: list[ClaimAssessment]

reliability: float

property fabricated: list[ClaimAssessment]

class FactualityModel(calibrator, signal, discrimination)

Bases: object

A fitted map from a per-prompt uncertainty signal to a calibrated P(answer is correct).

The signal (self-consistency, a token likelihood, …) is only a raw number; the calibrator turns it into a genuine probability of the information being correct, learned against labeled facts. discrimination (held-out AUC on the fit set) reports how much the signal actually knew about correctness – ~0.5 means the signal was unrelated to truth, no matter how confident it looked.

Parameters:

• calibrator (ProbabilityCalibrator)

• signal (Callable[[str], float])

• discrimination (float)

calibrator: ProbabilityCalibrator

signal: Callable[[str], float]

discrimination: float

probability(prompt)

Calibrated probability that the answer’s information is correct.

Parameters:

prompt (str)

Return type:

float

sentence_claims(text)

Split a response into atomic claims (sentence-ish units) – the default claim extractor.

Parameters:

text (str)

Return type:

list[str]

content_overlap(sample, claim, *, threshold=0.6)

Simple corroboration test: does sample cover >= threshold of claim’s content words?

Counts every content word equally, so boilerplate shared across responses (“the tower is located in …”) can mask that the informative word (the city) differs. information_corroborator() fixes that by weighting words by their information content; it is the default in LLMUncertainty.assess_claims().

Parameters:

• sample (str)

• claim (str)

• threshold (float)

Return type:

bool

information_corroborator(samples, *, overlap=0.5)

Build a corroboration test that weights each word by its information content over samples.

A word appearing in nearly every sample is boilerplate (low information, low weight); a rare word carries the actual claim (high weight). A sample corroborates a claim when it covers at least overlap of the claim’s information-weighted words – so whether the distinctive fact (a city, a number, a name) matches drives the decision, not the shared filler. Inverse-document-frequency weighting: w(word) = log((N + 1) / (df + 0.5)).

Parameters:

• samples (Sequence[str])

• overlap (float)

Return type:

Callable[[str, str], bool]

class GraphLLM(generate, parse, *, n=10)

Bases: object

Turn a generate(prompt) -> str LLM into a distribution over knowledge graphs.

Parameters:

• generate (Callable[[str], str]) – callable(prompt) -> str – one stochastic generation.

• parse (Callable[[str], Iterable[Any]]) – callable(str) -> iterable[triple] – extract the asserted facts (semantic parse / structured-output decode). Generations that parse to the same triple-set are the same meaning – exact equality, no fuzzy matching.

• n (int) – default number of samples per prompt.

sample_graphs(prompt, n=None)

Sample n generations and parse each into a canonical graph.

Parameters:

• prompt (str)

• n (int | None)

Return type:

list[frozenset]

distribution(prompt, n=None, *, log_probs=None, graphs=None)

Sample, parse, and marginalize strings onto graphs -> a GraphDistribution.

Marginalization uses Monte-Carlo counting by default (P(G) = fraction of samples that parse to G); pass log_probs (one log P(string) per sample) to instead sum the sequence likelihoods within each graph – the lower-variance, estimator-correct form that does not assume every string realizing a graph is equiprobable.

Parameters:

• prompt (str)

• n (int | None)

• log_probs (Sequence[float] | None)

• graphs (Sequence[frozenset] | None)

Return type:

GraphDistribution

class GraphDistribution(graphs, probs)

Bases: object

A distribution over knowledge graphs – the LLM’s belief about the information.

graphs are the distinct canonical graphs observed; probs[i] = P(graphs[i]) is the string distribution marginalized onto graphs (so it sums to 1 over distinct graphs). Every query is answered by marginalizing this distribution over the graphs that produce the queried outcome.

Parameters:

• graphs (list[frozenset])

• probs (ndarray)

graphs: list[frozenset]

probs: ndarray

marginalize(outcome)

P(outcome = c) = sum_{G : outcome(G) = c} P(G) – marginalize over subgraphs.

outcome maps a graph to a hashable value (a fact’s object, a boolean property, an aggregate). Returns [(value, probability), ...] sorted by descending probability.

Parameters:

outcome (Callable[[frozenset], Hashable])

Return type:

list[tuple[Any, float]]

entropy(outcome)

Entropy (nats) of the marginal P(outcome = .) – the model’s uncertainty about that query.

Parameters:

outcome (Callable[[frozenset], Hashable])

Return type:

float

edge_marginals()

P(triple in G) for every triple – the per-fact reliability (a KG edge posterior).

Return type:

dict[tuple, float]

fact_probability(triple)

P(triple in G) for one fact (0 if never asserted).

Parameters:

triple (Any)

Return type:

float

calibrated_edge_marginals(calibrator)

Edge marginals mapped through a fitted calibrator -> a calibrated P(fact is true).

A raw edge marginal is the model’s internal assertion rate for a fact, not a probability that the fact is true – a confidently-hallucinated fact has a high marginal yet is false. Fit the calibrator with fit_fact_calibrator() on labeled facts, then this reports, per fact, the empirical truth rate at that marginal. (Its residual limit – confident hallucinations that look exactly like known facts – is why an external check is needed; see the validation tests.)

Parameters:

calibrator (ProbabilityCalibrator)

Return type:

dict[tuple, float]

query(*prefix)

Answer-completion posterior: P(object | prefix) over triples whose leading fields match.

query("eiffel", "city") marginalizes over graphs, collecting the objects of every triple starting ("eiffel", "city", ...) weighted by P(G), then renormalizes over the objects actually asserted. Returns [(object, probability), ...] best-first.

Parameters:

prefix (Any)

Return type:

list[tuple[Any, float]]

most_likely_graph()

The single most probable graph and its probability.

Return type:

tuple[frozenset, float]

canonical_graph(triples)

Canonical form of a graph: the frozenset of its triples (order-independent, dedup, hashable).

Parameters:

triples (Iterable[Any])

Return type:

frozenset

fit_fact_calibrator(distributions, truth, *, method='isotonic')

Fit edge marginal -> P(fact is true) over the facts asserted across many graph distributions.

Turn the model’s internal assertion rate (the edge marginal) into a calibrated probability of truth, learned against ground-truth labels. Collect every (triple, marginal) the model asserts, label it with truth(triple), and fit a ProbabilityCalibrator.

This does NOT rescue confident hallucination – a fact the model reliably confabulates has a high marginal indistinguishable from a genuinely-known one, so calibration lowers the overall fact-ECE but cannot pull those specific facts down. Separating them needs a signal external to the model (retrieval / a checker); the validation tests quantify both the gain and this residual.

Parameters:

• distributions (Iterable[GraphDistribution])

• truth (Callable[[Any], bool])

• method (str)

Return type:

ProbabilityCalibrator

Submodules

• mixle.reason.core module
• mixle.reason.design module
• mixle.reason.discrete module
• mixle.reason.embedding module
• mixle.reason.encoder module
• mixle.reason.graph_llm module
• mixle.reason.llm module
• mixle.reason.model module
• mixle.reason.ontology module
• mixle.reason.store module