mixle.substrate.interop module

External model interop for reasoner delegation.

ExternalModel wraps a generate callable from an external model, agent, hosted LLM, or remote tool. It can estimate semantic uncertainty by sampling multiple answers and clustering them through an equivalence function.

external_action() adapts the wrapper into a reasoner Action. When the external model is above its uncertainty cutoff, the action contributes no evidence, allowing the reasoner to continue or abstain.

class ExternalAnswer(prompt, answer, entropy, confident)[source]

Bases: object

An external model’s answer plus its self-measured uncertainty (semantic entropy).

Parameters:
prompt: Any
answer: Any
entropy: float
confident: bool
class ExternalModel(generate, *, calibration_prompts=None, equivalent=None, max_entropy=None, alpha=0.1, samples=8)[source]

Bases: object

An external generate callable wrapped so each answer carries semantic-entropy UQ.

Parameters:

  • generate (Callable[[Any], Any]) – prompt -> answer (an external agent / LLM / remote tool). Called multiple times per query to measure how much its meaning varies (the uncertainty signal).

  • calibration_prompts (Any) – optional example prompts; the (1-alpha) quantile of their semantic entropy becomes the “too uncertain” cutoff. Without them, max_entropy must be given (or every answer is treated as confident).

  • equivalent (Callable[[Any, Any], bool] | None) – (a, b) -> bool meaning-equivalence for clustering samples (default: exact match).

  • max_entropy (float | None) – an explicit uncertainty cutoff, overriding the calibrated one.

  • samples (int) – how many resamples to draw when measuring entropy.

  • alpha (float)

property max_entropy: float
answer(prompt)[source]

Call the external model and attach its semantic-entropy UQ (confident iff below the cutoff).

Parameters:

prompt (Any)

Return type:

ExternalAnswer

confident(prompt)[source]
Parameters:

prompt (Any)

Return type:

bool

external_action(model, *, name='external', cost=8.0, description='', trust_uncertain=False)[source]

A reasoner DELEGATE action backed by a UQ-wrapped external model (see module docstring).

By default (trust_uncertain=False) the action contributes evidence ONLY when the external model is confident about the query; an uncertain external answer yields no fragment, so the reasoner treats it as no answer rather than a guess. The fragment carries the model’s entropy so the trace records how sure the external source was. Cost defaults high – external calls are the escalation of last resort.

Parameters:

  • model (ExternalModel)

  • name (str)

  • cost (float)

  • description (str)

  • trust_uncertain (bool)

Return type:

Any