mixle.substrate.accum module

Measure whether accumulated calibrated knowledge improves held-out answers.

The knowledge-accumulation flywheel measures whether adding calibrated knowledge to the belief store (mixle.substrate.belief) improves answers on a held-out question set, with no model retraining. Distillation improves the student model; accumulation improves what the student or teacher can retrieve.

Two guards keep the measurement grounded:

  • attribution – the improvement must disappear when the newly-assimilated items are withheld from retrieval, proving the gain came from the store growing rather than anything else (timing, caching, a lucky answer_fn).

  • credence weighting – retrieval goes through mixle.substrate.belief.retrieve_beliefs(), which ranks by relevance * credence and can be hard-thresholded with min_credence; a batch of low-credence (e.g. pure MODEL_ASSERTION) knowledge must not inflate the measured improvement – it is down-weighted, never treated as ground truth.

class QAItem(question, answer)[source]

Bases: object

One held-out question: answer_fn is judged correct on it if it produces answer from the retrieved context alone.

Parameters:
class FlywheelMeasurement(solve_rate, grounded_fraction)[source]

Bases: object

Held-out answer quality and grounding rate for one flywheel measurement.

Parameters:
class FlywheelReport(before, after, withheld, attribution_confirmed)[source]

Bases: object

Before/after/withheld flywheel measurements with an attribution check.

Parameters:

  • before (FlywheelMeasurement)

  • after (FlywheelMeasurement)

  • withheld (FlywheelMeasurement)

  • attribution_confirmed (bool)

measure_flywheel(sub, questions, answer_fn, assimilate_batch, *, k=5, min_credence=None)[source]

Measure answer_fn against questions before and after assimilate_batch(sub) adds a batch of calibrated beliefs (returning the ids it touched), with a THIRD measurement that excludes exactly those ids from retrieval – the attribution control. answer_fn and every other piece of the system stay fixed throughout: only the store’s content (and what it makes retrievable) changes.

Parameters:
Return type:

FlywheelReport