mixle.epistemic.journal module

EpistemicJournal – an append-only, replayable decision log for a sequence of loop steps.

The program plan’s decision journal (§3.4: “timestamp, model version, belief snapshot hash, options considered with their EIG/cost/risk scores, chosen action, rationale claims…”), following mixle.evolve.ledger.EvolutionLedger’s existing shape (flat, JSON-serializable, append-only, no model objects stored directly) rather than inventing a third bespoke logging pattern.

One deliberate refinement over a hash-only ledger: each record carries the full serialized belief snapshot (portfolio_snapshot, from to_dict()) alongside its content-address (belief_snapshot_hash). A hash alone cannot be reversed back into a portfolio, so “an auditor can reconstruct the full belief trajectory from the ledger alone” (program plan §2) requires the snapshot content to actually be there; the hash is what lets verify() catch tampering/corruption of that stored content, which is the property a bare append-only list doesn’t get for free.

Type fidelity of the payload content within a snapshot (Hypothesis.payload, and the journal’s own action_chosen/action_considered fields, both declared Any) is bounded by what JSON can actually represent. Plain json.dumps/json.loads silently turns a tuple into a list – a different Python value ((1, 2) != [1, 2]) that happens to render as identical JSON text, so a hash computed over that JSON text cannot tell the two apart either. _tag_encode()/ _tag_decode() close this for the types it is small and safe to do so for: JSON-native values plus tuple, numpy scalars (numpy.generic), and numpy arrays round-trip as themselves (see _tag_encode()’s docstring for the exact list and the versioned tag format), and verify() hashes this type-faithful encoding rather than JSON’s lossy shadow of it – a tuple silently replayed as a list now actually fails verification instead of passing it.

Anything else (an arbitrary custom object – e.g. a fitted model used directly as a hypothesis payload, which real callers such as mixle.task.discrepancy_invention_loop.run_discrepancy_invention_loop() do) has no small, general, always-correct JSON encoding of its own; building one is a substantially larger undertaking (already partially addressed, for the types that opt into it, by mixle.utils.serialization) and out of scope for this journal. Such a payload is instead stored as a one-way str() snapshot, with a UserWarning raised at journal time so the caller knows: EpistemicJournal.replay() hands back that string, not the original object, and verify() for such a record only attests that the string is unchanged – not that the original object would be recovered. This keeps the journal usable for arbitrary hypothesis/action payloads without ever silently overclaiming a fidelity it can’t deliver.

class DecisionRecord(step_index, belief_snapshot_hash, portfolio_snapshot, surprise, action_considered=<factory>, action_chosen=None, action_eig=None, timestamp=None, rationale=None, prev_hash='0000000000000000000000000000000000000000000000000000000000000000', record_hash='')

Bases: object

One journaled decision: what was believed, what was considered, what was chosen, and why.

belief_snapshot_hash content-addresses portfolio_snapshot alone. record_hash covers the whole record – surprise, actions considered, action chosen, EIG, timestamp, rationale, step index, and prev_hash – and because it includes prev_hash it is a hash chain: the hash of record i depends on every record before it, so removing, reordering, or splicing records breaks the chain rather than going unnoticed.

Parameters:

• step_index (int)

• belief_snapshot_hash (str)

• portfolio_snapshot (dict)

• surprise (float)

• action_considered (list[Any])

• action_chosen (Any | None)

• action_eig (float | None)

• timestamp (float | None)

• rationale (str | None)

• prev_hash (str)

• record_hash (str)

class EpistemicJournal(records=None)

Bases: object

An ordered, JSON-serializable, replayable log of EpistemicSteps.

See the module docstring for the precise type-fidelity contract: JSON-native values, tuple, numpy scalars, and numpy arrays round-trip through to_json()/from_json() (and replay()) exactly; any other payload type is stored as a one-way str() snapshot (with a UserWarning at journal time) and does not.

Parameters:

records (list[DecisionRecord] | None)

property records: tuple[DecisionRecord, ...]

The journaled records, as a read-only snapshot.

The backing list used to be the public attribute, so a caller could clear(), del, or reorder the journal in place – an append-only decision log that anything holding a reference could rewrite. Indexing, iteration, and len() are unchanged; use append() to add. (Record content is still reachable through the returned records, which is why verify() exists and now covers the whole record and its position in the chain.)

append(step, *, action_considered=(), rationale=None, timestamp=None)

Append one record for step and return it. timestamp is caller-supplied, never sampled here.

belief_snapshot_hash is computed over step.portfolio_after’s type-faithful encoding (see _tag_encode()): a hypothesis payload outside the supported type set is stored anyway (as a one-way str() snapshot) rather than rejected, since real callers journal arbitrary typed hypothesis payloads (e.g. fitted model objects) by design – but a UserWarning is raised so this is never silent.

Parameters:

• step (EpistemicStep)

• action_considered (list[Any])

• rationale (str | None)

• timestamp (float | None)

Return type:

DecisionRecord

replay(portfolio0=None)

Reconstruct the belief trajectory from the journal’s stored snapshots alone.

portfolio0 is accepted for interface symmetry with the loop’s own step(portfolio, ...) signature but is not required for reconstruction here: every record already carries its own full portfolio_snapshot, so replay is deserialization, not re-simulation (re-simulation would additionally need the original observations and likelihood callables, which are deliberately not journaled – they may not be JSON-serializable, and the snapshot is the thing an audit actually needs). If given, portfolio0 is prepended to the returned trajectory.

A hypothesis payload that round-tripped through to_json()/from_json() comes back as its original type for the supported set (JSON-native, tuple, numpy scalar/array); otherwise it comes back as the one-way str() snapshot that was actually stored (see the module docstring). Replaying a journal that was never serialized (no to_json()/from_json() round trip) always returns the original in-memory payload objects unchanged, since nothing was ever encoded.

Parameters:

portfolio0 (HypothesisPortfolio | None)

Return type:

list[HypothesisPortfolio]

head()

The chain’s current tip: the last record’s record_hash, or the genesis anchor if empty.

This is what verify() needs in order to detect truncation. Store it outside the journal’s own storage each time you finish appending, and hand it back to verify(expect_head=...).

Return type:

str

verify(*, expect_length=None, expect_head=None)

Return whether the journal is intact: every record’s content AND its place in the chain.

This used to compare only portfolio_snapshot against belief_snapshot_hash, so everything the log exists to attest – the surprise, the actions considered, the action actually chosen, its EIG, the timestamp, the rationale, the step index – could be rewritten and this still returned True. So could deleting or reordering records, since nothing tied a record to its position or its predecessor.

Four things are now checked per record, in order: its step_index matches its position; its prev_hash matches the previous record’s record_hash (_GENESIS_HASH for the first); its snapshot still matches belief_snapshot_hash; and its record_hash still matches the whole canonical record. Because each record_hash covers prev_hash, the records form a chain: altering record i invalidates every record after it, so no localized edit can be made to verify.

Hashing is over the type-faithful encoding (_tag_encode()), not a lossy JSON shadow of it: two snapshots differing only in payload type (a tuple silently replaced by an equal-valued list) hash differently and correctly fail. For a payload stored as a one-way str() snapshot (see the module docstring), this only proves that string is unchanged, not that the original object would be recovered – there is no stronger claim to make once the exact object is gone.

Truncation from the end cannot be detected from the records alone, and no hash chain can do it. Deleting the final record leaves a shorter chain that is entirely self-consistent: every remaining prev_hash still matches its predecessor and every record_hash still verifies, because a prefix of a valid chain is a valid chain. Catching it requires one fact the journal does not itself hold – how many records there should be, or what the tip was:

head = journal.head() # store this outside the journal … journal.verify(expect_head=head, expect_length=n)

expect_head is the stronger anchor: it pins the exact final record, so it also rejects a journal truncated and then re-extended back to its original length. expect_length alone only notices a change in count. With neither, this still proves internal consistency – but it is not evidence that nothing was dropped off the end, and an auditor should not read it as such.

Parameters:

• expect_length (int | None)

• expect_head (str | None)

Return type:

bool

to_json(**dumps_kwargs)

Serialize the journal as JSON that a strict parser actually accepts.

allow_nan is forced off rather than left at Python’s default. json.dumps otherwise writes bare NaN/Infinity/-Infinity tokens, which no JSON grammar admits: the journal round-tripped fine inside Python (json.loads reads those tokens back) while being unreadable to every other consumer – and an audit trail nobody else can read is the one thing this class exists to prevent. _tag_encode() now carries non-finite floats through the versioned tag scheme, so this switch should never fire; it is here so that a value that escapes the codec fails loudly instead of writing a document that lies about being JSON.

Parameters:

dumps_kwargs (Any)

Return type:

str

classmethod from_json(s)

Rebuild a journal from to_json() output, checking the anchor it carries.

A document written by to_json() is an envelope holding the records plus the length and chain tip they had when written; both are verified here, so a record deleted from the end – which verify() alone cannot see – is caught on load.

A bare JSON array is still accepted for documents written before the envelope existed. Those carry no anchor, so loading one proves only that the records are internally consistent; pass a separately stored expect_head to verify() for the stronger check.

Parameters:

s (str)

Return type:

EpistemicJournal