mixle.substrate.core module¶

Core substrate store and item model.

The substrate is a filesystem-backed, queryable surface for typed items with provenance, scope, freshness metadata, tags, and links. Raw data, documents, model artifacts, traces, simulation outputs, ontology triples, and context packets can all be represented as SubstrateItem records.

Text and document items rank by cosine similarity over a learned embedding when available. Structured records and other items fall back to lexical, tag, and provenance matching. Higher-level retrieval and context assembly build on this single local store.

class SubstrateItem(kind, text='', payload=<factory>, provenance=<factory>, scope='local', tags=<factory>, links=<factory>, id=<factory>, created_at=<factory>)[source]

Bases: object

One typed, provenanced, scoped item in the substrate.

Parameters:

kind (str)
text (str)
payload (dict[str, Any])
provenance (dict[str, Any])
scope (str)
tags (list[str])
links (list[str])
id (str)
created_at (float)

kind: str

text: str = ''

payload: dict[str, Any]

provenance: dict[str, Any]

scope: str = 'local'

tags: list[str]

links: list[str]

id: str

created_at: float

to_json()[source]

Return type:: dict[str, Any]

classmethod from_json(d)[source]

Parameters:: d (dict[str, Any])
Return type:: SubstrateItem

class Substrate(root=None)[source]

Bases: object

A local shard of the knowledge substrate: a filesystem-backed store with typed retrieval.

put / get / remove / all manage items; search retrieves the k most relevant items for a query, filtered by kind and scope, ranking text items semantically (a learned embedding over the current text corpus) and everything else lexically. save / load persist the shard as one items.jsonl under root.

Parameters:: root (str | None)

put(item)[source]

Add or replace an item; returns its id. Marks the semantic index dirty for text items.

Parameters:: item (SubstrateItem)
Return type:: str

add(kind, text='', **kw)[source]

Convenience: build a SubstrateItem and put() it.

Parameters:

kind (str)
text (str)
kw (Any)

Return type:

str

get(item_id)[source]

Parameters:: item_id (str)
Return type:: SubstrateItem | None

remove(item_id)[source]

Parameters:: item_id (str)
Return type:: bool

all(*, kind=None, scope=None)[source]

Parameters:

kind (str | None)
scope (str | None)

Return type:

list[SubstrateItem]

reindex()[source]

(Re)fit the embedding index over the current text-bearing items. Idempotent, lazy-called.

Return type:: None

search(query, k=5, *, kind=None, scope=None)[source]

The k most relevant items to query as (item, score), filtered by kind/scope.

Text-bearing items rank by cosine similarity in the learned embedding space; when there are too few items to learn one (or for a non-text query), ranking falls back to a lexical token overlap. Structured items with no text always rank lexically over their serialized payload + tags.

Parameters:

query (str)
k (int)
kind (str | None)
scope (str | None)

Return type:

list[tuple[SubstrateItem, float]]

save(root=None)[source]

Persist the shard to {root}/items.jsonl (one item per line).

Parameters:: root (str | None)
Return type:: str

load(root=None)[source]

Parameters:: root (str | None)
Return type:: None