mixle.data.encoded_io module

Serialize encoded (seq_encode) data to disk, so an expensive encode is done once and reused.

seq_encode turns raw records into an encoder-specific payload (nested tuples of NumPy arrays). Fitting re-encodes every call; for large datasets that is the dominant cost. save_encoded/load_encoded persist the encoded payload (with an integrity digest and the encoder’s identity) so subsequent fits load it directly. Format is pickle (the payloads are internal numeric structures); a content digest is stored and verified on load, and the encoder’s class name is recorded so a payload is not silently loaded against a mismatched encoder.

save_encoded(encoded, path, *, encoder=None)

Write encoded (the output of encoder.seq_encode(...)) to path; return its hex digest.

encoder (optional) records the encoder class so a load against a different encoder is flagged.

Parameters:

• encoded (Any)

• path (str)

• encoder (Any)

Return type:

str

load_encoded(path, *, encoder=None)

Load encoded data written by save_encoded(), verifying its integrity digest.

If encoder is given, its class must match the one recorded at save time (else ValueError).

Parameters:

• path (str)

• encoder (Any)

Return type:

Any