mixle.data.schema module

Schema and logical types – the bridge between external data and the Python types encoders expect.

A deliberately small, closed logical type system (not an open type algebra). Each FieldType knows its canonical NumPy dtype and how to coerce a raw value into the Python object the existing DataSequenceEncoder already consumes (a label for Categorical, an np.ndarray for Vector, …). A Schema is an ordered tuple of named Field s; it can be derived from a model (formalizing the fields/sources duck-probe in the DataFrame adapter) and used to conform raw records (coerce + validate) before encoding – the thing connectors silently get wrong today.

class FieldType

Bases: object

Base logical type: a canonical NumPy dtype plus a coercion to the encoder-ready Python value.

numpy_dtype

alias of float64

coerce(value)

Return value in the Python representation expected by the corresponding encoder.

Parameters:

value (Any)

Return type:

Any

class Real

Bases: FieldType

A real-valued scalar.

numpy_dtype

alias of float64

coerce(value)

Coerce a finite, non-Boolean value to a Python float.

Parameters:

value (Any)

Return type:

float

class Integer

Bases: FieldType

A signed integer logical type.

numpy_dtype

alias of int64

coerce(value)

Coerce an exact, finite, non-Boolean integer.

Parameters:

value (Any)

Return type:

int

class Count

Bases: FieldType

A non-negative integer count logical type.

numpy_dtype

alias of int64

coerce(value)

Coerce value to a non-negative Python int, validating exactness first.

Bare int(value) used to silently truncate (1.9 -> 1), silently accept True/ False (bool is an int subclass in Python) as 1/0, and silently accept negative values – turning a data problem into a wrong-but-plausible-looking observation instead of an error. A numeric string ("42") is still accepted, since that is common, legitimate connector input (CSV/SQL text columns); a bool, a non-integral value, a negative value, or a non-finite (NaN/Inf) value is rejected instead of silently coerced.

Parameters:

value (Any)

Return type:

int

class Boolean

Bases: FieldType

A boolean flag.

numpy_dtype

alias of bool

coerce(value)

Coerce value to a Python bool.

Parameters:

value (Any)

Return type:

bool

class Text

Bases: FieldType

A free-text string.

numpy_dtype

alias of object_

coerce(value)

Coerce value to str.

Parameters:

value (Any)

Return type:

str

class Categorical(categories=None, numpy_dtype=<class 'numpy.object_'>)

Bases: FieldType

A categorical label, optionally over a fixed set of categories.

Parameters:

• categories (tuple[Any, ...] | None)

• numpy_dtype (Any)

numpy_dtype

alias of object_

coerce(value)

Return value after validating membership in categories when categories are fixed.

Parameters:

value (Any)

Return type:

Any

class Vector(dim=None, numpy_dtype=<class 'numpy.float64'>)

Bases: FieldType

A fixed- or free-length real vector.

Parameters:

• dim (int | None)

• numpy_dtype (Any)

numpy_dtype

alias of float64

coerce(value)

Coerce value to a one-dimensional float64 array and validate dim when set.

A multi-dimensional input is flattened in C order ONLY when dim is set and the element count matches it. Declaring the flat length is what makes the reshape explicit rather than silent: sources that emit blocks – PatchSampler hands back a patch_size-shaped patch – have a genuinely rectangular observation whose model-facing form is the flat vector, and with no field kind for N-D arrays the alternative was to refuse them outright. Without a declared dim the shape is still an error, because then nothing states what length the caller expected and a silent flatten could hide a transposed or mis-sliced block.

Parameters:

value (Any)

Return type:

ndarray

class Timestamp

Bases: FieldType

A point in time (datetime / numpy datetime64 / ISO string / POSIX seconds).

coerce(value)

Coerce value to numpy.datetime64 unless it already has that representation.

Parameters:

value (Any)

Return type:

Any

class Optional(inner=<factory>)

Bases: FieldType

A value that may be missing (None passes through; otherwise the inner type coerces).

Parameters:

inner (FieldType)

property numpy_dtype: Any

Return the NumPy dtype of the wrapped field type.

coerce(value)

Pass missing None through; otherwise delegate coercion to inner.

Parameters:

value (Any)

Return type:

Any

class Nested(schema, numpy_dtype=<class 'numpy.object_'>)

Bases: FieldType

A sub-record with its own Schema.

Parameters:

• schema (Schema)

• numpy_dtype (Any)

numpy_dtype

alias of object_

coerce(value)

Conform a nested record with this field’s child schema.

Parameters:

value (Any)

Return type:

Any

class Field(name, type)

Bases: object

A named, typed column.

Parameters:

• name (str)

• type (FieldType)

class Schema(fields, container_value=False)

Bases: object

An ordered set of typed fields.

Parameters:

• fields (tuple[Field, ...])

• container_value (bool)

property names: tuple[str, ...]

Return field names in schema order.

conform_record(record)

Coerce one record (a scalar for a 1-field schema, else a tuple/dict) to the schema’s types.

Parameters:

record (Any)

Return type:

Any

conform(records)

Coerce every record in records to this schema (raising clear errors on mismatch).

Parameters:

records (Any)

Return type:

list[Any]

static for_model(model)

Best-effort schema a model expects, from its fields/sources + child distributions.

Formalizes the duck-probe in the DataFrame adapter: a record/composite model exposes fields and sources plus child distributions whose support fixes each field’s logical type; a bare leaf yields a single field typed by its support (discrete -> Count, continuous -> Real, …).

Parameters:

model (Any)

Return type:

Schema