mixle.data.validate module

Check that a dataset conforms to the spec a model expects, before fitting.

check_dataset(model, data) derives the model’s schema (mixle.data.schema.Schema.for_model()) and, over a sample of records, verifies each record (a) coerces to the schema’s logical types and (b) lies in the model’s support (finite log-density). It returns a report (and can raise on failure) so a bad feed – wrong column types, out-of-support values, malformed records – is caught up front rather than producing a silent NaN or a cryptic error deep in EM.

class DataReport(ok: 'bool', n_checked: 'int', schema: 'list[tuple[str, str]]', issues: 'list[str]' = <factory>)[source]

Bases: object

Parameters:
ok: bool
n_checked: int
schema: list[tuple[str, str]]
issues: list[str]
check_dataset(model, data, *, sample=1000, check_support=True, raise_on_error=False)[source]

Validate data against the schema/support model expects (over the first sample records).

Records both type-coercion failures (wrong shape/dtype for a field) and, when check_support is True, support violations (a value the model assigns probability 0 -> -inf log-density). With raise_on_error the first batch of issues is raised as a ValueError.

Parameters:
Return type:

DataReport