mixle.inference.production.drift module

Model / data drift detection for production: is current data still the data the model was trained on?

Two complementary views:

  • Feature drift – per-field distribution shift between a reference (training) sample and a current (production) sample: Population Stability Index (PSI), Kolmogorov-Smirnov, and Jensen-Shannon. These are model-agnostic and operate on the schema’s fields.

  • Score drift – the model-native signal: the distribution of the model’s own log-density on current data versus on reference data. A fitted mixle model is the reference distribution, so if current data scores systematically lower (or its log-likelihood distribution shifts) the world has moved away from the model – exactly when to retrain.

detect_drift() combines both into a DriftReport with a single drift flag against thresholds, suitable for a monitoring loop (see mixle.inference.production.monitor.Monitor).

population_stability_index(reference, current, *, bins=10)[source]

PSI between two 1-D numeric samples (bin edges from the reference quantiles).

Rule of thumb: < 0.1 no shift, 0.1-0.25 moderate, > 0.25 significant.

Parameters:
Return type:

float

ks_statistic(reference, current)[source]

Two-sample Kolmogorov-Smirnov statistic in [0, 1] (larger = more shift).

Parameters:

  • reference (Any)

  • current (Any)

Return type:

float

js_divergence(reference, current, *, bins=20)[source]

Jensen-Shannon divergence (bits) between two 1-D numeric samples (shared histogram support).

Parameters:
Return type:

float

score_drift(model, reference, current)[source]

The model-native drift signal: how the model’s log-density distribution shifts from reference to current data. Returns the KS statistic between the two log-likelihood samples and their mean shift (mean current log-density minus mean reference; negative => current data is less likely under the model).

Parameters:
Return type:

dict

class DriftReport(drift: 'bool', score: 'dict', per_feature: 'dict' = <factory>, thresholds: 'dict' = <factory>)[source]

Bases: object

Parameters:
drift: bool
score: dict
per_feature: dict
thresholds: dict
detect_drift(model, reference, current, *, psi_threshold=0.25, ks_threshold=0.2, loglik_shift_threshold=-0.5, per_feature=True)[source]

Combine score drift and per-feature drift into a single DriftReport.

drift is flagged if the score-distribution KS exceeds ks_threshold, OR the mean log-likelihood drops by more than -loglik_shift_threshold (i.e. mean_loglik_shift < loglik_shift_threshold), OR any feature’s PSI exceeds psi_threshold.

Parameters:
Return type:

DriftReport