mixle.inference.production.drift module¶
Model / data drift detection for production: is current data still the data the model was trained on?
Two complementary views:
Feature drift – per-field distribution shift between a reference (training) sample and a current (production) sample: Population Stability Index (PSI), Kolmogorov-Smirnov, and Jensen-Shannon. These are model-agnostic and operate on the schema’s fields.
Score drift – the model-native signal: the distribution of the model’s own log-density on current data versus on reference data. A fitted mixle model is the reference distribution, so if current data scores systematically lower (or its log-likelihood distribution shifts) the world has moved away from the model – exactly when to retrain.
detect_drift() combines both into a DriftReport with a single drift flag against
thresholds, suitable for a monitoring loop (see mixle.inference.production.monitor.Monitor).
- population_stability_index(reference, current, *, bins=10)[source]
PSI between two 1-D numeric samples (bin edges from the reference quantiles).
Rule of thumb: < 0.1 no shift, 0.1-0.25 moderate, > 0.25 significant.
- ks_statistic(reference, current)[source]
Two-sample Kolmogorov-Smirnov statistic in [0, 1] (larger = more shift).
- js_divergence(reference, current, *, bins=20)[source]
Jensen-Shannon divergence (bits) between two 1-D numeric samples (shared histogram support).
- score_drift(model, reference, current)[source]
The model-native drift signal: how the model’s log-density distribution shifts from reference to current data. Returns the KS statistic between the two log-likelihood samples and their mean shift (mean current log-density minus mean reference; negative => current data is less likely under the model).
Both populations are materialized here exactly once, so a one-shot iterable is safe and the reported counts describe the records actually scored. Alongside the shift statistics the report carries the evidence behind them – how many records each population held and what fraction of each was unscorable – because a shift measured over a handful of surviving finite scores is not the same claim as one measured over the whole sample.
- class DriftReport(drift, score, per_feature=<factory>, thresholds=<factory>, processed_count=0, reasons=<factory>)[source]
Bases:
objectDrift decision, aggregate score, feature details, and thresholds.
reasonsnames every condition that raised thedriftflag, so a verdict driven by lost scoring coverage is distinguishable from one driven by a genuine distribution shift.
- detect_drift(model, reference, current, *, psi_threshold=0.25, ks_threshold=0.2, loglik_shift_threshold=-0.5, min_scorable_fraction=0.5, unscorable_shift_threshold=0.1, per_feature=True)[source]
Combine score drift and per-feature drift into a single
DriftReport.driftis flagged if the score-distribution KS exceedsks_threshold, OR the mean log-likelihood drops by more than-loglik_shift_threshold(i.e.mean_loglik_shift < loglik_shift_threshold), OR any feature’s PSI exceedspsi_threshold, OR the current sample lost too much scoring coverage to support a no-drift verdict.Coverage is part of the verdict, not just of the report.
score_driftdrops unscorable records before comparing distributions, so a current sample of one matching score plus nine records the model cannot score at all used to produce KS 0, mean shift 0, anddrift=False– a confident “no drift” from 10% of the evidence. Two coverage conditions now fail closed:fewer than
min_scorable_fractionof current records scored finitely, andthe current unscorable rate exceeds the reference’s by more than
unscorable_shift_threshold(records moving outside the model’s support IS the world moving away from the model).
Both populations are materialized once and reused for scoring and for the per-field columns, so a one-shot iterable is safe: consuming the stream twice previously left the feature pass with empty columns and manufactured a PSI-infinity DRIFT verdict out of two identical inputs.