mixle.utils.automatic.factories module

Estimator builders for automatically-typed data.

Builds estimators for mixle.stats. By default the plain maximum-likelihood estimators are produced; pass use_bstats=True to build the Bayesian path, which attaches the conjugate default prior for each family so estimation performs the closed-form conjugate / MAP update.

get_optional_estimator(est, missing_value=None, use_bstats=False, est_prob=False)

Wrap an estimator with an optional/missing-value model.

est_prob decides what the wrapper MEANS, so callers pass it deliberately rather than taking a default. With it false the fitted OptionalDistribution carries no rate: it is transparent to the density (contributing 0) and is a marginalized likelihood factor that cannot generate observations. With it true the missingness rate is fit from the data the profiler has already seen, the wrapper contributes log(p)/log(1-p) to the density, and the model can be sampled.

Genuine missingness wants the rate, and at the automatic surface that is BOTH of its spellings: None and nan. They must mean the same model, because they are interchanged behind the caller’s back – a pandas float Series silently stores None as nan – so giving nan the transparent non-generative wrapper made the same data produce a different (and unsamplable) model depending on the container it arrived in.

The infinity sentinels (+/-inf) take the fitted rate for the SAME reason, though by a different argument. They are indeed values a numeric field can carry rather than absences, but that is precisely why the fitted-rate wrapper is the correct model for them: it is an atom of mass p at the sentinel beside the base family scaled by 1-p, which is exactly “this column is a continuous measurement that comes out infinite p of the time” and integrates to one. The transparent default asserts the opposite – .p reads 0.0, “missingness never happens”, while log_density(inf) returns 0.0, probability one – for total mass 2.0, an improper density whose sentinel rows cost exactly zero nats. Auto-inference used the transparent default here through 0.7.x, so a single overflow or a JSON 1e999 bought an unbounded free gain in log-likelihood against any proper competitor on the same data, invisible to the paired comparison tests in mixle.inference because those consume plain arrays and cannot see density_semantics(). Callers that genuinely want the marginalized factor – inference conditional on an externally modeled missingness mechanism – still get it by passing est_prob=False deliberately, which is what this argument is for.

Parameters:

• est (ParameterEstimator)

• missing_value (Any | None)

• use_bstats (bool)

• est_prob (bool)

parse_numeric_text(value)

Parse value as a decimal-point/exponent-bearing numeric literal, or return None.

Deliberately narrower than float() in two ways. First, it excludes the "nan"/"inf"/ "infinity" keyword spellings float() itself accepts, so a dirty marker spelled that way keeps routing to the string branch instead of being silently reinterpreted as the nan/inf sentinel here – that decision belongs to the missing/infinite-value wrapping in DatumNode.get_estimator, not this parser. Second, it requires a decimal point or exponent, so a bare digit string (“02139”, an integer-coded column read as text) is left as a string rather than folded into the integer-typing path: every continuous family already accepts a raw string query via float(value) coercion (see mixle.stats.univariate.continuous._observation_contracts.scored_observation), but the discrete/categorical families reached by the integer path are not proven to, and a bare digit string is exactly the shape (“02139”) where int() parsing would also silently drop a leading zero. Every value this DOES parse is read the way mixle already reads a Python float literal – a continuous measurement with support past what was observed, never a memorization table (see _numeric_side_is_memorizing) – so routing it into the float branch needs no further scrutiny.

Parameters:

value (str)

Return type:

float | None

get_typed_mixture_estimator(string_estimator, numeric_estimator, use_bstats=False, string_support=None)

Dispatch mixture for a scalar column that carries both numbers and strings.

One dirty cell in a CSV – a single "N/A", "NULL", "?" or "" in an otherwise continuous column – is the most ordinary data-quality defect there is, and until 0.8.0 it silently retyped the WHOLE column: 300 finite floats plus one "N/A" resolved to a categorical over the 301 observed values (wrapped in IgnoredDistribution once the identifier thresholds were crossed), a memorization table that scored every value it had not literally seen – the sample mean included – at -inf, with fit_provenance() reporting converged=True and repairs=().

The honest reading of such a column is that it carries two disjoint types, so the honest model is the one the library already has for disjoint types: a SelectDistribution dispatch mixture whose branch label is OBSERVED – nothing is guessed and nothing is iterated: the weights are the two branch proportions in closed form, each child is fit on its own subset by the ordinary leaf rules, and the result is a normalized law that samples. The branch is routed by _NumericTextDispatch rather than a bare TypeDispatch, so a value that ARRIVES as a string but READS as a number – the shape a pandas column takes once one dirty cell coerces the whole thing to object dtype – still routes with the numbers rather than the strings; see parse_numeric_text().

Two properties make this a repair rather than a re-design. The string branch’s density is unchanged – log(n_str/n) + log(count/n_str) is log(count/n), what the merged categorical gave, exactly in real arithmetic and to within a ULP in floating point – so nothing that scored finitely before scores differently now. And on the case above the mixture reproduces the model you get by spelling the dirty cell nan instead, digit for digit (mean 49.997748666666666, variance 93.32376314029874, rate 0.0033222591362126247), which is the answer the library already called correct.

string_support exists for one specific consequence of routing. A merged categorical is shown every label in every mixture component – CategoricalAccumulator records a label even at zero weight – but a ROUTED branch is only shown the rows that route to it, and a component that is initialized with no string rows at all hands its string branch an empty count map. Under the Bayesian path the symmetric DictDirichletDistribution(alpha) cannot widen that into anything (its parameters are a scalar, so there is no support to fall back on) and estimation raises “empty categorical fitting requires a prior with an explicit finite support” – which is how get_dpm_mixture met this repair. Naming the observed string labels in the prior gives the empty branch the uniform over exactly those labels instead. The plain MLE path needs no equivalent: get_categorical_estimator already carries the empirical map as suff_stat, and the empty case widens to a zero-count support over it.

Parameters:

• string_estimator (ParameterEstimator) – Leaf estimator for the str/bytes values.

• numeric_estimator (ParameterEstimator) – Leaf estimator for the (non-boolean) real values.

• use_bstats (bool) – Build the Bayesian path for the wrapper’s provider lookup.

• string_support (Sequence[Any] | None) – Observed string labels, used only to pin the Bayesian branch’s Dirichlet support as described above. Ignored on the plain path and for a frozen (Ignored) branch, which is never re-estimated and so never meets the empty-count case.

Returns:

A SelectEstimator that routes numeric-looking text with the numbers and estimates the branch weights.

Return type:

ParameterEstimator

get_length_estimator(len_dict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Length model for sequences.

Observed lengths are often bounded protocol/domain facts, not Poisson counts, so use an integer categorical model while the support is small. Fall back to a Poisson only when length support is broad enough to look count-like.

Parameters:

• len_dict (dict[int, int])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_sequence_estimator(est, len_dict=None, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a sequence estimator with an optional empirical length model.

Parameters:

• est (ParameterEstimator)

• len_dict (dict[int, int] | None)

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_set_estimator(member_dict, num_sets, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Bernoulli set model with membership probabilities from observed sets.

Parameters:

• member_dict (dict[Any, int])

• num_sets (int)

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_ignored_estimator(use_bstats=False)

Return the estimator used for ignored or non-modelable fields.

Parameters:

use_bstats (bool)

Return type:

ParameterEstimator

get_composite_estimator(ests, use_bstats=False)

Return a composite estimator over an ordered list of field estimators.

Parameters:

• ests (Sequence[ParameterEstimator])

• use_bstats (bool)

Return type:

ParameterEstimator

get_dict_record_estimator(keys, ests)

Return a record estimator keyed by dictionary field names.

Parameters:

• keys (Sequence[Any])

• ests (Sequence[ParameterEstimator])

Return type:

ParameterEstimator

get_categorical_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a categorical estimator from observed value counts.

Parameters:

• vdict (dict[T, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_integer_categorical_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return an integer-categorical estimator over the observed dense support.

Parameters:

• vdict (dict[int, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_poisson_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a Poisson count estimator from empirical integer counts.

Parameters:

• vdict (dict[int, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_gaussian_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a univariate Gaussian estimator from weighted numeric values.

Parameters:

• vdict (dict[floating | float, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_lognormal_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a LogGaussian (log-normal) estimator fit to the log of strictly-positive values.

Parameters:

• vdict (dict[floating | float, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_gamma_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a Gamma estimator seeded with empirical mean/log-mean statistics.

Parameters:

• vdict (dict[floating | float, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_student_t_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False)

Return a fixed-df Student-t estimator initialized from a likelihood fit.

Parameters:

• vdict (dict[floating | float, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

Return type:

ParameterEstimator

get_gaussian_mixture_estimator(vdict, pseudo_count=None, emp_suff_stat=True, use_bstats=False, n_components=2)

Return a K-component Gaussian mixture estimator (robust init) for multimodal numeric data.

Parameters:

• vdict (dict[floating | float, float])

• pseudo_count (float | None)

• emp_suff_stat (bool)

• use_bstats (bool)

• n_components (int)

Return type:

ParameterEstimator

get_multivariate_gaussian_estimator(dim, use_bstats=False)

Return a multivariate Gaussian estimator for vectors of dimension dim.

Parameters:

• dim (int)

• use_bstats (bool)

Return type:

ParameterEstimator

get_hybrid_embedding_estimator(dim)

An exact neural density (a coupling flow) over an embedding-shaped dim-vector field.

Parameters:

dim (int)

Return type:

ParameterEstimator

get_hybrid_image_estimator(dim=IMAGE_FEATURE_DIM)

A frozen image_features extractor composed with an exact neural density over the induced features.

Parameters:

dim (int)

Return type:

ParameterEstimator

get_dpm_mixture(data, rng=None, max_components=20, max_its=100, delta=1.0e-6, pseudo_count=1.0, print_iter=1, out=None)

Fit a Dirichlet process mixture to automatically-typed data.

Component estimators are constructed with get_estimator(use_bstats=True) (one independent conjugate-prior instance per stick), and the truncated stick-breaking posterior is fit with variational inference via mixle.inference.estimation.fit.

Parameters:

• data – Sequence of observations of any auto-detectable type.

• rng (Optional[RandomState]) – Source of randomness for initialization.

• max_components (int) – Truncation level of the stick-breaking representation.

• max_its (int) – Maximum number of variational iterations.

• delta (float) – Stop when the ELBO improves by less than delta.

• pseudo_count (Optional[float]) – Prior strength for the component priors.

• print_iter (int) – Progress print frequency.

• out – Output stream for iteration logging (defaults to sys.stdout).

Returns:

DirichletProcessMixtureDistribution fit to the data.

Raises:

ValueError – If data is empty, or if automatic typing found no modelable field at all (e.g. every column is identifier-like) – the mixture would then score every observation identically and “fit” nothing; the error names each field, its cardinality, and the remedy.