mixle.models.random_forest module

Random forests as a conditional leaf in the mixle estimation framework.

A random forest is discriminative and is not fit by accumulating additive sufficient statistics or by EM, so it does not look like the exponential-family leaves. It still fits the estimation contract cleanly if we treat it as a conditional distribution p(y | x): the observation is a pair (x, y), the accumulator’s “sufficient statistic” is the buffered weighted design matrix, combine() concatenates the per-partition buffers (the map-reduce step is the data shuffle), and estimate() trains the forest in a single non-EM pass over that buffer.

The result is a SequenceEncodableProbabilityDistribution whose seq_log_density returns log p(y | x), so a fitted forest composes with seq_encode / seq_log_density / the top-level log_density helper, and can sit in a slot of a composite/record model or act as a mixture-of-experts component. Because estimate() refits from scratch, run it through optimize(…, max_its=1) (there is no likelihood for EM to iterate); for classification log-density is the forest’s predict_log_proba, for regression it is a Gaussian residual model with a globally estimated noise scale.

The forest itself is a native numpy CART + bagging ensemble (mixle.models._forest), so mixle carries no scikit-learn dependency.

class RandomForestConditionalSampler(dist, seed=None, *, rng=None)

Bases: DistributionSampler

Sampler for the conditional forest. p(y | x) cannot generate x, so the unconditional sample() is disabled; use sample_y(X) to draw targets given features.

Parameters:

• dist (SequenceEncodableProbabilityDistribution)

• seed (int | None)

• rng (RandomState | None)

sample(size=None, *, batched=True)

Draw observations.

Combinator samplers (mixture/sequence/…) accept batched. With batched=True (the default) each child stream is drawn in one vectorized call instead of a per-draw Python loop – far faster. Because every child sampler owns an independent RandomState, batching consumes each stream in the same order as the loop, so the draws are identical to the legacy path. batched=False forces that legacy per-draw loop as a guaranteed- stable reference. Leaf samplers are already vectorized and ignore the flag.

Parameters:

• size (int | None)

• batched (bool)

Return type:

Any

sample_y(x)

Draw a target for each row of x: a class from predict_proba (classification) or mean+Gaussian-noise (regression).

Parameters:

x (Any)

Return type:

ndarray

class RandomForestConditional(forest, task, sigma=None, n_features=None, name=None, keys=None)

Bases: SequenceEncodableProbabilityDistribution

Fitted random forest viewed as a conditional distribution p(y | x).

Observations are (x, y) pairs: x is a feature vector and y is a class label (classification) or a real target (regression). seq_log_density returns log p(y | x) – predict_log_proba for classification, a Gaussian residual density with scale sigma for regression.

Parameters:

• forest (Any)

• task (str)

• sigma (float | None)

• n_features (int | None)

• name (str | None)

• keys (str | None)

density(x)

Return the probability density or mass at a single observation.

Concrete default: exponentiate log_density (the abstract method subclasses must provide). Leaves with a cheaper closed form may override this.

Parameters:

x (tuple[Any, Any])

Return type:

float

log_density(x)

Return the log-density or log-mass at a single observation.

Parameters:

x (tuple[Any, Any])

Return type:

float

seq_log_density(x)

Return vectorized log-density values for sequence-encoded observations.

Parameters:

x (tuple[ndarray, ndarray])

Return type:

ndarray

sample_y(x, rng)

Parameters:

• x (Any)

• rng (RandomState)

Return type:

ndarray

sampler(seed=None)

Return a sampler for drawing observations from this distribution.

Parameters:

seed (int | None)

Return type:

RandomForestConditionalSampler

estimator(pseudo_count=None)

Return an estimator for fitting this distribution from data.

Parameters:

pseudo_count (float | None)

Return type:

RandomForestEstimator

dist_to_encoder()

Return the data encoder used by this distribution for vectorized methods.

Return type:

RandomForestEncoder

class RandomForestAccumulator(keys=None, name=None)

Bases: SequenceEncodableStatisticAccumulator

Buffers the weighted (x, y) design matrix; combine() concatenates partition buffers into the full training set that estimate() fits the forest on.

Parameters:

• keys (str | None)

• name (str | None)

update(x, weight, estimate)

Parameters:

• x (tuple[Any, Any])

• weight (float)

• estimate (RandomForestConditional | None)

Return type:

None

initialize(x, weight, rng)

Parameters:

• x (tuple[Any, Any])

• weight (float)

• rng (RandomState | None)

Return type:

None

seq_update(x, weights, estimate)

Parameters:

• x (tuple[ndarray, ndarray])

• weights (ndarray)

• estimate (RandomForestConditional | None)

Return type:

None

seq_initialize(x, weights, rng)

Parameters:

• x (tuple[ndarray, ndarray])

• weights (ndarray)

• rng (Any)

Return type:

None

combine(suff_stat)

Parameters:

suff_stat (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestAccumulator

value()

Return type:

tuple[ndarray, ndarray, ndarray] | None

from_value(x)

Parameters:

x (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestAccumulator

key_merge(stats_dict)

Pool this accumulator’s statistics into stats_dict under its merge key.

The structural default implements the common single-key pattern: store the accumulator under self.keys the first time the key is seen, else combine into the one already there. Accumulators with several named keys (e.g. an HMM’s init/trans/state keys) or a non-accumulator stats payload override this. A keys of None (the default) is a no-op.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

key_replace(stats_dict)

Replace this accumulator’s statistics from the pooled stats_dict entry (see key_merge).

Parameters:

stats_dict (dict[str, Any])

Return type:

None

acc_to_encoder()

Return type:

RandomForestEncoder

class RandomForestAccumulatorFactory(name=None, keys=None)

Bases: StatisticAccumulatorFactory

Parameters:

• name (str | None)

• keys (str | None)

make()

Return type:

RandomForestAccumulator

class RandomForestEstimator(task='auto', n_estimators=100, max_depth=None, min_samples_split=2, min_samples_leaf=1, max_features='auto', random_state=None, min_sigma=1.0e-3, name=None, keys=None)

Bases: ParameterEstimator

Estimator that fits a native (numpy) random forest as a conditional leaf.

task is ‘classification’, ‘regression’, or ‘auto’ (inferred from the dtype of y). The forest hyperparameters (n_estimators, max_depth, min_samples_split, min_samples_leaf, max_features, random_state) are passed straight to the native ensemble. estimate() trains in one pass on the accumulated weighted data; there is no EM iteration, so drive it with optimize(max_its=1) or call the seq_encode / accumulate / estimate path directly.

Parameters:

• task (str)

• n_estimators (int)

• max_depth (int | None)

• min_samples_split (int)

• min_samples_leaf (int)

• max_features (Any)

• random_state (int | None)

• min_sigma (float)

• name (str | None)

• keys (str | None)

accumulator_factory()

Return type:

RandomForestAccumulatorFactory

estimate(nobs, suff_stat)

Parameters:

• nobs (float | None)

• suff_stat (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestConditional

class RandomForestEncoder

Bases: DataSequenceEncoder

Encodes a sequence of (x, y) observations into a (design-matrix, target-vector) pair.

seq_encode(x)

Encode the iid observation sequence x for vectorized evaluation.

Parameters:

x (list[tuple[Any, Any]])

Return type:

tuple[ndarray, ndarray]