mixle.models.random_forest module

Random forests as a conditional leaf in the mixle estimation framework.

A random forest is discriminative and is not fit by accumulating additive sufficient statistics or by EM, so it does not look like the exponential-family leaves. It still fits the estimation contract cleanly if we treat it as a conditional distribution p(y | x): the observation is a pair (x, y), the accumulator’s “sufficient statistic” is the buffered weighted design matrix, combine() concatenates the per-partition buffers (the map-reduce step is the data shuffle), and estimate() trains the forest in a single non-EM pass over that buffer.

The result is a SequenceEncodableProbabilityDistribution whose seq_log_density returns log p(y | x), so a fitted forest composes with seq_encode / seq_log_density / the top-level log_density helper, and can sit in a slot of a composite/record model or act as a mixture-of-experts component. Because estimate() refits from scratch, run it through optimize(…, max_its=1) (there is no likelihood for EM to iterate); for classification log-density is the forest’s predict_log_proba, for regression it is a Gaussian residual model with a globally estimated noise scale.

The forest itself is a native numpy CART + bagging ensemble (mixle.models._forest), so mixle carries no scikit-learn dependency.

class RandomForestConditionalSampler(dist, seed=None, *, rng=None)

Bases: DistributionSampler

Sampler for the conditional forest. p(y | x) cannot generate x, so the unconditional sample() is disabled; use sample_y(X) to draw targets given features.

Parameters:

• dist (SequenceEncodableProbabilityDistribution)

• seed (int | None)

• rng (RandomState | None)

sample(size=None, *, batched=True)

Raise because the conditional forest has no marginal model for x.

Parameters:

• size (int | None)

• batched (bool)

Return type:

Any

sample_y(x)

Draw a target for each row of x: a class from predict_proba (classification) or mean+Gaussian-noise (regression).

Parameters:

x (Any)

Return type:

ndarray

class RandomForestConditional(forest, task, sigma=None, n_features=None, name=None, keys=None)

Bases: SequenceEncodableProbabilityDistribution

Fitted random forest viewed as a conditional distribution p(y | x).

Observations are (x, y) pairs: x is a feature vector and y is a class label (classification) or a real target (regression). seq_log_density returns log p(y | x) – predict_log_proba for classification, a Gaussian residual density with scale sigma for regression.

Parameters:

• forest (Any)

• task (str)

• sigma (float | None)

• n_features (int | None)

• name (str | None)

• keys (str | None)

density(x)

Return p(y | x) for one feature/target pair.

Parameters:

x (tuple[Any, Any])

Return type:

float

log_density(x)

Return log p(y | x) for one feature/target pair.

Parameters:

x (tuple[Any, Any])

Return type:

float

seq_log_density(x)

Return per-row conditional log densities for encoded (X, y) data.

Parameters:

x (tuple[ndarray, ndarray])

Return type:

ndarray

sample_y(x, rng)

Draw target values from the fitted conditional forest at feature rows x.

Parameters:

• x (Any)

• rng (RandomState)

Return type:

ndarray

sampler(seed=None)

Return a conditional sampler for drawing targets given features.

Parameters:

seed (int | None)

Return type:

RandomForestConditionalSampler

estimator(pseudo_count=None)

Return a fresh estimator with the same task, name, and keyed-accumulation settings.

Parameters:

pseudo_count (float | None)

Return type:

RandomForestEstimator

dist_to_encoder()

Return the encoder for feature/target observation pairs.

Return type:

RandomForestEncoder

class RandomForestAccumulator(keys=None, name=None)

Bases: SequenceEncodableStatisticAccumulator

Buffers the weighted (x, y) design matrix; combine() concatenates partition buffers into the full training set that estimate() fits the forest on.

Parameters:

• keys (str | None)

• name (str | None)

update(x, weight, estimate)

Add one weighted feature/target observation to the training buffer.

Parameters:

• x (tuple[Any, Any])

• weight (float)

• estimate (RandomForestConditional | None)

Return type:

None

initialize(x, weight, rng)

Initialize from one observation using the ordinary update path.

Parameters:

• x (tuple[Any, Any])

• weight (float)

• rng (RandomState | None)

Return type:

None

seq_update(x, weights, estimate)

Add an encoded batch and weights to the training buffer.

Parameters:

• x (tuple[ndarray, ndarray])

• weights (ndarray)

• estimate (RandomForestConditional | None)

Return type:

None

seq_initialize(x, weights, rng)

Initialize from an encoded batch using the ordinary batch update path.

Parameters:

• x (tuple[ndarray, ndarray])

• weights (ndarray)

• rng (Any)

Return type:

None

combine(suff_stat)

Merge a buffered (X, y, weights) tuple from another accumulator.

Parameters:

suff_stat (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestAccumulator

value()

Return the buffered design matrix, targets, and weights, or None if empty.

Return type:

tuple[ndarray, ndarray, ndarray] | None

from_value(x)

Restore the accumulator from a buffered value tuple.

Parameters:

x (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestAccumulator

key_merge(stats_dict)

Merge this accumulator into stats_dict under keys when keyed accumulation is enabled.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

key_replace(stats_dict)

Replace this accumulator from stats_dict under keys when present.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

acc_to_encoder()

Return the encoder expected by this accumulator.

Return type:

RandomForestEncoder

class RandomForestAccumulatorFactory(name=None, keys=None)

Bases: StatisticAccumulatorFactory

Factory for random-forest accumulators.

Parameters:

• name (str | None)

• keys (str | None)

make()

Create a fresh random-forest accumulator.

Return type:

RandomForestAccumulator

class RandomForestEstimator(task='auto', n_estimators=100, max_depth=None, min_samples_split=2, min_samples_leaf=1, max_features='auto', random_state=None, min_sigma=1.0e-3, name=None, keys=None)

Bases: ParameterEstimator

Estimator that fits a native (numpy) random forest as a conditional leaf.

task is ‘classification’, ‘regression’, or ‘auto’ (inferred from the dtype of y). The forest hyperparameters (n_estimators, max_depth, min_samples_split, min_samples_leaf, max_features, random_state) are passed straight to the native ensemble. estimate() trains in one pass on the accumulated weighted data; there is no EM iteration, so drive it with optimize(max_its=1) or call the seq_encode / accumulate / estimate path directly.

Parameters:

• task (str)

• n_estimators (int)

• max_depth (int | None)

• min_samples_split (int)

• min_samples_leaf (int)

• max_features (Any)

• random_state (int | None)

• min_sigma (float)

• name (str | None)

• keys (str | None)

accumulator_factory()

Return an accumulator factory for weighted feature/target buffers.

Return type:

RandomForestAccumulatorFactory

estimate(nobs, suff_stat)

Fit the native forest from buffered data and return it as a conditional leaf.

Parameters:

• nobs (float | None)

• suff_stat (tuple[ndarray, ndarray, ndarray] | None)

Return type:

RandomForestConditional

class RandomForestEncoder

Bases: DataSequenceEncoder

Encodes a sequence of (x, y) observations into a (design-matrix, target-vector) pair.

seq_encode(x)

Convert feature/target pairs into a design matrix and target vector.

Parameters:

x (list[tuple[Any, Any]])

Return type:

tuple[ndarray, ndarray]