mixle.stats.combinator.hurdle module

Hurdle count models: a two-part model that decouples whether a count is zero from how big it is.

A hurdle model splits the count into a binary “hurdle” and a zero-truncated count part:

P(X = 0) = pi, P(X = k) = (1 - pi) * P_base(k) / (1 - P_base(0)) for k > 0.

With probability pi the observation fails to cross the hurdle and is zero; otherwise it is a positive count drawn from base conditioned on being > 0 (the base zero is truncated away and the remaining mass renormalized). Contrast with ZeroInflatedDistribution, where a zero can come from either the inflation or the base, so an observed zero is latent-ambiguous and needs EM. In a hurdle model every zero is structural and every positive is a (truncated) base draw – there is no latent variable, so the two parts are estimated independently and in closed form: pi is just the zero rate, and the base is fit to the positive observations. Wrapping any count base gives the whole family: a Poisson base is the hurdle Poisson, a NegativeBinomial base the hurdle NB, and so on.

class HurdleDistribution(base, pi, name=None, keys=None)[source]

Bases: SequenceEncodableProbabilityDistribution

A zero hurdle (probability pi) followed by a zero-truncated base for the positive counts.

Parameters:

  • base (SequenceEncodableProbabilityDistribution)

  • pi (float)

  • name (str | None)

  • keys (str | None)

density(x)[source]

Return the hurdle probability at x.

Parameters:

x (Any)

Return type:

float

log_density(x)[source]

Return log pi at x == 0, else log[(1-pi) p_base(x) / (1 - p_base(0))].

Parameters:

x (Any)

Return type:

float

seq_log_density(x)[source]

Vectorized hurdle log-density for an encoded batch.

Parameters:

x (tuple[Any, ndarray])

Return type:

ndarray

sampler(seed=None)[source]

Return a HurdleSampler for this distribution.

Parameters:

seed (int | None)

Return type:

HurdleSampler

estimator(pseudo_count=None)[source]

Return an estimator that fits pi (the zero rate) and the base on the positives – closed form.

Parameters:

pseudo_count (float | None)

Return type:

HurdleEstimator

dist_to_encoder()[source]

Return the data encoder (base encoding + a boolean is-zero mask).

Return type:

HurdleDataEncoder

class HurdleSampler(dist, seed=None)[source]

Bases: DistributionSampler

Draw a zero with probability pi, otherwise draw from the zero-truncated base.

Parameters:

  • dist (HurdleDistribution)

  • seed (int | None)

sample(size=None)[source]

Draw one observation or a list of size observations.

Parameters:

size (int | None)

class HurdleAccumulator(base_accumulator, keys=None)[source]

Bases: SequenceEncodableStatisticAccumulator

Accumulate the zero count (for pi) and the base sufficient statistics over the positives only.

Parameters:

  • base_accumulator (SequenceEncodableStatisticAccumulator)

  • keys (str | None)

update(x, weight, estimate)[source]

Accumulate one observation, sending only nonzeros to the base.

Parameters:

  • x (Any)

  • weight (float)

  • estimate (HurdleDistribution | None)

Return type:

None

seq_update(x, weights, estimate)[source]

Accumulate encoded observations, masking zeros out of the base update.

Parameters:
Return type:

None

initialize(x, weight, rng)[source]

Initialize the accumulator with one weighted observation.

Parameters:
Return type:

None

seq_initialize(x, weights, rng)[source]

Initialize encoded observations, excluding zeros from the base.

Parameters:
Return type:

None

combine(suff_stat)[source]

Merge serialized base statistics and zero counts into this accumulator.

Parameters:

suff_stat (tuple[Any, float, float])

Return type:

HurdleAccumulator

value()[source]

Return base statistics, weighted zero count, and total weight.

Return type:

tuple[Any, float, float]

from_value(x)[source]

Restore the accumulator from serialized hurdle statistics.

Parameters:

x (tuple[Any, float, float])

Return type:

HurdleAccumulator

scale(c)[source]

Scale base statistics and hurdle counts by a constant.

Parameters:

c (float)

Return type:

HurdleAccumulator

key_merge(stats_dict)[source]

Merge base and hurdle statistics into a keyed statistics dictionary.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

key_replace(stats_dict)[source]

Replace base and hurdle statistics from a keyed statistics dictionary.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

acc_to_encoder()[source]

Return an encoder that augments the base encoding with a zero mask.

Return type:

HurdleDataEncoder

class HurdleAccumulatorFactory(base_factory, keys=None)[source]

Bases: StatisticAccumulatorFactory

Factory for HurdleAccumulator.

Parameters:

  • base_factory (StatisticAccumulatorFactory)

  • keys (str | None)

make()[source]

Create an empty hurdle accumulator.

Return type:

HurdleAccumulator

class HurdleEstimator(base_estimator, pseudo_count=None, name=None, keys=None, trunc_max_iter=100, trunc_threshold=1.0e-10)[source]

Bases: ParameterEstimator

Closed-form pi (the zero rate) plus the zero-truncated MLE of the base over the positives.

The two parts are independent. pi is the observed zero rate. The count part is the base fit by maximum likelihood under the zero truncation – NOT the base fit naively to the positives, which is biased (it recovers the truncated mean, so the fitted model would not match the data). The truncated MLE is obtained by a short EM that treats the removed zeros as missing data: given the current base, the positives imply N_missing = N_pos * P0/(1-P0) hypothetical base zeros; refit the base to the positives plus those pseudo-zeros and iterate. (If the base has no mass at 0 there is no truncation and the positives are fit directly.)

Parameters:

  • base_estimator (ParameterEstimator)

  • pseudo_count (float | None)

  • name (str | None)

  • keys (str | None)

  • trunc_max_iter (int)

  • trunc_threshold (float)

accumulator_factory()[source]

Return a factory for hurdle sufficient-statistic accumulators.

Return type:

HurdleAccumulatorFactory

estimate(nobs, suff_stat)[source]

Estimate the hurdle probability and zero-truncated base distribution.

Parameters:
Return type:

HurdleDistribution

class HurdleDataEncoder(base_encoder)[source]

Bases: MaskedBaseEncoder

Encode observations via the base encoder, plus a boolean x == 0 mask.

Parameters:

base_encoder (DataSequenceEncoder)