mixle.stats.combinator.backoff module

Backoff: reserve a small share of mass for outcomes a fitted support cannot represent.

A distribution fitted on observed data can only score what it observed. A bounded family is the sharp case – IntegerCategoricalDistribution fitted on a training split gives -inf to any integer outside (or inside a hole of) that split’s support, and one such row drives a whole held-out mean log-density to -inf.

BackoffDistribution is the composable answer: a two-component mixture of a base (the sharp fitted model) and a fallback whose support covers the outcomes base cannot reach, with the mixing weight pinned small rather than freely estimated:

log p(x) = logaddexp( log(1 - w) + log p_base(x),  log w + log p_fallback(x) )

The pin is the whole point, and it is what separates this from MixtureEstimator over the same two components. Free EM on a categorical-plus-count pair is model selection, not smoothing: on 400 Poisson draws it settles near [0.46, 0.54] and the fallback swallows the detail the base was there to capture. max_escape_weight bounds that, so the base keeps explaining the data it can explain and the fallback only covers the tail.

Two things the caller owns:

• The fallback sets the tail. It must have support wherever base does not. A PoissonDistribution fallback scores a far-out integer at roughly -74 because its tail decays factorially; NegativeBinomialDistribution or a broad bounded-uniform is usually the better choice. Whatever you pass is what the tail behaves like.

• This needs a fallback family to exist. That holds for integers. It does not hold for arbitrary string labels – there is no natural distribution over unseen strings – so CategoricalDistribution.default_value remains the right tool there, and this combinator is not a replacement for it. IntegerCategoricalDistribution has no such knob – it was removed in MXR-080-1839 because a constant out-of-support mass over an unbounded integer support diverges – which is exactly why integers get this combinator instead.

• A mixture of factors is a factor, not a law. Several mixle families deliberately admit unnormalized weight vectors – CategoricalDistribution and its integer counterpart both document that – and this class cannot detect it: there is no general “are you normalized?” query to ask a child. So BackoffDistribution inherits its children’s status rather than conferring one, and wrapping a factor does not make it exactly generative however proper the mixing weights are (MXR-080-1844). sampler() in particular is only meaningful when both children are true laws. Pass normalized components if you need the result to be one.

Usage (MXR-080-1854):

from mixle.stats import BackoffDistribution, BackoffEstimator
from mixle.stats import IntegerCategoricalEstimator, PoissonDistribution, PoissonEstimator

# score-time: a sharp support plus a fallback that covers what it never saw
scorer = BackoffDistribution(sharp_fit, PoissonDistribution(16.0), escape_weight=0.01)
scorer.log_density(84)          # finite even though sharp_fit alone gives -inf

# fit-time: escape_weight is a FLOOR, max_escape_weight a ceiling
est = BackoffEstimator(
    IntegerCategoricalEstimator(min_val=0, max_val=99),
    PoissonEstimator(),
    escape_weight=0.01,         # never estimated below this; zero is absorbing
    max_escape_weight=0.05,     # above this it becomes model selection, not smoothing
)

Set max_escape_weight equal to escape_weight to freeze the weight, or to None to let EM choose freely – which is model selection between the two components and usually not what you want.

class BackoffDistribution(base, fallback, escape_weight=DEFAULT_ESCAPE_WEIGHT, name=None, keys=None)

Bases: SequenceEncodableProbabilityDistribution

A sharp base mixed with a broad fallback under a pinned escape weight.

Parameters:

• base (SequenceEncodableProbabilityDistribution)

• fallback (SequenceEncodableProbabilityDistribution)

• escape_weight (float)

• name (str | None)

• keys (str | None)

compute_declaration()

Declare the backoff’s support as the support its two children agree on.

A backoff mixes a sharp base with a broad fallback over the SAME sample space, so when both children declare the same support the mixture has it too. Declaring nothing made the composite unusable wherever a caller has to prove what a distribution ranges over: SequenceDistribution refuses a length model that cannot “prove support on non-negative integers”, and mixle.utils.automatic builds exactly this shape – a backoff length model over sequence records – so the automatic path produced a model the library then rejected.

Children that disagree (or do not declare) leave the support unstated, which is the honest answer rather than a guess.

density_semantics()

Join both children’s semantics – a backoff is only exact when everything it mixes is.

The base-class default returns EXACT, which is a claim about the whole mixture. A backoff over a child explicitly marked LIKELIHOOD_FACTOR (a categorical used as an unnormalized factor, say) inherits that child’s missing normalizer: the escape-weighted sum of an exact law and an unnormalized factor is itself unnormalized. Reporting EXACT there tells a caller a density integrates to one when it does not.

density(x)

Return the mixture density at x.

Parameters:

x (Any)

Return type:

float

log_density(x)

Return the mixture log-density at x.

Finite whenever either component can score x, which is the reason this class exists: the base may return -inf for an unobserved outcome and the total stays finite.

Parameters:

x (Any)

Return type:

float

component_log_densities(x)

Return the two weighted component log-density vectors for encoded data x.

Parameters:

x (tuple[Any, Any])

Return type:

tuple[ndarray, ndarray]

seq_log_density(x)

Vectorized mixture log-density over sequence-encoded observations.

Parameters:

x (tuple[Any, Any])

Return type:

ndarray

sampler(seed=None)

Return a sampler that draws the component first, then the value.

Parameters:

seed (int | None)

Return type:

BackoffSampler

estimator(pseudo_count=None)

Return an estimator for this backoff structure, reusing the children’s own estimators.

Parameters:

pseudo_count (float | None)

Return type:

BackoffEstimator

dist_to_encoder()

Return an encoder that encodes each observation for both components.

Return type:

BackoffDataEncoder

class BackoffSampler(dist, seed=None)

Bases: DistributionSampler

Sampler for BackoffDistribution.

Parameters:

• dist (BackoffDistribution)

• seed (int | None)

sample(size=None, *, batched=True)

Draw size observations (or one when size is None).

size must be an exact non-negative integer. Bare int(size) silently accepted every near-miss as a request: 2.7 became two draws, True became one, and -3 became an empty result that reads like a successful zero-row sample. Each of those is a caller mistake producing a differently-sized dataset than the one asked for.

Parameters:

• size (int | None)

• batched (bool)

Return type:

Any

class BackoffAccumulator(base_accumulator, fallback_accumulator, keys=None)

Bases: SequenceEncodableStatisticAccumulator

Accumulates child statistics weighted by EM responsibilities, plus the escape counts.

Parameters:

• base_accumulator (SequenceEncodableStatisticAccumulator)

• fallback_accumulator (SequenceEncodableStatisticAccumulator)

• keys (str | None)

update(x, weight, estimate)

Accumulate one raw observation.

Parameters:

• x (Any)

• weight (float)

• estimate (BackoffDistribution | None)

Return type:

None

seq_update(x, weights, estimate)

Accumulate a sequence-encoded batch.

Parameters:

• x (tuple[Any, Any])

• weights (ndarray)

• estimate (BackoffDistribution)

Return type:

None

initialize(x, weight, rng)

Cold start: give both components the full observation, and no escape evidence yet.

The escape weight starts at the estimator’s pin rather than being inferred from an initialization that has no responsibilities to work with.

Parameters:

• x (Any)

• weight (float)

• rng (RandomState | None)

Return type:

None

seq_initialize(x, weights, rng)

Vectorized initialize().

Parameters:

• x (tuple[Any, Any])

• weights (ndarray)

• rng (RandomState | None)

Return type:

None

combine(suff_stat)

Merge another accumulator’s value into this one.

Parameters:

suff_stat (tuple[Any, Any, float, float])

Return type:

BackoffAccumulator

value()

Return the merged sufficient statistic.

Return type:

tuple[Any, Any, float, float]

from_value(x)

Replace this accumulator’s state with x.

Parameters:

x (tuple[Any, Any, float, float])

Return type:

BackoffAccumulator

scale(c)

Scale every accumulated statistic by c.

Parameters:

c (float)

Return type:

BackoffAccumulator

key_merge(stats_dict)

Merge keyed statistics, delegating to both children.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

key_replace(stats_dict)

Adopt keyed statistics, delegating to both children.

Parameters:

stats_dict (dict[str, Any])

Return type:

None

acc_to_encoder()

Return the encoder matching this accumulator’s children.

Return type:

BackoffDataEncoder

class BackoffAccumulatorFactory(base_factory, fallback_factory, keys=None)

Bases: StatisticAccumulatorFactory

Factory for BackoffAccumulator.

Parameters:

• base_factory (StatisticAccumulatorFactory)

• fallback_factory (StatisticAccumulatorFactory)

• keys (str | None)

make()

Return a fresh backoff accumulator.

Return type:

BackoffAccumulator

class BackoffEstimator(base_estimator, fallback_estimator, escape_weight=DEFAULT_ESCAPE_WEIGHT, max_escape_weight=DEFAULT_MAX_ESCAPE_WEIGHT, name=None, keys=None)

Bases: ParameterEstimator

Estimator for BackoffDistribution with a bounded escape weight.

Parameters:

• base_estimator (ParameterEstimator)

• fallback_estimator (ParameterEstimator)

• escape_weight (float)

• max_escape_weight (float | None)

• name (str | None)

• keys (str | None)

accumulator_factory()

Return the matching accumulator factory.

Return type:

BackoffAccumulatorFactory

estimate(nobs, suff_stat)

Fit both components and the (bounded) escape weight.

Parameters:

• nobs (float | None)

• suff_stat (tuple[Any, Any, float, float])

Return type:

BackoffDistribution

class BackoffDataEncoder(base_encoder, fallback_encoder)

Bases: DataSequenceEncoder

Encodes each observation for both components.

Parameters:

• base_encoder (DataSequenceEncoder)

• fallback_encoder (DataSequenceEncoder)

seq_encode(x)

Encode x once per component.

Both components see every observation – unlike the zero-inflated/hurdle encoders, there is no mask here, because a backoff does not partition the data. Which component explains a row is a posterior quantity settled at fit time, not a property of the row.

The row count travels with the encoding so scoring can check its answer against the input it was asked about. Requiring only that the two children AGREE was not enough: both can drop the same row and return a plausible shorter vector, which then silently re-aligns every score with the wrong observation (MXR-080-1843).

Parameters:

x (Sequence[Any])

Return type:

tuple[Any, Any]