mixle.stats.rankings.spearman_rho module¶

Create, estimate, and sample from a Spearman ranking distribution.

Defines the SpearmanRankingDistribution, SpearmanRankingSampler, SpearmanRankingAccumulatorFactory, SpearmanRankingAccumulator, SpearmanRankingEstimator, and the SpearmanRankingDataEncoder classes for use with mixle.

Data type: List[int] (Component-wise rank of K dimensional observation vector)

The Spearman ranking distribution with dimension K, has probability function

p_mat(x_k;rho, sigma) = exp(-rho * ||x_k-sigma||^2 ) / sum_{k=0}^{K-1} exp(-rho * ||x_k-sigma||^2 ), for k = 0,1,..,K-1

where x_k list of integers containing a permutation of the integers 0,1,2,…K-1. Note sigma is a list of floats with dimension equal to K representing the mean of the rank variables, and rho is a correlation coefficient.

class SpearmanRankingDistribution(sigma, rho=1.0, name=None, keys=None)[source]

Bases: SequenceEncodableProbabilityDistribution

Spearman ranking distribution over permutations of 0,…,K-1 with location sigma and decay rate rho.

Data type: List[int] (a permutation of the integers 0,1,…,K-1).

Parameters:

sigma (Sequence[float] | ndarray)
rho (float)
name (str | None)
keys (str | None)

classmethod compute_capabilities()[source]

classmethod compute_declaration()[source]

static backend_legacy_sufficient_statistics(x, params, engine)[source]

Return row-wise legacy sufficient statistics for resident reductions.

Parameters:

x (Any)
params (dict[str, Any])
engine (Any)

Return type:

tuple[Any, …]

static backend_log_density_from_params(x, sigma, rho, log_const, engine)[source]

Engine-neutral Spearman ranking log-density from fitted parameters.

Parameters:

x (Any)
sigma (Any)
rho (Any)
log_const (Any)
engine (Any)

Return type:

Any

density(x)[source]

Density of Spearman ranking distribution at observation x.

See log_density() for details.

Parameters:: x (List[int]) – Permutation of the integers 0,1,…,K-1.
Returns:: Density at observation x.
Return type:: float

log_density(x)[source]

Log-density of Spearman ranking distribution at observation x.

The log-density is given by

log(p(x; rho, sigma)) = -rho * ||x - sigma||^2 - log_const,

where log_const normalizes over all K! permutations.

Parameters:: x (List[int]) – Permutation of the integers 0,1,…,K-1.
Returns:: Log-density at observation x.
Return type:: float

seq_log_density(x)[source]

Vectorized evaluation of log-density at sequence encoded input x.

Parameters:: x (np.ndarray) – 2-d numpy array of N permutations with K columns.
Returns:: Numpy array of log-density (float) of length N.
Return type:: ndarray

backend_seq_log_density(x, engine)[source]

Engine-neutral vectorized log-density for encoded permutations.

Parameters:

x (ndarray)
engine (Any)

Return type:

Any

classmethod backend_stacked_params(dists, engine)[source]

Return stacked parameters for equal-dimensional Spearman ranking mixtures.

Parameters:

dists (Sequence[SpearmanRankingDistribution])
engine (Any)

Return type:

dict[str, Any]

classmethod backend_stacked_log_density(x, params, engine)[source]

Return an (n, k) matrix of Spearman ranking component log densities.

Parameters:

x (ndarray)
params (dict[str, Any])
engine (Any)

Return type:

Any

classmethod backend_stacked_sufficient_statistics(x, weights, params, engine)[source]

Return component-stacked legacy (count, weighted_rank_sum) statistics.

Parameters:

x (ndarray)
weights (Any)
params (dict[str, Any])
engine (Any)

Return type:

tuple[Any, Any]

sampler(seed=None)[source]

Create a SpearmanRankingSampler object from parameters of SpearmanRankingDistribution instance.

Parameters:: seed (Optional[int]) – Used to set seed in random sampler.
Returns:: SpearmanRankingSampler object.
Return type:: SpearmanRankingSampler

estimator(pseudo_count=None)[source]

Create a SpearmanRankingEstimator with matching dimension and concentration rho.

Parameters:: pseudo_count (Optional[float]) – Used to inflate sufficient statistics.
Returns:: SpearmanRankingEstimator object.
Return type:: SpearmanRankingEstimator

dist_to_encoder()[source]

Returns a SpearmanRankingDataEncoder object for encoding sequences of data.

Return type:: SpearmanRankingDataEncoder

enumerator()[source]

Returns SpearmanRankingEnumerator iterating permutations in descending probability order.

Return type:: SpearmanRankingEnumerator

class SpearmanRankingEnumerator(dist)[source]

Bases: DistributionEnumerator

Enumerate permutations in descending Spearman probability order, lazily.

The Spearman distance sum_i (x_i - sigma_i)^2 is a linear assignment cost (assigning value j to position i costs (j - sigma_i)^2), so descending probability is increasing assignment cost: Murty’s k-best assignment streams the permutations in order without materializing the K! support.

Parameters:: dist (SpearmanRankingDistribution)

class SpearmanRankingSampler(dist, seed=None)[source]

Bases: DistributionSampler

Sampler for the SpearmanRankingDistribution. Draws permutations of 0,…,K-1.

Parameters:

dist (SpearmanRankingDistribution)
seed (int | None)

sample(size=None)[source]

Draw iid samples (permutations of 0,…,K-1) from the Spearman ranking distribution.

Parameters:: size (Optional[int]) – Number of samples to draw. If None, a single permutation is returned.
Returns:: A single permutation (List[int]) if size is None, else a list of size permutations.
Return type:: list[int] | Sequence[list[int]]

class SpearmanRankingAccumulator(dim, name=None, keys=None)[source]

Bases: SequenceEncodableStatisticAccumulator

Accumulator for the SpearmanRankingDistribution. Tracks the weighted sum of ranks and total weight.

Parameters:

dim (int)
name (str | None)
keys (str | None)

update(x, weight, estimate)[source]

Update sufficient statistics with a weighted observation.

Parameters:

x (Union[List[int], np.ndarray]) – Permutation of the integers 0,1,…,K-1.
weight (float) – Weight for observation.
estimate (Optional[SpearmanRankingDistribution]) – Previous estimate (unused).

Return type:

None

initialize(x, weight, rng)[source]

Initialize sufficient statistics with a weighted observation.

Parameters:

x (Union[List[int], np.ndarray]) – Permutation of the integers 0,1,…,K-1.
weight (float) – Weight for observation.
rng (RandomState) – Random number generator (unused).

Return type:

None

seq_update(x, weights, estimate)[source]

Vectorized update of sufficient statistics from sequence encoded data.

Parameters:

x (np.ndarray) – 2-d numpy array of N permutations with K columns.
weights (np.ndarray) – Weights for each of the N observations.
estimate (Optional[SpearmanRankingDistribution]) – Previous estimate (unused).

Return type:

None

seq_initialize(x, weights, rng)[source]

Vectorized initialization of sufficient statistics from sequence encoded data.

Parameters:

x (np.ndarray) – 2-d numpy array of N permutations with K columns.
weights (np.ndarray) – Weights for each of the N observations.
rng (RandomState) – Random number generator (unused).

Return type:

None

combine(suff_stat)[source]

Combine sufficient statistics from another accumulator into this one.

Parameters:: suff_stat (Tuple[float, np.ndarray]) – Tuple of count and component-wise rank sums.
Returns:: Self, with aggregated sufficient statistics.
Return type:: SpearmanRankingAccumulator

value()[source]

Returns sufficient statistics as a Tuple of count and component-wise rank sums.

Return type:: tuple[float, ndarray]

from_value(x)[source]

Set sufficient statistics of accumulator from value x.

Parameters:: x (Tuple[float, np.ndarray]) – Tuple of count and component-wise rank sums.
Returns:: Self, with sufficient statistics set to x.
Return type:: SpearmanRankingAccumulator

key_merge(stats_dict)[source]

Merge sufficient statistics of object instance with suff stats containing matching keys.

Parameters:: stats_dict (Dict[str, Any]) – Dict mapping keys to shared sufficient statistics.
Returns:: None.
Return type:: None

key_replace(stats_dict)[source]

Set sufficient statistics of object instance to suff stats with matching keys.

Parameters:: stats_dict (Dict[str, Any]) – Dict mapping keys to shared sufficient statistics.
Returns:: None.
Return type:: None

acc_to_encoder()[source]

Returns a SpearmanRankingDataEncoder object for encoding sequences of data.

Return type:: SpearmanRankingDataEncoder

class SpearmanRankingAccumulatorFactory(dim, name=None, keys=None)[source]

Bases: StatisticAccumulatorFactory

Factory for creating SpearmanRankingAccumulator objects.

Parameters:

dim (int)
name (str | None)
keys (str | None)

make()[source]

Returns a new SpearmanRankingAccumulator object.

Return type:: SpearmanRankingAccumulator

class SpearmanRankingEstimator(dim, rho=None, pseudo_count=None, suff_stat=None, name=None, keys=None, max_rho=1.0e6)[source]

Bases: ParameterEstimator

Estimator for the SpearmanRankingDistribution from aggregated sufficient statistics.

The consensus ranking sigma and, by default, the concentration rho are estimated by maximum likelihood. Pass a numeric rho to hold the concentration fixed.

Parameters:

dim (int)
rho (float | None)
pseudo_count (float | None)
suff_stat (tuple[float, ndarray] | None)
name (str | None)
keys (str | None)
max_rho (float)

accumulator_factory()[source]

Returns a SpearmanRankingAccumulatorFactory for creating SpearmanRankingAccumulator objects.

Return type:: SpearmanRankingAccumulatorFactory

estimate(nobs, suff_stat)[source]

Estimate a SpearmanRankingDistribution from sufficient statistics.

The consensus ranking sigma is the maximum likelihood estimate, given by the rank order (argsort) of the component-wise rank sums. When rho is None, the concentration is the nonnegative MLE satisfying E_rho[||X-sigma||^2] = observed mean squared distance. If no data was observed, rho is set to 0.0.

Parameters:

nobs (Optional[float]) – Number of observations (unused).
suff_stat (Tuple[float, np.ndarray]) – Tuple of count and component-wise rank sums.

Returns:

SpearmanRankingDistribution object.

Return type:

SpearmanRankingDistribution

class SpearmanRankingDataEncoder[source]

Bases: DataSequenceEncoder

Data encoder for sequences of rank vector (permutation) observations.

seq_encode(x)[source]

Encode a sequence of N rank vectors for vectorized functions.

Parameters:: x (Sequence[List[int]]) – Sequence of N permutations of 0,1,…,K-1.
Returns:: 2-d numpy array with N rows and K columns.
Return type:: ndarray