mixle.utils.hvis package

Model-based (hierarchical) t-SNE and UMAP for heterogeneous data.

Pairwise affinities are derived from a fitted mixture model rather than from Euclidean distances, so anything mixle can model (tuples, sequences, sets, variable-length data, …) can be embedded. Six affinity definitions are supported (the affinity argument):

  • ‘local’ (the ‘auto’ default whenever raw data is available): the model is flattened into leaf fields and each field contributes a local statistical affinity combining the per-field posterior (between-cluster structure) with a component-local Mahalanobis metric (within-cluster structure). Continuous/count fields use their native coordinates; every other leaf – HMMs, Markov chains, categoricals, sequence-of-discrete element fields – uses typicality coordinates (per-component log-density; per-token rate plus a log-length axis for sequence-valued leaves), so no field type degrades to posterior-only geometry. Thus the same component is never a zero-distance quotient: sharp posteriors stop collapsing clusters into tiny structureless points, variable-length fields keep length as one honest axis instead of the dominant one, and mixed continuous/discrete fields are made commensurate by the per-component whitening. See affinity_health() for measurable receipts of these degeneracies.

  • ‘balanced’: the model is flattened into its leaf fields (nested composites, sequence element/length models, and optional wrappers all decompose), a field-restricted posterior z^f is computed from each field’s likelihoods alone, and the pair distance is the sum over fields of per-field Bhattacharyya distances -log sum_k sqrt(z^f_ik z^f_jk), each Winsorized at evidence_cap nats. The per-field posteriors keep every field’s structure visible regardless of its likelihood scale (by default, a 15-token sequence field contributes summed sequence evidence while length is a separate field; if the sequence model was explicitly fit with len_normalized=True, the sequence field instead contributes a per-token composition quotient), and the cap bounds each field’s influence so one spuriously sharp field cannot veto a pair’s similarity that every other field supports.

  • ‘fisher’: each observation is mapped through the model’s to_fisher() view to posterior-expected sufficient statistics and, by default, whitened by the empirical observed Fisher covariance of those score vectors. Pair affinities are Gaussian in that Fisher-vector space, so htsne can use the same sufficient-statistic geometry exposed to downstream tools.

  • ‘bhattacharyya’: the Bhattacharyya coefficient between joint posteriors, s_ij = sum_k sqrt(z_ik z_jk); -log s_ij is the Bhattacharyya distance on the posterior simplex. The square root amplifies shared low-probability components, so affinities stay graded even when hard assignments coincide - which is what gives the embedding within-cluster geometry. Like ‘coassign’, it depends on the data only through posteriors, so variable-length observations need no adjustments.

  • ‘coassign’: the co-assignment probability

    s_ij = P(z_i = z_j | x_i, x_j) = sum_k z_ik z_jk,

    the posterior similarity matrix of Bayesian clustering - an exact probability under the fitted model. The principled choice when the affinity itself must be a probability, but near-deterministic posteriors make it almost binary: every same-component pair ties at ~1, and t-SNE renders tied groups as rings/blobs with no internal structure.

  • ‘likelihood’: the predictive affinity s_ij = sum_k p(x_i | theta_k) z_jk (likelihood of x_i under the posterior mixture of x_j). Retains within- component likelihood detail, but for variable-length data the evidence in x_i grows with its length, so long observations reduce to their single best component while short ones stay blended.

For t-SNE the affinities are converted to input probabilities by row-conditional normalization p_{j|i} = softmax_j(log s_ij), optionally calibrated to a target perplexity per row, and symmetrized P = (P + P^T) / (2n).

Two t-SNE engines are provided:

  • ‘exact’: a full-matrix gradient descent supporting a heavy-tailed student-t kernel q_ij ~ (1 + d_ij^2 / alpha)^{-(alpha+1)/2} whose tail parameter alpha can be optimized along with the embedding. O(n^2) per iteration.

  • ‘barnes_hut’: scalable O(n log n) t-SNE run by an internal Barnes-Hut optimizer on a sparse model-neighbor probability matrix. The dense affinity matrix is never materialized; neighbor search can be exact blockwise or approximate via a random-projection candidate forest.

humap embeds the same model-based kNN graph with UMAP (umap-learn).

This package preserves the public API of the former single-module mixle.utils.hvis: every name below remains importable from mixle.utils.hvis. The implementation is split into:

  • affinity - factor/affinity computation and probability calibration

  • neighbors - sparse model-distance graphs, RP-trees, and kNN

  • tsne - the t-SNE embedding cores (exact and Barnes-Hut)

  • embed - the htsne/humap/dpmsne entry points

htsne(data, emb_dim=2, alpha=1.0, max_components=50, Y=None, perplexity=30.0, max_its=1000, print_iter=100, eta=None, momentum=0.8, min_gain=0.01, min_value=1.0e-128, optimize_alpha=False, min_alpha=1.0e-6, max_alpha_its=3, seed=None, mix_model=None, enc_data=None, method='auto', early_exaggeration=None, tol=1.0e-7, dpm_max_its=200, affinity='auto', field_weights=None, evidence_cap=1.0, fisher_metric='diagonal', fisher_ridge=1.0e-8, fisher_information='observed', out=None, variable_length=False, barnes_hut_theta=0.5, barnes_hut_leaf_size=16, neighbor_method='auto', neighbor_threshold=5000, neighbor_trees=8, neighbor_leaf_size=None, candidate_multiplier=8, repulsion_method='auto', exact_repulsion_threshold=5000, goals=None)[source]

Embed heterogeneous data with model-based t-SNE.

goals: optional sequence of embedding goals (mixle.utils.hvis.goals) – Anchor pins for anchoring, LabelCohesion for partial labeling, AxisAlign for layout objectives. Goal gradients join the data gradient every iteration on BOTH engines; hard anchors are re-projected exactly after every step.

Y=’barycentric’ initializes every observation at its posterior-weighted combination of component vertices laid out by overlap geometry (see affinity.barycentric_init): the layout’s global arrangement comes from the model instead of the random seed, so runs are globally consistent and mixed-membership points start (and tend to stay) between their clusters.

early_exaggeration=None (the default) resolves to 12.0 for a random init and 1.0 for an informative one (Y=’barycentric’ or a supplied array): exaggeration exists to FORM global structure from randomness, and given a meaningful init it does the opposite – crushes confusable clusters together before the refine phase can save them (measured: 0.71-0.89 purity and seed-dependent arrangements at 12.0, a seed-stable 0.96 at 1.0). Pass a number to override.

A mixture model is fit to the data (a Dirichlet process mixture with automatically typed components by default, or pass mix_model), pairwise affinities are computed from the model, and the affinities are embedded with t-SNE. Passing affinity=’fisher’ with any model that exposes to_fisher(), or passing a pre-built affinity factor list, bypasses the mixture-posterior affinity path and does not require a DPM/mixture model.

method:

‘exact’ - full-matrix gradient descent (supports optimize_alpha) ‘barnes_hut’ - sparse model probabilities + internal Barnes-Hut t-SNE ‘auto’ - barnes_hut for n > 10 unless optimize_alpha is set

affinity:
‘auto’ (default) - ‘local’ whenever raw data is available and the

model decomposes into leaf fields, else ‘bhattacharyya’

‘local’ - per-field posterior overlap plus component-local

Mahalanobis geometry for continuous/count fields, estimated from the realized data; discrete fields fall back to posterior overlap

‘balanced’ - per-field posteriors (the model’s flattened leaves:

nested composites, sequence element/length models, and optional wrappers all decompose) combined by per-field Bhattacharyya, so a sharp discrete field cannot drown an overlapping continuous one (or vice versa); optional field_weights sets exponents on whole field-level Bhattacharyya coefficients

‘fisher’ - posterior-expected sufficient statistics from

mix_model.to_fisher(), whitened by an observed Fisher metric; fisher_information=’observed’ uses the empirical covariance of observed score vectors, while ‘model’ uses the view’s model metric; fisher_metric is ‘diagonal’ by default, with ‘identity’ and ‘full’ also accepted

‘bhattacharyya’ - Bhattacharyya coefficient between joint posteriors;

graded even under hard assignments, so embeddings retain within-cluster geometry

‘coassign’ - co-assignment probability P(z_i = z_j | x); exact but

near-binary when posteriors are sharp

‘likelihood’ - predictive affinity sum_k p(x_i|theta_k) z_jk

variable_length is retained for backward compatibility and does not rescale densities. Variable-length behavior is determined by the fitted sequence model: ordinary SequenceDistribution leaves use summed element log-likelihood with length as a separate field, while SequenceDistribution(len_normalized=True) intentionally uses a per-token composition quotient for the element field.

evidence_cap (default 1.0 nats) bounds the dissimilarity evidence any single field may contribute to a pair’s distance under multi-field affinities: without it, one spuriously sharp field (a serial-number-like categorical the model micro-clustered) drives its per-field affinity to zero and vetoes the pair’s similarity no matter what every other field says. None disables the cap; single-field affinities ignore it.

barnes_hut_theta controls the Barnes-Hut opening angle for method=’barnes_hut’; 0.0 gives exact repulsive forces and larger values are faster/coarser.

repulsion_method controls repulsive forces for method=’barnes_hut’: ‘exact’ uses a vectorized all-pairs calculation, ‘barnes_hut’ uses the tree approximation, and ‘auto’ uses exact repulsion when n is at most exact_repulsion_threshold.

neighbor_method controls graph construction for method=’barnes_hut’: ‘exact’ uses blockwise all-pairs top-k, ‘approx’ uses a random-projection candidate forest, and ‘auto’ switches to ‘approx’ when n >= neighbor_threshold.

Returns the n x emb_dim embedding.

Parameters:

  • emb_dim (int)

  • alpha (float)

  • max_components (int)

  • Y (ndarray | None)

  • perplexity (float | None)

  • max_its (int)

  • print_iter (int)

  • eta (float | None)

  • momentum (float)

  • min_gain (float)

  • min_value (float)

  • optimize_alpha (bool)

  • min_alpha (float)

  • max_alpha_its (int)

  • seed (int | None)

  • method (str)

  • early_exaggeration (float | None)

  • tol (float)

  • dpm_max_its (int)

  • evidence_cap (float | None)

  • fisher_metric (str)

  • fisher_ridge (float)

  • fisher_information (str)

  • variable_length (bool)

  • barnes_hut_theta (float)

  • barnes_hut_leaf_size (int)

  • neighbor_method (str)

  • neighbor_threshold (int)

  • neighbor_trees (int)

  • neighbor_leaf_size (int | None)

  • candidate_multiplier (int)

  • repulsion_method (str)

  • exact_repulsion_threshold (int)

class StreamingHvis(mix_model, landmark_data, *, atlas=None, emb_dim=2, alpha=1.0, perplexity=30.0, affinity='balanced', evidence_cap=1.0, field_weights=None, estimator=None, drift_threshold_nats=2.0, seed=None, **htsne_kwargs)[source]

Bases: object

A frozen model-based atlas that arriving points are placed into, with drift accounting.

Parameters:

  • mix_model (Any) – the fitted mixture the affinities come from (any model htsne accepts).

  • landmark_data (list) – the reservoir the atlas is built over. The model’s components make this easy to keep representative – e.g. sample a quota per component.

  • atlas (np.ndarray | None) – optional precomputed (len(landmark_data), emb_dim) coordinates (e.g. a humap layout). When omitted, the atlas is built here with htsne().

  • affinity (str) – any named HViS affinity. Note 'local' learns component-local metrics from the data the factors are built over, which during streaming is landmarks + batch – with a reasonably sized reservoir the landmarks dominate, but 'balanced' (the default) is a pure per-point function of the model and has no such coupling.

  • estimator (Any) – optional ParameterEstimator consistent with mix_model. When given, the MODEL streams too, by incremental EM (Neal & Hinton 1998): every add() batch is E-stepped once, at arrival time, under the model current at that moment, and its sufficient statistics accumulate; refresh() then performs one M-step over the reservoir’s statistics (E-stepped under the current model) combined with the accumulated stream statistics, adopting the re-estimated model before re-embedding. One honest EM sweep per refresh – NOT full-batch EM to convergence; pass refresh(mix_model=...) with your own fully re-fit model when that is what you want (an explicit model always wins, and discards the pending stream statistics).

  • drift_threshold_nats (float) – how far (in nats) the arrivals’ mean log-density may fall below the landmark reference before drifted trips.

  • htsne_kwargs (Any) – forwarded to htsne() for atlas builds and refresh().

  • emb_dim (int)

  • alpha (float)

  • perplexity (float | None)

  • evidence_cap (float | None)

  • seed (int | None)

add(batch, *, max_its=250, eta=None)[source]

Place a batch of arriving observations into the frozen atlas; returns (B, emb_dim).

Landmark coordinates are guaranteed unchanged by this call – stability is structural, not a tuning outcome. Also updates the running drift score from the batch’s log-density.

Parameters:
Return type:

ndarray

extend_landmarks(data, coords=None)[source]

Promote observations into the landmark reservoir (typically recent arrivals), placing them first if coordinates are not supplied. Grows the atlas without moving anything.

Parameters:
Return type:

None

drift_score()[source]

Nats of mean log-density the recent stream sits BELOW the landmark reference (>=0-ish; near zero or negative means the stream fits the model at least as well as the reservoir).

Return type:

float

refresh(mix_model=None)[source]

Re-embed the landmark reservoir (optionally under an updated model), warm-started from the current coordinates and rigidly aligned back onto them.

With an estimator configured and stream statistics pending, the model is re-estimated first (one incremental-EM M-step over reservoir + stream statistics) and the re-embed runs under the NEW model. An explicit mix_model argument always wins and discards the pending stream statistics – passing both a stream-updated posture and an external model would make the vintage of the statistics unaccountable.

Returns {"alignment_residual_rms", "alignment_scale", "atlas_spread", "n_landmarks", "model_updated", "n_stream_obs_consumed"}. A residual small relative to the spread means visual continuity is real; a large one means the embedding geometry genuinely changed and the report says so rather than hiding it in the alignment. Resets the drift accumulator (a refresh is the response to drift, so scoring restarts).

Parameters:

mix_model (Any)

Return type:

dict[str, Any]

place_in_atlas(p_rows, atlas, *, alpha=1.0, max_its=250, eta=None, momentum=0.8, tol=1.0e-7)[source]

Place each row’s point into a FROZEN atlas by minimizing its own row-KL under the t-kernel.

p_rows is (B, L) row-stochastic (each arriving point’s calibrated affinities over the L landmarks); atlas is (L, d). Each point’s objective involves only itself and the frozen landmarks, so the whole batch optimizes as one vectorized gradient descent. Initialized at the affinity-weighted barycenter of landmark coordinates.

Parameters:
Return type:

ndarray

class Anchor(indices, coordinates, weight=None)[source]

Bases: object

Pin indices to coordinates: hard when weight is None (exact projection each step), soft for weight in (0, 1] (close that fraction of the remaining gap per step).

Parameters:

  • indices (Sequence[int])

  • coordinates (Any)

  • weight (float | None)

class LabelCohesion(labels, weight=0.1, margin=None)[source]

Bases: object

Partial labels shape the layout: each labeled point moves toward its label’s centroid at weight (a per-step fraction in (0, 1]); with margin, centroid pairs closer than margin are pushed apart (every member displaced alike, which moves the centroid by exactly the intended amount). labels has one entry per point; None marks a point unlabeled.

Parameters:

  • labels (Sequence[Any])

  • weight (float)

  • margin (float | None)

class AxisAlign(values, axis=0, weight=0.5)[source]

Bases: object

A per-point scalar should run along embedding axis axis: each step ascends the Pearson correlation r(y[:, axis], values) along its scale-normalized direction (bounded norm <= 2, so weight – recommended at most ~1 – is a stable per-step rate in embedding units). Pass -values to reverse direction.

Parameters:
humap(data, emb_dim=2, n_neighbors=15, min_dist=0.1, max_components=50, seed=None, mix_model=None, enc_data=None, dpm_max_its=200, print_iter=100, affinity='auto', field_weights=None, evidence_cap=1.0, fisher_metric='diagonal', fisher_ridge=1.0e-8, fisher_information='observed', n_epochs=None, out=None, engine='auto', goals=None, **umap_kwargs)[source]

Embed heterogeneous data with model-based UMAP.

The same mixture-model affinities as htsne (see the affinity and evidence_cap arguments there), but the k-nearest-neighbor graph of model distances -log s_ij is handed to UMAP’s fuzzy simplicial set construction and layout instead of t-SNE. Scales like UMAP: the dense affinity matrix is never built.

engine selects the layout backend:
‘umap-learn’ - the optional umap-learn package (extra keyword

arguments are passed to umap.UMAP). Cannot honor goals: its numba SGD loop takes no external gradients, so goals raise rather than being silently dropped.

‘internal’ - mixle.utils.hvis.umap_np, a dependency-free UMAP core

(same construction: smoothed-kNN fuzzy graph, fitted a/b curve, epochs-per-sample SGD with negative sampling). Slower than the numba path but always available, and the only engine that can steer the layout with goals.

‘auto’ - umap-learn when it is installed AND no goals were

given; the internal engine otherwise.

goals: optional sequence of embedding goals (mixle.utils.hvis.goals) – Anchor / LabelCohesion / AxisAlign, as in htsne. Requires the internal engine (auto selects it when goals are present).

Parameters:

  • emb_dim (int)

  • n_neighbors (int)

  • min_dist (float)

  • max_components (int)

  • seed (int | None)

  • dpm_max_its (int)

  • print_iter (int)

  • evidence_cap (float | None)

  • fisher_metric (str)

  • fisher_ridge (float)

  • fisher_information (str)

  • n_epochs (int | None)

  • engine (str)

dpmsne(P=None, emb_dim=2, alpha=1.0, Y=None, max_its=1000, print_iter=100, eta=None, momentum=0.8, min_gain=0.01, min_value=1.0e-128, optimize_alpha=False, min_alpha=1.0e-6, max_alpha_its=3, seed=None, early_exaggeration=12.0, tol=1.0e-7, out=None, **_compat_kwargs)[source]

Embed a precomputed (symmetric, non-negative) affinity matrix P with exact t-SNE.

Parameters:
model_log_affinity(posterior_mat, ll_mat=None, affinity='bhattacharyya', evidence_cap=None)[source]

Dense n x n matrix of log affinities (see module docstring) with -inf diagonal.

Rows are comparable up to a per-row shift, which both the row-conditional normalization and per-row perplexity calibration are invariant to.

evidence_cap bounds the dissimilarity evidence any single factor (field) may contribute: each factor’s log affinity is floored at -evidence_cap nats before the factors are summed. Without the cap a single sharp field with (near-)disjoint per-field posteriors drives its log affinity to -inf and vetoes the pair no matter what every other field says; with it, a field can at most testify “these differ by evidence_cap nats”. The cap is only applied to multi-factor (per-field) affinities - for a single factor it could only create ties.

Parameters:
Return type:

ndarray

affinity_health(mix_model, data, *, affinity='auto', perplexity=30.0, field_weights=None, evidence_cap=1.0, max_rows=400, seed=0)[source]

Receipts for “why does my embedding look like this”: measure the affinity’s degeneracies BEFORE spending an optimization on them.

The classic failure this catches is posterior collapse: sharp posteriors make every same-component pair an exact tie, rows cannot reach the requested perplexity, and t-SNE renders each cluster as a tiny structureless point. That is a property of the AFFINITY, measurable in milliseconds – not a property of the optimizer, discoverable after a thousand iterations.

Returns a dict with per-field entries (geometry: 'local'/'fisher'/ 'posterior-only'; posterior_sharpness: mean max field-posterior, 1.0 = fully hard) and overall numbers on a row subsample of at most max_rows:

  • top_tie_fraction – mean fraction of each row’s neighbors tied (within 1e-9) with its best neighbor. Near 0 is healthy; large means nearest-neighbor structure is degenerate.

  • row_entropy_deficit_nats – mean shortfall between the requested log(perplexity) and the entropy each row can actually reach (ties saturate the calibration). 0 is healthy.

  • diagnosis – plain-language findings, empty when healthy.

Parameters:

  • perplexity (float | None)

  • evidence_cap (float | None)

  • max_rows (int)

  • seed (int)

Return type:

dict

log_affinity_block(factors, row_idx, col_idx, evidence_cap=None)[source]

Rectangular (rows x cols) log-affinity block – model_log_affinity() for a sub-block.

Mirrors the square path exactly: per-factor similarity blocks, log, per-factor evidence cap (multi-factor affinities only), weighted sum. Used by streaming placement (new points x landmarks) and by affinity_health() (subsampled diagnostics).

Parameters:
Return type:

ndarray

mixture_coordinates(mix_model, data, field_weights=None)[source]

The observation decomposition made first-class: x -> (posterior, remainder per field).

The mixture describes every observation at two levels, and this returns both explicitly: "posterior" – the (n, K) component posterior, literally barycentric coordinates on the simplex whose vertices are the components (the between-cluster geometry); "fields" – one entry per flattened leaf field with its per-component log-densities, its within-component coordinates ("coords": native value coordinates where the leaf has them, universal typicality coordinates otherwise), and "native" recording which. This is exactly the decomposition the ‘local’ affinity is built from; exposing it lets a caller inspect or plot the two levels directly (e.g. a ternary plot of the posterior for K=3) instead of trusting the embedding blindly.

Return type:

dict

component_map(z, emb_dim=2, *, method='nerve', edge_threshold=0.02)[source]

Lay out the K components as vertices by their overlap geometry on the data.

method='nerve' (default): geodesic layout of the cover’s nerve – edge lengths are -log BC on STRONG edges only (see mixle.utils.hvis.topology.fuzzy_nerve()), all-pairs shortest paths give the target metric, and deterministic stress majorization embeds it. This is Isomap on the nerve: a ring of components renders as a ring and a chain as a line, where bare MDS on the clipped dense -log BC matrix (every non-overlapping pair saturating at the same huge distance) distorts both – the classic horseshoe failure. Disconnected pieces of the nerve are laid out separately and placed side by side with an explicit gap; their on-screen separation is a RENDERING choice, which mixle.utils.hvis.topology.nerve_report() also says outright.

method='mds': the previous behavior – classical MDS on the dense clipped -log BC matrix. Kept as the fallback and for comparison.

Component confusability itself is unchanged: the Bhattacharyya coefficient between the components’ responsibility profiles. These vertices anchor barycentric_init() and mixle.utils.hvis.direct.model_map().

Parameters:
Return type:

ndarray

barycentric_init(z, emb_dim=2, *, jitter=0.15, seed=None)[source]

Initial embedding coordinates from the barycentric reading of the posterior.

Each observation starts at z @ vertices – its posterior-weighted combination of the component vertices from component_map() – so the layout’s GLOBAL arrangement (which clusters sit near which, where mixed-membership points fall) is decided by the model’s own geometry rather than by the random seed, and t-SNE’s optimization refines locally from there.

jitter is a fraction of the smallest nonzero inter-vertex distance and matters more than it looks: sharp posteriors put every same-regime point EXACTLY on its vertex, and t-SNE from near-coincident starts is chaotic (microscopic noise decides the layout) and slow to develop local structure. The decomposition needs both levels even at init time – the barycentric base supplies the between geometry, the jitter stands in for the within spread the optimization then makes real. Rescaled to the conventional 1e-4 standard deviation so optimizer dynamics (early exaggeration, learning rates) match the random-init path.

Parameters:
Return type:

ndarray

model_map(data, mix_model=None, emb_dim=2, *, spread=0.35, chart='linear', occlusion=True, occlusion_margin=1.05, edge_threshold=0.02, field_weights=None, max_components=50, dpm_max_its=200, seed=None, refine=False, refine_kwargs=None)[source]

The deterministic model-native layout (see module docstring). Returns a ModelMap.

spread sets how large regime fibers render relative to the smallest inter-vertex gap – a LEGIBILITY choice made explicit, unlike t-SNE where cluster sizes are a meaningless artifact. chart is 'linear' (default) or 'quadratic' (explicit degree-2 features – a curved within-regime chart that stays closed-form and placeable); either way chart_residuals reports the linear chart’s leftover variance per regime. occlusion=True enforces that components with no measured overlap never overlap on screen. seed/max_components/ dpm_max_its only matter when mix_model is None and a DPM must be fit first; the layout itself uses no randomness. refine=True polishes local neighborhoods with t-SNE initialized FROM this layout (exaggeration off), leaving the global arrangement model-decided.

Parameters:
Return type:

ModelMap

class ModelMap(coords, vertices, responsibilities, loadings, coord_labels, frames=<factory>, chart='linear', chart_residuals=<factory>, _model=None, _transforms=<factory>, _pre=<factory>, _fiber_means=<factory>, _fiber_scale=1.0, _emb_dim=2)[source]

Bases: object

A fitted direct layout: coordinates plus everything needed to read and extend the map.

vertices are the component anchors (post occlusion resolution); loadings[k] names what regime k’s chart axes measure (rows = chart features, see coord_labels); frames[k] is the chart’s on-screen frame (row 0 = the major axis’s direction); chart_residuals[k] is the fraction of fiber variance the LINEAR chart leaves beyond emb_dim (high = this regime’s within-structure is not 2-D-linear – consider chart='quadratic' or refine=True); place(data) maps NEW observations with the fit-time transforms – closed form, so streaming is one call.

Parameters:
place(data)[source]

Closed-form out-of-sample placement with the FIT-TIME transforms (means, whiteners, chart directions, frames, scale). Placing the training data reproduces coords exactly.

Return type:

ndarray

fuzzy_nerve(z, *, edge_threshold=0.02, triangle_threshold=0.02)[source]

Weighted 1- and 2-skeleton of the cover’s nerve, from the posteriors alone.

Edge weight w(k,l) = sum_i z_ik z_il / min(mass_k, mass_l) – the co-claimed fraction of the smaller component’s mass (1 when one component’s points are entirely co-claimed by the other, 0 when they never co-claim). Triangle weight is the same with a triple product. Simplices at or above their threshold are “strong” and drive nerve_report(); all nonzero weights are returned so thresholds are inspectable choices, not hidden ones.

Parameters:
Return type:

dict

nerve_report(nerve)[source]

Topology receipts from a fuzzy_nerve(): connected pieces, cycles, and candidate holes.

A hole is an independent cycle of strong edges not directly filled by a strong 2-simplex (3-cycles are checked exactly; longer cycles are conservatively reported as candidates). The diagnosis strings are the user-facing half: a loop in the cover is real data topology that a 2-D layout may distort silently.

Parameters:

nerve (dict)

Return type:

dict

embedding_health(coords, mix_model, data, *, affinity='auto', k=10, field_weights=None, evidence_cap=1.0, max_rows=400, seed=0)[source]

Rendering-fidelity receipt: do the map’s neighborhoods agree with the model affinity?

Standard trustworthiness (are map-neighbors genuinely close under the model?) and continuity (are model-neighbors kept close in the map?), computed on a row subsample. This audits the LAYOUT against the MODEL – a low score means the picture misrepresents the affinities that produced it (bad init, unconverged optimizer, non-embeddable topology). It does NOT audit the model against the raw data; that receipt is still open (design review R2).

Parameters:
Return type:

dict

model_fit_health(mix_model, data, *, holdout=None, field_weights=None, coverage_q=0.9, merged_sep_threshold=None, shattered_weight=0.5, min_component_points=20)[source]

The model<->data receipt (design review R2, second half): does the FITTED MODEL describe the data it is about to be a map of? Measured from the model’s own residual structure – no raw feature space is assumed, which is the whole point of HViS.

  • fiber calibration – per component, the squared Mahalanobis of its dominant points’ whitened fiber coordinates should look chi-squared: the fraction inside the coverage_q ball is compared against coverage_q. A large gap means the component’s shape claim is wrong (too wide, too narrow, or mis-shaped).

  • merged-regime detector – a deterministic 2-means split (top-PC sign init) of each component’s dominant fiber coordinates; a separation ratio above the threshold with a non-trivial minority says one component is covering what the data treats as two regimes (K too small). The threshold has a derivation plus a measured finite-sample correction: for a UNIMODAL normal the population statistic is 2 E|x| / sqrt(1 - 2/pi) ~ 2.65 regardless of scale, but at n=40 sample noise inflates it to ~3.4 (observed), so the default threshold is 2.65 + 6/sqrt(n) – ~3.6 at n=40, tightening toward the population value as n grows. Two unit-variance regimes 4 sigma apart score ~4.0 either way. Pass an explicit merged_sep_threshold to pin it.

  • shattered detector – nerve edges with weight >= shattered_weight are near-duplicate components claiming largely the same points (K too large).

  • held-out check – with holdout data, a mean log-density drop > 1 nat vs training is flagged (memorization / drift).

Parameters:

  • coverage_q (float)

  • merged_sep_threshold (float | None)

  • shattered_weight (float)

  • min_component_points (int)

Return type:

dict

component_tree(nerve)[source]

Single-linkage merge tree over components by nerve edge weight – the hierarchy skeleton.

Merges are emitted strongest-overlap-first: [{"a": frozenset, "b": frozenset, "weight": w, "merged": frozenset}, ...]. Cutting the tree at any weight gives coarse super-components (mixle.utils.hvis.front.Map uses it for zoom groups); the merge order is itself a receipt – which regimes are almost one regime.

Parameters:

nerve (dict)

Return type:

list[dict]

hvis_map(data, mix_model=None, emb_dim=2, *, spread=0.35, chart='linear', occlusion=True, refine=False, goals=None, health=True, holdout=None, field_weights=None, max_components=50, dpm_max_its=200, seed=None, refine_kwargs=None)[source]

One call, one finished map (see module docstring). Deterministic unless a DPM must be fit.

goals (anchoring / partial labels / axis objectives) require the optimizer pass, so passing them implies refine=True. health=False skips the receipt computations (they are cheap and subsampled; skip only in tight loops).

Parameters:
Return type:

Map

class Map(base, posterior_entropy=<factory>, typicality=<factory>, nerve=<factory>, nerve_health=<factory>, fit_health=<factory>, render_health=<factory>, merge_tree=<factory>, zoom_alignment_rms=None, _data=<factory>, _params=<factory>)[source]

Bases: object

A finished map: coordinates, anchors, per-point uncertainty, and every receipt.

Parameters:
property diagnosis: list[str]

Every receipt’s findings, one flat list – empty means no receipt has a complaint.

zoom(components)[source]

Re-chart one regime group with its own fibers: the sub-mixture over components maps the points they dominate, then the child layout is rigidly aligned (rotation/translation + uniform scale) onto those points’ PARENT positions – continuity is measured (zoom_alignment_rms), never assumed. Component indices in the child are positional within components.

Parameters:

components (list[int])

Return type:

Map

map(data, mix_model=None, emb_dim=2, *, spread=0.35, chart='linear', occlusion=True, refine=False, goals=None, health=True, holdout=None, field_weights=None, max_components=50, dpm_max_its=200, seed=None, refine_kwargs=None)

One call, one finished map (see module docstring). Deterministic unless a DPM must be fit.

goals (anchoring / partial labels / axis objectives) require the optimizer pass, so passing them implies refine=True. health=False skips the receipt computations (they are cheap and subsampled; skip only in tight loops).

Parameters:
Return type:

Map

sparse_model_distances(posterior_mat, ll_mat=None, k=90, block_size=1024, affinity='bhattacharyya', evidence_cap=None)[source]

Sparse n x n matrix of model distances d_ij = -log s_ij.

Keeps the k nearest neighbors (largest affinity) per row. Built blockwise so the dense n x n affinity matrix is never materialized. Distances are non-negative. evidence_cap as in model_log_affinity.

Parameters:
Return type:

csr_matrix

approx_sparse_model_distances(posterior_mat, ll_mat=None, k=90, affinity='bhattacharyya', evidence_cap=None, n_trees=8, leaf_size=None, candidate_multiplier=8, seed=None)[source]

Approximate sparse model distances without all-pairs graph construction.

A random-projection forest proposes candidate neighbors in normalized model-factor coordinates. Candidate pairs are then rescored with the exact model affinity used by sparse_model_distances, so approximation only enters through candidate recall. This is local/non-distributed today, but the proposal/evaluation split is the intended boundary for future distributed graph construction.

Parameters:
Return type:

csr_matrix

model_knn(posterior_mat, ll_mat=None, k=15, block_size=1024, affinity='bhattacharyya', evidence_cap=None)[source]

k-nearest-neighbor arrays under the model distance d_ij = -log s_ij.

Returns (indices, distances), each n x k, sorted ascending per row with each point as its own first neighbor at distance 0 (the convention expected by umap-learn, where self counts toward n_neighbors). Built blockwise; the dense affinity matrix is never materialized. evidence_cap as in model_log_affinity.

Parameters:
Return type:

tuple[ndarray, ndarray]

get_pmat(posterior_mat, ll_mat=None, targ_perplexity=None, vlen=False, affinity='bhattacharyya', evidence_cap=None)[source]

Symmetrized t-SNE input probabilities from model posteriors (and optionally component log-likelihoods, for affinity=’likelihood’).

The vlen flag is kept for backward compatibility and ignored.

Parameters:

  • affinity (str)

  • evidence_cap (float | None)

balanced_factors(mix_model, data, field_weights=None)[source]

Per-field Bhattacharyya affinity factors for heterogeneous models.

The joint posterior is dominated by whichever field has the largest log-likelihood contrast across components - sharp categorical fields, long token-sequence fields, or collapsed continuous components can contribute many nats of contrast while overlapping continuous fields contribute fractions of one. The drowned fields’ relationships then become invisible to any affinity computed from the joint posterior.

‘balanced’ fixes the scale problem at the affinity level: a field- restricted posterior z^f is computed from each field’s likelihoods alone (fields are the model’s flattened leaves - nested composites, sequence element/length models, and optional wrappers all decompose; see _field_log_densities), and the affinity combines per-field Bhattacharyya coefficients, so every field contributes comparably regardless of its likelihood scale. field_weights apply as exponents on whole field coefficients, i.e. weights on log field-affinities. Combined with an evidence cap (see model_log_affinity) no single field can veto a pair’s similarity either.

local_factors(mix_model, data, field_weights=None)[source]

Per-field local statistical affinity factors.

Each leaf field is first represented by its field-restricted component posterior, and EVERY field also carries within-component local geometry: continuous/count leaves (and averages of such leaves inside sequences) use their native coordinates; every other leaf – HMMs, Markov chains, categoricals, sequence-of-discrete element fields – uses typicality coordinates (per-component log-density; per-token rate plus a log-length axis for sequence-valued leaves, see _typicality_coordinates). Without that universal fallback, sharp posteriors make all same-component pairs exact ties and clusters render as tiny structureless points – the collapse this affinity exists to prevent. The factor carries component-local inverse covariances estimated from the realized data (which also makes heterogeneous fields dimensionless, so continuous, discrete, and sequence evidence are commensurate). Pair affinities then use

sum_k sqrt(z_ik z_jk) exp(-delta_ijk / 8),

where delta_ijk is the component-local Mahalanobis distance in that field’s coordinates. This is the local Fisher quadratic in the plug-in model, with posterior overlap handling component uncertainty.

fisher_factors(model, data=None, enc_data=None, metric='diagonal', ridge=1.0e-8, weight=1.0, information='observed')[source]

Fisher-vector affinity factor for a model and observations.

The model supplies posterior-expected sufficient statistics through to_fisher(). By default those statistics are treated as observed score vectors and whitened by their empirical observed Fisher covariance. Set information=’model’ to use the view’s model Fisher metric directly. Pair affinities are s_ij = exp(-0.5 ||v_i - v_j||^2).

Parameters:
tsne_barnes_hut(P, emb_dim=2, max_its=1000, eta=None, momentum=0.8, early_exaggeration=12.0, min_gain=0.01, tol=1.0e-7, print_iter=100, theta=0.5, leaf_size=16, repulsion_method='auto', exact_repulsion_threshold=5000, seed=None, Y=None, out=None)[source]

Embed a precomputed sparse t-SNE probability matrix with Barnes-Hut.

P must be a symmetric, non-negative affinity/probability matrix. It is normalized internally. This function is self-contained and does not call sklearn.

Parameters:
Return type:

ndarray

Submodules