Model Families

mixle.models is an incubating namespace for applied model helpers that sit above the core distribution families. Use it when a specialized model family is genuinely part of the problem: a neural likelihood leaf, a Gaussian-process surrogate, a random-forest conditional, a graph model, an induced grammar, a knowledge-graph embedding, a POMDP, a truncated DPM helper, or a training-loop utility.

For ordinary distribution modeling, mixle.stats and mixle.inference remain the primary surface. mixle.models is the applied-model layer for specialized families that need additional training, optional dependencies, or domain-specific conventions.

The design goal is still compositional. Where practical, helpers expose the same estimator, distribution, sampler, scoring, or fit-result conventions used by the rest of Mixle. That makes it possible for a Gaussian process, a random forest conditional, a Transformer leaf, or a learned grammar to participate in a larger heterogeneous model instead of living in a separate terminal pipeline.

Maturity Summary

Area	Public surface	Maturity	Use when
Neural and language leaves	LM, TransformerLMEstimator, StreamingTransformer, DPOModel, CategoricalEmbedding, NeuralDensity, NeuralConditionalDensity, VAE, Flow, MAF, DiscreteAR	Incubating	A neural likelihood is one channel inside a larger model, or you are experimenting with compact local language-model helpers.
Gaussian processes	GaussianProcessRegressor, sparse GP helpers	Usable, evolving	You need a smooth surrogate or uncertainty-aware response surface.
Random-forest conditionals	RandomForestEstimator, RandomForestConditional	Usable, evolving	You want a nonlinear conditional leaf p(y | x) that can be scored as part of a broader model.
Random graphs	ErdosRenyiGraphModel, StochasticBlockGraphModel	Usable for small graph workflows	The observation is a graph or block structure is the quantity of interest.
DPMs, grammars, knowledge graphs, POMDPs	fit_truncated_dpm, fit_induced_pcfg, TransEKnowledgeGraphModel, PartiallyObservableMarkovDecisionProcessModel	Research/prototype	You need a specialized family and are prepared to validate the result against held-out data or task loss.
Dependence discovery and training search	learn_pc_skeleton, orient_v_structures, TrainingSpace, tune_training, ewc	Research/prototype	You want proposals, diagnostics, or experiment scaffolding rather than a final model contract.

Choosing A Family

Data pattern	Public surface	Typical role
Text, token streams, or sequence context	LM, TransformerLMEstimator, StreamingTransformer	Incubating neural likelihood leaf inside a hybrid model.
Preference triples	DPOModel	Experimental preference-optimized leaf over chosen/rejected responses.
Smooth regression with uncertainty	GaussianProcessRegressor	Smooth surrogate or uncertainty-aware response surface.
Tabular conditional prediction	RandomForestEstimator, RandomForestConditional	Nonlinear conditional leaf p(y | x).
Flexible neural density	NeuralDensity, NeuralConditionalDensity, VAE, Flow, MAF, DiscreteAR, build_mdn, build_conditional_flow	Torch-backed density leaf for exact, bounded, mixture, or conditional neural likelihoods.
Unknown clustering structure	fit_truncated_dpm, TruncatedDirichletProcessMixtureModel	Variational truncated Dirichlet-process mixture helper.
Conditional independence structure	learn_pc_skeleton, orient_v_structures	Propose a dependency skeleton before modeling records.
Symbolic or structured sequences	fit_induced_pcfg, viterbi_parse	Learn an induced heterogeneous PCFG.
Knowledge graph triples	TransEKnowledgeGraphModel	Entity/relation embedding model.
Graph-valued observations	ErdosRenyiGraphModel, StochasticBlockGraphModel	Random graph likelihoods and block structure.
Controlled latent dynamics	PartiallyObservableMarkovDecisionProcessModel, baum_welch_pomdp	Hidden state filtering with action-conditioned transitions.
Hyperparameter and training policy	TrainingSpace, tune_training, ewc	Experiment scaffolding for neural components.

Neural And Language Leaves

For Transformer and LLM-centered modeling, start with Neural and LLM Models. Treat these objects as experimental adapters around Torch-backed models. They are useful when a neural likelihood has to compose with classical fields, but they carry more dependency, reproducibility, and training-state risk than the core stats families.

That guide covers:

• LM for a compact causal language model with direct fit, nll, and generate methods.

• TransformerLMEstimator for fitting (context, next_token) observations as a Mixle estimator-compatible leaf.

• StreamingTransformer when you already have a Torch module and want it to participate in Mixle’s accumulation and scoring contract.

• DPOModel for direct preference optimization.

• CategoricalEmbedding for tying token embeddings across leaves.

• NeuralDensity and NeuralConditionalDensity for Torch modules that expose explicit log-density methods.

• VAE, Flow, MAF, and DiscreteAR for constructible neural density families that can be dropped into estimator trees without manually building and wrapping a Torch module.

• build_mdn for multimodal conditional density and build_conditional_flow for exact conditional normalizing flows.

Neural Builder Inventory

Import	Role
make_mlp	Build a small MLP body for neural helpers and experiments.
CategoricalClassificationNeuralNetwork	Categorical classifier helper.
GaussianRegressionNeuralNetwork	Gaussian regression helper.
PoissonRegressionNeuralNetwork	Count regression helper.
NeuralGaussian / NeuralCategorical	Torch-backed likelihood leaves for regression and classification.
VAE / Flow / MAF / DiscreteAR	Constructible neural-density distribution families.
build_causal_lm	Low-level causal language-model module builder used by LM.
stream_fit	Streaming fit helper for Transformer leaves.

The model-family view is that these objects are likelihood components. They can be used alone, but their strategic value is larger: they can be children of mixtures, HMM emissions, record fields, density gates, or task-distillation cascades.

Serialization support has been broadened for neural leaves, direct LMs, streaming Transformer leaves, DPO leaves, and neural-density leaves. Prefer the documented save/load or to_dict/to_json routes for artifacts, and still keep a held-out behavioral check around restored neural models.

Gaussian Processes

GaussianProcessRegressor is an exact GP regression model with stationary kernels and Gaussian observation noise. It supports RBF, Matern-3/2, Matern-5/2, and related stationary kernels through the kernel= argument.

Use a GP when you need a smooth response surface, uncertainty over predictions, or sample-efficient modeling of an expensive scientific or operational signal. For large datasets or production use, validate runtime, numerical conditioning, and calibration on the target problem.

from mixle.models import GaussianProcessRegressor

gp = GaussianProcessRegressor(kernel="matern52", lengthscale=1.0, noise=0.05)
gp.fit(x_train, y_train, steps=200)
mean, cov = gp.predict(x_train, y_train, x_query, return_cov=True)

GPs are a natural partner for design-of-experiments workflows in Design of Experiments, Bayesian optimization in Evolution And Search, and calibrated prediction in Uncertainty.

Random Forest Conditionals

RandomForestEstimator fits a native NumPy random forest as a conditional distribution. Observations are (x, y) pairs, and the fitted RandomForestConditional scores log p(y | x).

This is a pragmatic conditional leaf, not a substitute for a full feature engineering and model-governance stack. Use held-out proper scores and calibration checks before embedding it in a larger decision system.

from mixle.inference import optimize
from mixle.models import RandomForestEstimator

rows = [
    ([0.2, 1.0, 3.5], "approve"),
    ([1.7, 0.4, 2.2], "review"),
]

model = optimize(rows, RandomForestEstimator(task="classification"), max_its=1)
score = model.log_density(([0.4, 0.9, 3.1], "approve"))

Because the result is a probability distribution over targets conditional on features, it can be embedded into a broader model where other fields are generative, latent, neural, or calibrated.

Dirichlet-Process Mixtures

fit_truncated_dpm fits a truncated Dirichlet-process mixture by variational updates. Component M-steps are delegated to ordinary Mixle estimators, so the component family can be Gaussian, categorical, composite, or another compatible distribution family.

from mixle.models import fit_truncated_dpm
from mixle.stats import GaussianDistribution, GaussianEstimator

initial = [GaussianDistribution(-2.0, 1.0), GaussianDistribution(2.0, 1.0)]
result = fit_truncated_dpm(
    data,
    initial_components=initial,
    component_estimator=GaussianEstimator(),
    alpha=1.0,
)

model = result.model

Use this when the number of clusters is uncertain but you still want a finite, inspectable fitted object. Treat truncation level, initialization, and held-out likelihood as part of the model specification.

Low-level DPM helpers include stick_breaking_weights, mean_stick_weights, expected_log_stick_weights, and sample_crp_assignments. TruncatedDirichletProcessMixtureFitResult records the fitted model and variational diagnostics.

Dependence Discovery

The dependence helpers expose conditional-independence tests and PC-style structure discovery:

• ConditionalIndependenceResult records the result of an independence test.

• gaussian_partial_correlation estimates partial correlation.

• gaussian_conditional_independence tests conditional independence for continuous Gaussian-like data.

• discrete_conditional_mutual_information measures discrete conditional dependence.

• learn_pc_skeleton returns a CausalSkeleton.

• orient_v_structures turns a skeleton into a PartiallyDirectedGraph.

These functions are not a substitute for domain assumptions. They are a way to turn a wide heterogeneous table into a candidate dependency structure that can then guide a record model, graphical model, or PPL specification.

from mixle.models import learn_pc_skeleton, orient_v_structures

skeleton = learn_pc_skeleton(table, alpha=0.01, method="gaussian")
graph = orient_v_structures(skeleton)

For production use, treat discovered structure as a proposal. Hold out data, compare alternatives with proper scores, and keep a record of rejected edges.

Grammars And Structured Sequences

fit_induced_pcfg learns a heterogeneous probabilistic context-free grammar from sequences. Terminal emissions are delegated to ordinary Mixle estimators, which lets a grammar cover mixed token types rather than assuming a single flat vocabulary.

After fitting:

• pcfg_log_likelihood scores sequences under the grammar.

• viterbi_parse returns the most likely parse tree.

• grammar_rule_table exposes learned rule probabilities.

• GrammarLearningResult and PCFGParseNode carry fitted grammar and parse-tree metadata.

This is useful when a sequence has latent compositional structure: commands, event traces, symbolic scientific strings, semi-structured logs, or mixed categorical/numeric segments.

Use induced grammars as candidate structure. Inspect learned rules and compare against simpler sequence models before treating a parse as meaningful.

Knowledge Graphs

TransEKnowledgeGraphModel models triples by embedding entities and relations into a shared space. KnowledgeGraphFitResult records fitted model metadata and training diagnostics.

Use it when the core object is relational rather than tabular: (head, relation, tail) triples, entity completion, relation plausibility, and knowledge-graph-derived features for larger models.

This helper is best understood as an embedding baseline for relational data, not as a complete knowledge-graph platform.

Random Graphs

The random graph helpers cover both homogeneous and block-structured graphs:

• ErdosRenyiGraphModel and fit_erdos_renyi_mle for one global edge probability.

• StochasticBlockGraphModel and fit_stochastic_block_mle for block membership and block-pair edge probabilities.

• hard_em_stochastic_block_model when block assignments are unknown.

• HardEMResult for hard-EM diagnostics and assignments.

These are useful for graph-valued observations, network monitoring, and extracting block assignments as latent features for another model.

POMDPs

PartiallyObservableMarkovDecisionProcessModel is the action-conditioned counterpart to an HMM. It tracks hidden state beliefs when transitions depend on actions and observations are noisy. baum_welch_pomdp fits the model from sequences with expectation-maximization.

PartiallyObservableMarkovDecisionProcessFitResult records fitted parameters and training diagnostics. PartiallyObservableMarkovDecisionProcessFilterResult records filtered beliefs for a sequence.

Use POMDPs when sequences include interventions, decisions, or controls: support tickets with actions, robot trajectories, treatment histories, or interactive agents.

The current surface is appropriate for experiments and small controlled models. Larger decision systems still need explicit simulator, policy, and evaluation infrastructure around it.

Training Search And Continual Learning

The neural training utilities are deliberately small:

• TrainingSpace describes tunable training parameters.

• tune_training evaluates a training function over that space.

• lm_train_fn adapts LM training into the search surface.

• extrapolate_learning_curve estimates future loss from observed steps.

• snapshot, fisher_diagonal, and ewc support elastic weight consolidation for continual learning.

• TrainingSearchResult records search outcomes.

Compatibility note: NeuralLeaf, SoftmaxNeuralLeaf, StreamingTransformerLeaf, and DPOLeaf remain importable aliases for the preferred names NeuralGaussian, NeuralCategorical, StreamingTransformer, and DPOModel.

For broader model-level search and anti-regression gates, use Evolution And Search. For task-level local model selection and LLM teachers, use Task Distillation.

Treat these as helpers for experiments. They are intentionally not a complete training platform.

Compositional Practice

The healthiest way to use mixle.models is to keep each family honest about its role:

• Use neural leaves for high-dimensional context, not as a place to hide every modeling assumption.

• Use GP and forest conditionals when a target is conditional on observed features.

• Use dependence discovery to propose structure, then verify the structure with held-out likelihood or decision loss.

• Use random graphs, grammars, and POMDPs when the observation itself carries structure that a flat table would destroy.

• Promote a candidate model only after it improves a proper score, calibration, or decision objective.

That is the forward direction for Mixle model families: specialist model classes that still compose under one inference, scoring, and uncertainty interface.