mixle.inference.objectives module¶

Generic objective optimization and variational projection utilities.

This module is intentionally model-facing rather than engine-facing: callers provide objective functions, distributions provide their own scoring math, and compute engines provide only tensor arithmetic/autograd.

class ObjectiveFitResult(model, value, iterations, history=(), converged=False, initial_value=None, final_delta=None, maximize=True, best_value=None, best_iteration=None, final_gradient_norm=None)[source]

Bases: object

Optimization result for arbitrary differentiable objectives.

By default objective helpers restore the best trainable state seen during optimization. In that mode value and model refer to best_value / best_iteration while history still records every attempted iterate and final_delta describes the last attempted step.

Parameters:
model: Any
value: float
iterations: int
history: tuple[float, ...] = ()
converged: bool = False
initial_value: float | None = None
final_delta: float | None = None
maximize: bool = True
best_value: float | None = None
best_iteration: int | None = None
final_gradient_norm: float | None = None
as_tuple()[source]

Return the historical (model, value) shape used by fit helpers.

Return type:

tuple[Any, float]

property objective_change: float | None

Return the signed objective change from the start of optimization.

property improvement: float | None

Return positive improvement in the requested optimization direction.

property best_improvement: float | None

Return best positive improvement seen during optimization.

class ObjectiveParameter(name, value, constraint='real')[source]

Bases: object

Named trainable parameter for arbitrary differentiable objectives.

Supported constraints are real, positive / positive_vector / positive_matrix, unit_interval, simplex / simplex_vector, row_simplex_matrix, column_simplex_matrix, greater_than:<name>, and less_than:<name>. Coupled bound constraints refer to an earlier parameter in the same set. The optimizer stores unconstrained raw tensors and presents constrained values to the objective callable.

Parameters:
name: str
value: Any
constraint: str = 'real'
class ObjectiveParameterSet(parameters, engine=None, precision=None, torch=None)[source]

Bases: object

Engine-backed named parameters for user-supplied objectives.

Parameters:

  • parameters (Any)

  • engine (Any | None)

  • precision (Any | None)

  • torch (Any | None)

trainable_tensors()[source]

Return raw tensors passed to the optimizer.

Return type:

Sequence[Any]

values()[source]

Return constrained tensors keyed by parameter name.

Return type:

Mapping[str, Any]

detached_values()[source]

Return plain Python/NumPy constrained values keyed by parameter name.

Return type:

Mapping[str, Any]

class ExpectedLogDensity(weights=None, normalize=False)[source]

Bases: object

Objective sum_i w_i log q_model(x_i) for encoded observations.

Parameters:

  • weights (Sequence[float] | None)

  • normalize (bool)

class ObjectiveSum(*objectives)[source]

Bases: object

Add several model objectives into one scalar objective.

Parameters:

objectives (ObjectiveCallable)

class UnnormalizedLogLikelihood(log_unnormalized, log_partition=None, partition_enc=None, reference_log_density=None, weights=None, normalize=False)[source]

Bases: object

Objective for models specified by unnormalized log likelihoods.

The objective is

sum_i w_i * log f_theta(x_i) - sum_i w_i * log Z(theta)

where log f_theta is supplied by the caller. log Z(theta) can be supplied exactly with log_partition or estimated from reference samples using self-normalized importance form logmeanexp(log f_theta(y_j) - log q(y_j)).

Parameters:

  • log_unnormalized (ObjectiveCallable)

  • log_partition (Callable[[Any, Any], Any] | None)

  • partition_enc (Any | None)

  • reference_log_density (Callable[[Any, Any], Any] | None)

  • weights (Sequence[float] | None)

  • normalize (bool)

class CallableObjective(fn, name='callable_objective')[source]

Bases: object

Small adapter naming arbitrary objective callables.

Parameters:

  • fn (ObjectiveCallable)

  • name (str)

fit_objective(enc, model, objective, engine=None, max_its=500, lr=0.05, optimizer='adam', tol=1.0e-7, maximize=True, out=None, print_iter=100, return_result=False, precision=None, restore_best=True)[source]

Optimize a user-supplied differentiable objective over a distribution tree.

The objective is called as objective(shadow_model, enc, engine) and must return a scalar engine tensor. Distribution parameters are obtained from the same declaration-driven raw state used by fit_mle / fit_map. When restore_best is true, the returned model/value correspond to the best objective value seen, not necessarily the last attempted optimizer step.

Parameters:
Return type:

Any

variational_projection(source, target, data=None, enc=None, sample_size=1000, seed=None, engine=None, max_its=500, lr=0.05, optimizer='adam', tol=1.0e-7, out=None, print_iter=100, return_result=False, precision=None, restore_best=True)[source]

Project source onto the family represented by target.

This minimizes a Monte-Carlo estimate of the forward KL KL(source || target) by maximizing E_source[log target(X)]. Pass data/enc to reuse common random numbers or a curated design set; otherwise samples are drawn from source.

Parameters:

  • source (SequenceEncodableProbabilityDistribution)

  • target (SequenceEncodableProbabilityDistribution)

  • data (Sequence[Any] | None)

  • enc (Any | None)

  • sample_size (int)

  • seed (int | None)

  • engine (Any | None)

  • max_its (int)

  • lr (float)

  • optimizer (str)

  • tol (float)

  • out (Any | None)

  • print_iter (int)

  • return_result (bool)

  • precision (Any | None)

  • restore_best (bool)

Return type:

Any

optimize_torch_objective(parameters, objective, engine=None, max_its=500, lr=0.05, optimizer='adam', tol=1.0e-7, maximize=True, out=None, print_iter=100, precision=None, return_result=False, restore_best=True)[source]

Optimize an arbitrary Torch objective over supplied tensor parameters.

This is the escape hatch for models whose likelihood is not an iid distribution score, such as Gaussian-process marginal likelihoods or supervised neural-network losses. When restore_best is true, supplied tensors are copied back to their best seen values before returning.

Parameters:
Return type:

Any

fit_parameter_objective(parameters, objective, enc=None, engine=None, max_its=500, lr=0.05, optimizer='adam', tol=1.0e-7, maximize=True, out=None, print_iter=100, precision=None, return_result=False, restore_best=True)[source]

Optimize an arbitrary objective over named constrained parameters.

parameters may be a mapping of name -> initial_value for real parameters, a sequence of ObjectiveParameter objects, or a sequence of (name, initial_value, constraint) tuples. objective is called as objective(params, enc, engine), where params is a mapping of constrained engine tensors. When restore_best is true, returned detached values are the best seen named parameters.

Parameters:
Return type:

Any

projection_samples(source, sample_size, seed=None)[source]

Draw reusable samples for Monte-Carlo projection experiments.

Parameters:

  • source (SequenceEncodableProbabilityDistribution)

  • sample_size (int)

  • seed (int | None)

Return type:

Sequence[Any]