PosteriorStats.jl
Documentation for PosteriorStats.jl ↗
Interval estimation
PosteriorStats.jl provides the hdi and eti functions for computing highest density intervals and equal-tailed intervals, respectively. These are overloaded for FlexiChain objects in much the same way as Statistics.mean, Statistics.std, etc. (see the Summarising page for more information on those). Here is an example:
using FlexiChains, PosteriorStatschn = FlexiChains.EXAMPLE_CHAINPosteriorStats.hdi(chn; prob=0.95)╭─FlexiSummary (2 statistics) ─────────────────────────────────────────────────╮
│ iter collapsed │
│ chain collapsed │
│ ↓ stat = [hdi_lower, hdi_upper] │
│ │
│ Parameters (10) ── VarName │
│ Float64 mu, tau, theta[1], theta[2], theta[3], theta[4], theta[5] │
│ theta[6], theta[7], theta[8] │
│ │
│ Extras (2) │
│ Float64 logjoint │
│ Bool is_divergent │
│ │
│ Summary │
│ param hdi_lower hdi_upper │
│ mu 0.2848 11.3517 │
│ tau 0.8460 7.4828 │
│ theta[1] -0.3124 15.5577 │
│ theta[2] -2.2672 11.4664 │
│ theta[3] -0.7276 11.5705 │
│ theta[4] -8.1504 12.1310 │
│ theta[5] 0.0934 10.9676 │
│ theta[6] -4.6604 12.0114 │
│ theta[7] 0.1807 18.4144 │
│ theta[8] -1.4280 12.7658 │
╰──────────────────────────────────────────────────────────────────────────────╯By default, each interval is split into its lower and upper bounds, which are stored as separate statistics (hdi_lower and hdi_upper, or eti_lower and eti_upper). If you would rather have a single statistic containing IntervalSets.ClosedIntervals, pass split_interval=false:
PosteriorStats.hdi(chn; prob=0.95, split_interval=false)╭─FlexiSummary ────────────────────────────────────────────────────────────────╮
│ iter collapsed │
│ chain collapsed │
│ stat collapsed │
│ │
│ Parameters (10) ── VarName │
│ IntervalSets.ClosedInterval{Float64} mu, tau, theta[1], theta[2], theta[3] │
│ theta[4], theta[5], theta[6] │
│ theta[7], theta[8] │
│ │
│ Extras (2) │
│ IntervalSets.ClosedInterval{Float64} logjoint │
│ IntervalSets.ClosedInterval{Bool} is_divergent │
│ │
│ Summary │
│ param │
│ mu 0.284841948… │
│ tau 0.846032171… │
│ theta[1] -0.31238771… │
│ theta[2] -2.26717921… │
│ theta[3] -0.72758344… │
│ theta[4] -8.15037228… │
│ theta[5] 0.093401193… │
│ theta[6] -4.66043740… │
│ theta[7] 0.180668752… │
│ theta[8] -1.42800850… │
╰──────────────────────────────────────────────────────────────────────────────╯LOO-CV
PosteriorStats.loo, which computes Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO), is also overloaded for FlexiChain objects.
To make use of this, you will need to have a chain that contains log-likelihood values. Specifically, instead of mapping VarNames to posterior samples, you will need to map them to log-likelihood values for each observation in your dataset. For example, if you have x ~ Normal() and chn[@vn(x)] contains the posterior samples for x, you will need to make a new chain llike_chn for which llike_chn[@vn(x)] = logpdf.(Normal(), chn[@vn(x)]).
If you have a Turing model, this functionality is made much easier by the DynamicPPL.pointwise_loglikelihoods function. However, you almost certainly want to look at the corresponding documentation page for FlexiChains v0.6, since Turing currently depends only on v0.6.
Docstrings
PosteriorStats.hdi — Method
PosteriorStats.hdi(
chain::FlexiChain{TKey};
dims::Symbol=:both,
warn::Bool=true,
split_varnames::Bool=true,
kwargs...
) where {TKey}Calculate the highest density interval across all iterations and chains for each key in the chain. If the statistic cannot be computed for a key, that key is skipped and a warning is issued (which can be suppressed by setting warn=false).
The dims keyword argument specifies which dimensions to collapse. The default value of :both collapses both the iteration and chain dimensions. Other valid values are :iter or :chain, which respectively collapse only the iteration or chain dimension.
The split_varnames keyword argument, if true, will first split up variables in the chain such that each key corresponds to a single scalar value. This is only supported for chains with TKey<:VarName or TKey==Symbol; for other key types this will be a no-op as long as the data already contain scalar values for every key, but will error if the data contain non-scalar values.
Other keyword arguments are forwarded to PosteriorStats.hdi; please see its documentation for details of supported keyword arguments.
Splitting intervals
By default (split_interval=true), the output FlexiSummary has one statistic column per interval bound, i.e., hdi_lower and hdi_upper. Set split_interval=false to instead get a single statistic column named hdi containing IntervalSets.ClosedIntervals.
Splitting is only possible when a single interval is computed per parameter (i.e., method=:unimodal, which is the default in PosteriorStats.jl). If you use a multimodal method (e.g. method=:multimodal), each parameter can have several intervals, so the returned FlexiSummary will always have a single statistic column named hdi containing a vector of intervals. In this case, a warning is emitted unless you pass split_interval=false.
PosteriorStats.eti — Method
PosteriorStats.eti(
chain::FlexiChain{TKey};
dims::Symbol=:both,
warn::Bool=true,
split_varnames::Bool=true,
kwargs...
) where {TKey}Calculate the equal-tailed interval across all iterations and chains for each key in the chain. If the statistic cannot be computed for a key, that key is skipped and a warning is issued (which can be suppressed by setting warn=false).
The dims keyword argument specifies which dimensions to collapse. The default value of :both collapses both the iteration and chain dimensions. Other valid values are :iter or :chain, which respectively collapse only the iteration or chain dimension.
The split_varnames keyword argument, if true, will first split up variables in the chain such that each key corresponds to a single scalar value. This is only supported for chains with TKey<:VarName or TKey==Symbol; for other key types this will be a no-op as long as the data already contain scalar values for every key, but will error if the data contain non-scalar values.
Other keyword arguments are forwarded to PosteriorStats.eti; please see its documentation for details of supported keyword arguments.
Splitting intervals
By default (split_interval=true), the output FlexiSummary has one statistic column per interval bound, i.e., eti_lower and eti_upper. Set split_interval=false to instead get a single statistic column named eti containing IntervalSets.ClosedIntervals.
PosteriorStats.loo — Method
PosteriorStats.loo(chn::FlexiChains; kwargs...)Calculates the leave-one-out cross-validation (LOO) statistic for a FlexiChain object. The chain must contain only log-likelihood values, and they must all be stored as parameters (not extras). Parameters that map to arrays of log-likelihood values are supported: they will be flattened before being passed to PosteriorStats.loo.
Returns a struct with the following fields:
param_names::Vector: A vector of parameter names whose log-likelihood values were used.loo::PosteriorStats.PSISLOOResult: The return value ofPosteriorStats.looapplied to the log-likelihood values extracted from theFlexiChain. This contains the statistics of interest.
Additional keyword arguments are forwarded to PosteriorStats.loo.
PosteriorStats.loo — Method
PosteriorStats.loo(
model::DynamicPPL.Model,
posterior_chn::FlexiChains;
factorize::Bool=false,
kwargs...
)Calculates the leave-one-out cross-validation (LOO) statistic, given a model plus a posterior chain. This first uses the model and posterior chain to calculate pointwise log-likelihoods, and then uses those to calculate the LOO statistic.
Returns a struct with the following fields:
param_names::Vector: A vector of parameter names whose log-likelihood values were used.loo::PosteriorStats.PSISLOOResult: The return value ofPosteriorStats.looapplied to the log-likelihood values extracted from theFlexiChain. This contains the statistics of interest.
The factorize keyword argument is passed to DynamicPPL.pointwise_loglikelihoods. If factorize=true, factorised log-densities will be calculated for distributions that can be partitioned into blocks (e.g. MvNormal). Please see the docstring of DynamicPPL.pointwise_loglikelihoods for more details.
Additional keyword arguments are forwarded to PosteriorStats.loo.