API

struct FlexiChain{
    TKey,TMetadata<:NTuple{N,FlexiChainMetadata}
} <: AbstractMCMC.AbstractChains

An MCMC chain.

TODO: Document further.

Fields

• _data::OrderedCollections.OrderedDict{Union{FlexiChains.Extra, FlexiChains.Parameter{var"#s2"}} where var"#s2"<:TKey, Matrix} where TKey: Internal per-iteration data for parameters and extra keys. To access the data in here, you should index into the chain.

• _metadata::FlexiChains.FlexiChainMetadata: Other items associated with the chain. These are not necessarily per-iteration (for example there may only be one per chain).

  You should not access this field directly; instead you should use the accessor functions provided: FlexiChains.iter_indices, FlexiChains.chain_indices, FlexiChains.sampling_time, and FlexiChains.last_sampler_state.

• _structures::Matrix: A 'structure' object, one per sample, that can be used to upconvert the OrderedDict representation of the data into a richer information format. This is in general nothing, but Turing.jl in particular hijacks this field to store a skeleton VarNamedTuple, allowing for functions like parameters_at to return VarNamedTuples instead of OrderedDict{VarName}.

FlexiChains.FlexiSummary{
    TKey,
    TIIdx<:Union{DimensionalData.Lookup,Nothing},
    TCIdx<:Union{DimensionalData.Lookup,Nothing},
    TSIdx<:DimensionalData.Categorical,
}

A data structure containing summary statistics of a FlexiChain.

Construction

Calling summary functions such as mean or std on a FlexiChain will return a FlexiSummary. For more flexibility, you can use FlexiChains.collapse to apply one or more summary functions to a FlexiChain.

Users should not need to construct FlexiSummary objects directly.

Indexing

A FlexiSummary{TKey} can be indexed into using exactly the same techniques as a FlexiChain{TKey}. That is to say:

• with Parameter{TKey} or Extra to unambiguously get the summary statistics for that key
• with TKey for automatic conversion to Parameter{TKey}
• with Symbol to find unambiguous matches
• if TKey<:VarName, using VarName or sub-VarNames to additionally extract part of the data.

The returned value will be either a DimensionalData.DimArray (if there are one or more non-collapsed dimensions), or a single value (if all dimensions are collapsed).

If a DimArray is returned, the dimensions that you will see are: :iter (if the summary function was only applied over chains), :chain (same but for iterations), and :stat (typically seen when multiple summary functions were applied).

Extended help

Internal data layout

A FlexiSummary, much like a FlexiChain, contains a mapping of keys to arrays of data. However, the dimensions of a FlexiSummary are substantially different. In particular:

• The :iter and/or :chain dimensions may have been collapsed via the act of calculating a summary over iterations or chains.
• There is an additional, third, dimension: the statistic dimension, represented by `:stat. This dimension records which statistic(s) have been calculated.
• The :stat dimension may also have been collapsed. This can happen if only one statistic was computed and drop_stat_dim=true was used when calling FlexiChains.collapse. The purpose of this is to avoid making the user deal with a redundant singleton dimension when calling a function such as mean(chain).

Regardless of which dimensions have been collapsed, the internal data of a FlexiSummary always contains all three dimensions (some of which may have size 1).

Information about which dimensions are collapsed is therefore not stored in the arrays. Instead, it is stored in the _iter_indices and _chain_indices fields of the FlexiSummary, as well as their types. If either of these is nothing, then that dimension has been collapsed. For the stat dimension, _stat_indices always contains a DimensionalData.Categorical with the stat name(s), but the _drop_stat_dim field controls whether the stat dimension is presented to the user.

This information is later used in the _get_raw_data and _raw_to_user_data functions.

VNChain

Alias for FlexiChain{VarName}.

SymChain

Alias for FlexiChain{Symbol}.

Base.keytype(cs::ChainOrSummary{TKey})

Returns TKey.

FlexiChain{TKey}(
    niters::Int,
    nchains::Int,
    array_of_dicts::AbstractArray{<:AbstractDict,N};
    structures::AbstractArray{<:Any,N}=fill(nothing, size(array_of_dicts)),
    iter_indices::AbstractVector{Int}=1:niters,
    chain_indices::AbstractVector{Int}=1:nchains,
    sampling_time::AbstractVector{<:Union{Real,Missing}}=fill(missing, nchains),
    last_sampler_state::AbstractVector=fill(missing, nchains),
) where {TKey,N}

Construct a FlexiChain from a vector or matrix of dictionaries. Each dictionary corresponds to one iteration.

Data

Each dictionary must be a mapping from a ParameterOrExtra{<:TKey} (i.e., either a Parameter{<:TKey} or an Extra) to its value at that iteration.

If array_of_dicts is a vector (i.e., N = 1), then niter is the length of the vector and nchains is 1. If array_of_dicts is a matrix (i.e., N = 2), then (niter, nchains) = size(dicts).

Other values of N will error.

Structures

The structures keyword argument can be used to provide an array of 'structures'. See the docstring for the _structures field for more information. The array must have the same size as array_of_dicts.

Metadata

iter_indices and chain_indices can be used to specify the iteration and chain indices, respectively. By default, these are 1:niters and 1:nchains, but can be any vector of integers of the appropriate length. sampling_time and last_sampler_state are used to store metadata about each chain. They should be given as vectors of length nchains (even if there is only one chain).

Example usage

using FlexiChains: FlexiChain, Parameter, Extrad = fill(    Dict(Parameter(:x) => rand(), Extra(:y) => rand()), 200, 3)chn = FlexiChain{Symbol}(200, 3, d)

FlexiChain{TKey}(
    niters::Int,
    nchains::Int,
    dict_of_arrays::AbstractDict{<:Any,<:AbstractArray{<:Any,N}};
    structures::AbstractArray{<:Any,N}=fill(nothing, niters, nchains),
    iter_indices::AbstractVector{Int}=1:niters,
    chain_indices::AbstractVector{Int}=1:nchains,
    sampling_time::AbstractVector{<:Union{Real,Missing}}=fill(missing, nchains),
    last_sampler_state::AbstractVector=fill(missing, nchains),
) where {TKey,N}

Construct a FlexiChain from a dictionary of arrays.

Data

Each key in the dictionary must subtype ParameterOrExtra{<:TKey} (i.e., it is either a Parameter{<:TKey} or an Extra). The values of the dictionary must all be of the same size.

If the values are vectors (i.e., N = 1), then niters will be the length of the vector, and nchains will be 1. If the values are matrices (i.e., N = 2), then (niter, nchains) = size(array).

Other values of N will error.

Metadata

iter_indices and chain_indices can be used to specify the iteration and chain indices, respectively. By default, these are 1:niters and 1:nchains, but can be any vector of integers of the appropriate length. sampling_time and last_sampler_state are used to store metadata about each chain. They should be given as vectors of length nchains (even if there is only one chain).

Example usage

using FlexiChains: FlexiChain, Parameter, Extrad = Dict(    Parameter(:x) => rand(200, 3),    Extra(:y) => rand(200, 3),)chn = FlexiChain{Symbol}(200, 3, d)

FlexiChains.EXAMPLE_CHAIN

An example FlexiChain{VarName} object, obtained by sampling from eight-schools-centered using NUTS.

This chain contains enough "interesting" behaviour to illustrate the core capabilities of FlexiChains. Specifically, it has:

• multiple iterations and chains;
• iteration numbers beginning from 101 (this number is made up, but doesn't matter);
• both Parameters and Extras;
• a non-Float64 type (the Bool of is_divergent);
• a vector parameter (theta).

Parameter{T}(name::T)

A named parameter in a FlexiChain.

If you have a FlexiChain{TKey}, then all parameters must be of type Parameter{TKey}.

To obtain the name of a Parameter, use FlexiChains.get_name.

Extra(name::Symbol)

A key in a FlexiChain that is not a parameter. This is used to store per-draw information, such as the log-density associated with the draw.

The name must be a Symbol, regardless of the parameter key type of the FlexiChain.

To obtain the name of a Extra, use FlexiChains.get_name.

ParameterOrExtra{T}

Alias for Union{Parameter{T},Extra}.

All keys in a FlexiChain{T} must satisfy k isa ParameterOrExtra{<:T}.

FlexiChains.get_name(p::Parameter{T})::T
FlexiChains.get_name(e::Extra)::Symbol

Unwrap the Parameter or Extra to get its name.

Base.:(==)(c1::FlexiChain{TKey1}, c2::FlexiChain{TKey2}) where {TKey1,TKey2}
Base.:(==)(c1::FlexiSummary{TKey1}, c2::FlexiSummary{TKey2}) where {TKey1,TKey2}

Equality operator for FlexiChains and FlexiSummarys. Two chains (or summaries) are equal if they have the same key type, the same size, the same data for each key, and the same metadata (which includes dimensional indices, sampling time, and sampler states).

Base.isequal(c1::FlexiChain{TKey1}, c2::FlexiChain{TKey2})::Bool where {TKey1,TKey2}
Base.isequal(c1::FlexiSummary{TKey1}, c2::FlexiSummary{TKey2})::Bool where {TKey1,TKey2}

Equality operator for FlexiChains that treats missing and NaN values as equal if they appear in the same positions.

FlexiChains.has_same_data(
    c1::FlexiChain{TKey1},
    c2::FlexiChain{TKey2};
    strict=false
) where {TKey1,TKey2}

Check if two FlexiChains have the same data, ignoring metadata such as sampling time, iteration indices, and chain indices.

If strict=true, then Base.:(==) is used to compare the data, which propagates missing values and treats NaN values as unequal. If strict=false (the default), then Base.isequal is used, which treats missing and NaN values as equal to themselves.

Base.size(chain::FlexiChain[, dim::Int])

Returns (niters, nchains), or niters or nchains if dim=1 or dim=2 is specified.

Base.size(summary::FlexiSummary[, dim::Int])

Returns (niters, nchains, nstats), or niters, nchains, or nstats if dim=1, dim=2, or dim=3 is specified. If any of the dimensions have been collapsed, the corresponding value will be 0.

FlexiChains.niters(chain::FlexiChain)

The number of iterations in the FlexiChain. Equivalent to size(chain, 1).

FlexiChains.niters(summary::FlexiSummary)

The number of iterations in the FlexiSummary. Equivalent to size(summary, 1). Returns 0 if the iteration dimension has been collapsed.

FlexiChains.nchains(chain::FlexiChain)

The number of chains in the FlexiChain. Equivalent to size(chain, 2).

FlexiChains.nchains(summary::FlexiSummary)

The number of chains in the FlexiSummary. Equivalent to size(summary, 2). Returns 0 if the chain dimension has been collapsed.

FlexiChains.nstats(summary::FlexiSummary)

The number of statistics in the FlexiSummary. Equivalent to size(summary, 3). Returns 0 if the statistics dimension has been collapsed (this means that there is a single statistic, but its name is not stored or displayed to the user).

iter_indices(chain::FlexiChain)::DimensionalData.Lookup

The indices of each MCMC iteration in the chain. This tries to reflect the actual iteration numbers from the sampler: for example, if you discard the first 100 iterations and sampled 100 iterations but with a thinning factor of 2, this will be 101:2:300.

The accuracy of this field is reliant on the sampler providing this information, though.

iter_indices(summary::FlexiSummary)::DimensionalData.Lookup

The iteration indices, which are either the same as in the original chain, or nothing if the iter dimension has been collapsed.

chain_indices(chain::FlexiChain)::DimensionalData.Lookup

The indices of each MCMC chain in the chain. This will pretty much always be 1:nchains(chain) (unless the chain has been subsetted, or chain indices have been manually specified).

chain_indices(summary::FlexiSummary)::DimensionalData.Lookup

The chain indices, which are either the same as in the original chain, or nothing if the chain dimension has been collapsed.

stat_indices(summary::FlexiSummary)

The indices for each statistic in the summary. Returns nothing if the stat dimension has been collapsed (i.e. drop_stat_dim=true was used).

renumber_iters(
    chain::FlexiChain{TKey},
    iter_indices::AbstractVector{<:Integer}=1:niters(chain)
)::FlexiChain{TKey} where {TKey}

Return a copy of chain with the iteration indices replaced by iter_indices.

renumber_chains(
    chain::FlexiChain{TKey},
    chain_indices::AbstractVector{<:Integer}=1:nchains(chain)
)::FlexiChain{TKey} where {TKey}

Return a copy of chain with the chain indices replaced by chain_indices.

Prefixed(vn::VarName)

A struct that represents a VarName that might have an arbitrary prefix. This is useful for indexing into a chain with VarNames when the exact prefix is not known (or too verbose to construct). For example, this allows for data-processing code that is agnostic towards whether a submodel was used or not.

When indexing into a chain with a Prefixed{VarName} key, the chain will be searched for any VarName that ends with the target VarName. For example, if the key is Prefixed(@vn(x)), and the chain contains @vn(a.x), the data corresponding to @vn(a.x) will be returned. (If there are multiple matches, an error is thrown.)

Note that Prefixed is only supported for VarNames, and not for general keys.

Base.keys(cs::ChainOrSummary)

Returns the keys of the FlexiChain (or summary thereof).

To obtain only the parameters, use FlexiChains.parameters.

Base.haskey(cs::ChainOrSummary{TKey}, key::TKey) where {TKey}

Returns true if the FlexiChain or summary contains the given name as a parameter key.

Base.haskey(cs::ChainOrSummary{TKey}, key::ParameterOrExtra{<:TKey}) where {TKey}

Returns true if the FlexiChain or summary contains the given key.

VarNames.parameters(cs::ChainOrSummary{TKey}) where {TKey}

Returns a vector of parameter names in the FlexiChain or summary thereof.

VarNames.extras(cs::ChainOrSummary)

Returns a vector of non-parameter names in the FlexiChain or summary thereof.

Base.values(cs::ChainOrSummary; parameters_only::Bool=false)

Returns the values of the FlexiChain or FlexiSummary, i.e., the matrices obtained by indexing into the chain with each key.

If parameters_only is true, only the values corresponding to parameter keys are returned.

Base.pairs(cs::ChainOrSummary)

Returns an iterator over the key-value pairs of the FlexiChain or FlexiSummary.

If parameters_only is true, only the values corresponding to parameter keys are returned.

sampling_time(chain::FlexiChain):Vector

Return the time taken to sample each chain (in seconds). If the time was not recorded, this will be missing.

Note that this always returns a vector with length equal to the number of chains.

last_sampler_state(chain::FlexiChain)::Vector

Return the final state of the sampler used to generate the chain, if the save_state=true keyword argument was passed to sample. This can be used for resuming MCMC sampling.

If the state was not saved for a given chain, its entry in the vector will be missing.

Base.vcat(cs...::FlexiChain{TKey}) where {TKey}

Concatenate one or more FlexiChains along the iteration dimension. Both c1 and c2 must have the same number of chains and the same key type.

The resulting chain's keys are the union of both input chains' keys. Any keys that only have data in one of the arguments will be assigned missing data in the other chain during concatenation.

The resulting chain's sampling time is the sum of the input chains' sampling times, and the last sampler state is taken from the second chain.

Base.hcat(cs...::FlexiChain{TKey}) where {TKey}

Concatenate one or more FlexiChains along the chain dimension. Both c1 and c2 must have the same number of iterations and the same key type.

The resulting chain's keys are the union of both input chains' keys. Any keys that only have data in one of the arguments will be assigned missing data in the other chain during concatenation.

The resulting chain's sampling times and last sampler states are obtained by concatenating those of the input chains.

Base.merge(
    c1::FlexiChain{TKey1},
    c2::FlexiChain{TKey2}
) where {TKey1,TKey2}

Merge the contents of two FlexiChains. If there are keys that are present in both chains, the values from c2 will overwrite those from c1.

If the key types are different, the resulting FlexiChain will have a promoted key type, and a warning will be issued.

The two FlexiChains being merged must have the same dimensions.

The chain indices and metadata are taken from the second chain. Those in the first chain are silently ignored.

subset_parameters(cs::ChainOrSummary)

Subset a chain or summary, retaining only the Parameter keys.

subset_extras(cs::ChainOrSummary)

Subset a chain or summary, retaining only the keys that are Extras (i.e. not parameters).

DynamicPPL.logprior(model::DynamicPPL.Model, chain::FlexiChain{<:VarName})

Returns a DimMatrix of the log-prior probabilities, re-evaluated using the parameters at each iteration of the chain.

DynamicPPL.loglikelihood(model::DynamicPPL.Model, chain::FlexiChain{<:VarName})

Returns a DimMatrix of the log-likelihoods, re-evaluated using the parameters at each iteration of the chain.

DynamicPPL.logjoint(model::DynamicPPL.Model, chain::FlexiChain{<:VarName})

Returns a DimMatrix of the log-joint probabilities, re-evaluated using the parameters at each iteration of the chain.

DynamicPPL.returned(model::DynamicPPL.Model, chain::FlexiChain{<:VarName})

Returns a DimMatrix of the model's return values, re-evaluated using the parameters in each iteration of the chain.

If the return value is an AbstractArray, the dimensions will be stacked.

DynamicPPL.predict(
    [rng::Random.AbstractRNG,]
    model::DynamicPPL.Model,
    chain::FlexiChain{<:VarName};
    include_all::Bool=true,
)

Returns a new FlexiChain containing predictions for variables in the model, conditioned on the parameters in each iteration of the input chain.

The returned FlexiChain by default will contain all the predicted variables, as well as the variables already present in the input chain. If you only want the predicted variables, set include_all=false.

The returned chain will also contain log-probabilities corresponding to the re-evaluation of the model. In particular, the log probability for the newly predicted variables are now considered as prior terms. However, note that the log-prior of the returned chain will also contain the log-prior terms of the parameters already present in the input chain. Thus, if you want to obtain the log-probability of the predicted variables only, you can subtract the two log-prior terms. The include_all keyword argument has no effect on the log-probability fields.

DynamicPPL.pointwise_logdensities(
    model::Model,
    chain::FlexiChain{<:VarName};
    factorize::Bool=false,
)::FlexiChain{VarName}

Calculate the log probability density associated with each variable in the model, for each iteration in the FlexiChain.

Returns a new FlexiChain with the same structure as the input chain, mapping the variables to their log probabilities.

If factorize=true, additionally attempt to provide factorised log-densities for distributions that can be partitioned into blocks, using PartitionedDistributions.jl.

For example, if factorize=true, then y ~ MvNormal(...) will return a vector of log-densities, one for each element of y. The i-th element of this vector will be the conditional log-probability of y[i] given all the other elements of y (often denoted log p(y_{i} | y_{-i})): in particular this is exactly the log-density required for leave-one-out cross-validation.

In contrast, if factorize=false, then the log-density for y ~ MvNormal(...) will be a single scalar corresponding to logpdf(MvNormal(...), y).

Note that the sum of the factorised log-densities may not, in general, be equal to the log-density of the full distribution: they will only be equal if the original distribution can be completely factorised into independent components. For example, if y ~ MvNormal(μ, Σ) where Σ is diagonal, then each element of y is independent and the sum of the factorised log-densities will be equal to the log-density of the full distribution. In contrast, if Σ has off-diagonal entries, then the elements of y are not independent.

DynamicPPL.pointwise_loglikelihoods(
    model::Model,
    chain::FlexiChain{<:VarName};
    factorize::Bool=false,
)::FlexiChain{VarName}

Calculate the log likelihood associated with each observed variable in the model, for each iteration in the FlexiChain.

Returns a new FlexiChain with the same structure as the input chain, mapping the observed variables to their log probabilities.

If factorize=true, additionally attempt to provide factorised log-densities for distributions that can be partitioned into blocks, using PartitionedDistributions.jl.

For example, if factorize=true, then y ~ MvNormal(...) will return a vector of log-densities, one for each element of y. The i-th element of this vector will be the conditional log-probability of y[i] given all the other elements of y (often denoted log p(y_{i} | y_{-i})): in particular this is exactly the log-density required for leave-one-out cross-validation.

In contrast, if factorize=false, then the log-density for y ~ MvNormal(...) will be a single scalar corresponding to logpdf(MvNormal(...), y).

Note that the sum of the factorised log-densities may not, in general, be equal to the log-density of the full distribution: they will only be equal if the original distribution can be completely factorised into independent components. For example, if y ~ MvNormal(μ, Σ) where Σ is diagonal, then each element of y is independent and the sum of the factorised log-densities will be equal to the log-density of the full distribution. In contrast, if Σ has off-diagonal entries, then the elements of y are not independent.

DynamicPPL.pointwise_prior_logdensities(
    model::Model,
    chain::FlexiChain{<:VarName};
    factorize::Bool=false, 
)::FlexiChain{VarName}

Calculate the log prior associated with each random variable in the model, for each iteration in the FlexiChain.

Returns a new FlexiChain with the same structure as the input chain, mapping the observed variables to their log probabilities.

If factorize=true, additionally attempt to provide factorised log-densities for distributions that can be partitioned into blocks, using PartitionedDistributions.jl.

For example, if factorize=true, then y ~ MvNormal(...) will return a vector of log-densities, one for each element of y. The i-th element of this vector will be the conditional log-probability of y[i] given all the other elements of y (often denoted log p(y_{i} | y_{-i})): in particular this is exactly the log-density required for leave-one-out cross-validation.

In contrast, if factorize=false, then the log-density for y ~ MvNormal(...) will be a single scalar corresponding to logpdf(MvNormal(...), y).

Note that the sum of the factorised log-densities may not, in general, be equal to the log-density of the full distribution: they will only be equal if the original distribution can be completely factorised into independent components. For example, if y ~ MvNormal(μ, Σ) where Σ is diagonal, then each element of y is independent and the sum of the factorised log-densities will be equal to the log-density of the full distribution. In contrast, if Σ has off-diagonal entries, then the elements of y are not independent.

DynamicPPL.InitFromParams(
    chn::FlexiChain{<:VarName},
    iter::Union{Int,DD.At},
    chain::Union{Int,DD.At},
    fallback::Union{AbstractInitStrategy,Nothing}=InitFromPrior()
)::DynamicPPL.InitFromParams

Use the parameters stored in a FlexiChain as an initialisation strategy.

AbstractMCMC.from_samples(
    ::Type{<:OldVNChain},
    params_and_stats::AbstractMatrix{<:DynamicPPL.ParamsWithStats}
)::OldVNChain

Convert a matrix of DynamicPPL.ParamsWithStats to a OldVNChain.

AbstractMCMC.from_samples(
    ::Type{<:OldVNChain},
    params_and_stats::AbstractMatrix{<:DynamicPPL.VarNamedTuple}
)::OldVNChain

Convert a matrix of DynamicPPL.VarNamedTuple to a OldVNChain.

AbstractMCMC.from_samples(
    ::Type{<:VNChain},
    draws::AbstractMatrix{<:VarNames.Draw}
)::OldVNChain

Convert a matrix of VarNames.Draw to a VNChain.

AbstractMCMC.from_samples(
    ::Type{<:VNChain},
    params_and_stats::AbstractMatrix{<:VarNamedTuple}
)::VNChain

Convert a matrix of VarNames.VarNamedTuple to a VNChain.

AbstractMCMC.to_samples(
    ::Type{DynamicPPL.ParamsWithStats},
    chain::OldVNChain,
    [model::DynamicPPL.Model]
)::DimensionalData.DimMatrix{DynamicPPL.ParamsWithStats}

Convert a OldVNChain to a DimMatrix of DynamicPPL.ParamsWithStats.

The axes of the DimMatrix are the same as those of the input OldVNChain.

AbstractMCMC.to_samples(
    ::Type{VarNames.Draw},
    chain::FlexiChain{T},
)::DD.DimMatrix{<:VarNames.Draw} where {T<:VarName}

Convert a VNChain to a matrix of VarNames.Draw objects.

AbstractMCMC.to_samples(
    ::Type{VarNamedTuple},
    chain::FlexiChain{T},
)::DD.DimMatrix{<:VarNamedTuple} where {T<:VarName}

Convert a VNChain to a matrix of VarNames.VarNamedTuple objects.