Why not FlexiChains?

Since I wrote one whole page saying why you should use FlexiChains, here are the counterarguments, or at least those I can think of.

Performance and type stability

FlexiChains uses dictionaries as its internal storage, and it is fundamentally not type stable to index into a dictionary with an abstract value type. Consequently, the vast majority of operations in FlexiChains are not type stable. They can sometimes also be slower than equivalent operations in MCMCChains.

Personally, I don't consider this to be important. Chain manipulation and data access are hardly performance bottlenecks in a typical Bayesian workflow. However, that isn't an excuse for gratuitously poor performance! If you find an instance where FlexiChains is unbearably slow, please do open an issue. I'm more than happy to look into it, or to suggest ways of working around it.

For the interested reader...

Performance differences typically arise because of the different data representation. In general, FlexiChains has a richer, more high-level, data structure, which avoids destroying information at early stages of processing. In contrast, MCMCChains immediately flattens everything into a 3D array.

If you feed a chain back into a Turing model, e.g. with predict, Turing is actually much happier with the high-level, original data representation. With MCMCChains you have to pay the cost of 'unflattening'. That's why FlexiChains is typically faster when interfacing with Turing models.

Now, flattening is actually quite an expensive operation since it involves essentially reshuffling the entire chain's data in memory and is therefore O(niters * nchains * nparams). Generally the means that things that involve flat representations of data (a simple example being conversion to DataFrame) are faster with MCMCChains, because this cost has already been paid upfront. With FlexiChains, you have to pay this cost every time you want to do something that requires a flat representation.

using FlexiChains: FlexiChain, Parameter
using Chairmarks, DataFrames

niter, nchain, nparam = 1000, 4, 100
d = [Dict(Parameter(:x) => randn(nparam)) for _ in 1:niter, _ in 1:nchain]
c = FlexiChain{Symbol}(niter, nchain, d)

# FlexiChains
@be DataFrame(c) samples=50 evals=1

Benchmark: 4 samples with 1 evaluation
        4.276 ms (10348 allocs: 15.887 MiB)
        5.056 ms (10348 allocs: 15.887 MiB)
        5.595 ms (10348 allocs: 15.887 MiB)
        1.740 s (10348 allocs: 15.887 MiB, 99.64% gc time)

using MCMCChains
m = MCMCChains.Chains(c)

# MCMCChains
@be DataFrame(m) samples=50 evals=1

Benchmark: 50 samples with 1 evaluation
 min    1.037 ms (1841 allocs: 6.300 MiB)
 median 1.113 ms (1841 allocs: 6.300 MiB)
 mean   1.477 ms (1841 allocs: 6.300 MiB)
 max    2.748 ms (1841 allocs: 6.300 MiB)

However, you should be aware that this is not entirely a fair comparison, and the 'flattening' part of MCMCChains has already taken place when the chain is constructed. In particular, if you are doing some sampling via sample(...; chain_type=MCMCChains.Chains), note that the chain construction is part of the sample call, and thus it will appear as if you are just waiting for MCMC sampling to finish, when in fact you are also waiting for the chain to be flattened. (If you have noticed MCMC sampling often being stuck at 100% for a while, this is why.)

To show a fairer comparison, we can flatten the FlexiChain first. Note that DataFrame(c) defers to DataFrame(Wide(c)) (see the Tables.jl integration section for more details). The constructor of Wide does the flattening for us, so if we pre-compute that, you will find that FlexiChains' performance is not so bad after all!

using FlexiChains: Wide
w = Wide(c)
@be DataFrame(w) samples=50 evals=1

Benchmark: 50 samples with 1 evaluation
 min    527.012 μs (1143 allocs: 3.288 MiB)
 median 560.947 μs (1143 allocs: 3.288 MiB)
 mean   2.746 ms (1143 allocs: 3.288 MiB, 1.95% gc time)
 max    109.523 ms (1143 allocs: 3.288 MiB, 97.45% gc time)

Feature set

There are still one or two more plotting and statistics functions that MCMCChains has that FlexiChains does not have yet.

(Note that this isn't entirely a drawback: there are things in FlexiChains that MCMCChains doesn't have too.)

I would be very happy to accept PRs porting some of this functionality to FlexiChains!

In the meantime, you can always convert your FlexiChain to a DataFrame, or an MCMCChains.Chains if you really need to.