API

VarNamedTuple{names,Values}

A NamedTuple-like structure with VarName keys.

VarNamedTuple is a data structure for storing arbitrary data, keyed by VarNames, in an efficient and type stable manner. It is mainly used through getindex, setindex!!, templated_setindex!!, and haskey, all of which only accept VarNames as keys. Other notable methods are merge and subset.

VarNamedTuple has an ordering to its elements, and two VarNamedTuples with the same keys and values but in different orders are considered different for equality and hashing. Iterations such as keys and values respect this ordering. The ordering is dependent on the order in which elements were inserted into the VarNamedTuple, though isn't always equal to it. More specifically

• Any new keys that have a joint parent VarName with an existing key are inserted after that key. For instance, if one first inserts, in order, @vn(a.x), @vn(b), and @vn(a.y), the resulting order will be (@vn(a.x), @vn(a.y), @vn(b)).
• Index keys, like @vn(a[3]) or @vn(b[2,3,4:5]), are always iterated in the same order an Array with the same indices would be iterated. For instance, if one first inserts, in order, @vn(a[2]), @vn(b), and @vn(a[1]), the resulting order will be (@vn(a[1]), @vn(a[2]), @vn(b)).

Otherwise insertion order is respected.

setindex!! and getindex on VarNamedTuple are type stable as long as one does not store heterogeneous data under different indices of the same symbol. That is, if either

• one sets a[1] and a[2] to be of different types, or
• if a[1] and a[2] both exist, one sets a[1].b without setting a[2].b,

then getting values for a[1] or a[2] will not be type stable.

@vnt begin ... end

Construct a VarNamedTuple from a block of assignments. Each assignment should be of the form var := value, where var is a variable name. This is best illustrated by example:

julia> using VarNames

julia> @vnt begin
           a := 1
           b := 2
       end
VarNamedTuple
├─ a => 1
└─ b => 2

You can set entirely arbitrary variables:

julia> @vnt begin
           a.b.c.d.e := "hello"
       end
VarNamedTuple
└─ a => VarNamedTuple
        └─ b => VarNamedTuple
                └─ c => VarNamedTuple
                        └─ d => VarNamedTuple
                                └─ e => "hello"

For variables that have indexing, it is often necessary to provide a template, so that the VNT 'knows' what kind of array is being used to store the values, and can set the values in the appropriate places (consider e.g. OffsetArrays where x[1] may not mean what it usually does).

This is done by inserting a @template macro call in the block. The @template macro accepts whitespace-separated arguments, which must either be

• a single symbol (e.g. @template x), in which case the template is the value of x (and x must already be defined in the current scope); or
• an assignment of the form y = expr, in which case the template for y is the value of expr. In this case y does not need to be defined in the current scope, but any symbols referenced in expr must be.

For example:

julia> x = zeros(5); outside_y = zeros(3, 3);

julia> @vnt begin
            @template x y=outside_y
            x[1] := 1.0
            y[1, 1] := 2.0
       end
VarNamedTuple
├─ x => PartialArray size=(5,) data::Vector{Float64}
│       └─ (1,) => 1.0
└─ y => PartialArray size=(3, 3) data::Matrix{Float64}
        └─ (1, 1) => 2.0

If no template is provided, the VNT will use a GrowableArray. This can produce correct results in simple cases, but is not recommended for general use. Please see the VarNamedTuple documentation for more details.

empty!!(vnt::VarNamedTuple)

Create an empty version of vnt in place.

This differs from Base.empty in that any PartialArrays contained within vnt are kept but have their contents deleted, rather than being removed entirely. This means that

1. The result has a "memory" of how many dimensions different variables had, and you cannot, for example, set a[1,2] after emptying a VarNamedTuple that had only a[1] defined.
2. Memory allocations may be reduced when reusing VarNamedTuples, since the internal PartialArrays do not need to be reallocated from scratch.

Base.isempty(vnt::VarNamedTuple)

Check if vnt is empty.

Base.getindex(vnt::VarNamedTuple, vn::VarName)

Retrieve the value of vn stored in vnt.

julia> using VarNames

julia> vnt = @vnt begin
            a := 10
            b := [1, 2, 3, 4, 5]
            c := (; x = 1, y = 2)
       end
VarNamedTuple
├─ a => 10
├─ b => [1, 2, 3, 4, 5]
└─ c => (x = 1, y = 2)

julia> vnt[@vn(a)]
10

julia> vnt[@vn(b[3:5])]
3-element Vector{Int64}:
 3
 4
 5

julia> vnt[@vn(c.x)]
1

Base.view(vnt::VarNamedTuple, vn::VarName)

Retrieve vn from vnt, using views for array slices where possible. Integer and Cartesian scalar indices, and non-array values, behave as in getindex.

Array views share storage with vnt, so mutations through a view change the stored array.

julia> using VarNames

julia> vnt = @vnt begin
            a := 10
            b := [1, 2, 3, 4, 5]
            c := (; x = 1, y = 2)
       end
VarNamedTuple
├─ a => 10
├─ b => [1, 2, 3, 4, 5]
└─ c => (x = 1, y = 2)

julia> view(vnt, @vn(a))
10

julia> view(vnt, @vn(b[3:5]))
3-element view(::Vector{Int64}, 3:5) with eltype Int64:
 3
 4
 5

julia> view(vnt, @vn(c.x))
1

BangBang.setindex!!(vnt::VarNamedTuple, value, vn::VarName)

This is similar to templated_setindex!!, but does not take a template argument. In effect the template passed through is NoTemplate().

It is sometimes the case that the template is not actually needed. For example:

julia> vnt = VarNamedTuple()
VarNamedTuple()

julia> # Setting a top-level VarName.
       vnt = setindex!!(vnt, 10, @vn(a)) 
VarNamedTuple
└─ a => 10

julia> # Setting a value inside an array that already exists.
       vnt = setindex!!(vnt, zeros(2), @vn(b));
       vnt = setindex!!(vnt, 1.0, @vn(b[1]))
VarNamedTuple
├─ a => 10
└─ b => [1.0, 0.0]

julia> # Setting a value which only uses Property optics.
       vnt = setindex!!(vnt, 2.0, @vn(c.d.e))
VarNamedTuple
├─ a => 10
├─ b => [1.0, 0.0]
└─ c => VarNamedTuple
        └─ d => VarNamedTuple
                └─ e => 2.0

However, setindex!! may throw an error if the template is actually needed. Specifically, this is likely to happen if you set a VarName with dynamic optics or colons, since in those cases the size and shape of the underlying arrays cannot be inferred without a template. For example, this will error, since it is not known what x should be:

vnt = VarNamedTuple()
setindex!!(vnt, 10, @vn(x[1]))

apply!!(func, vnt::VarNamedTuple, name::VarName)

Apply func to the subdata at name in vnt, and set the result back at name.

Like map_values!!, but only for a single VarName.

julia> using VarNames: VarNamedTuple, setindex!!, apply!!

julia> vnt = VarNamedTuple()
VarNamedTuple()

julia> vnt = setindex!!(vnt, [1, 2, 3], @vn(a))
VarNamedTuple
└─ a => [1, 2, 3]

julia> apply!!(x -> x .+ 1, vnt, @vn(a))
VarNamedTuple
└─ a => [2, 3, 4]

map_pairs!!(func, vnt::VarNamedTuple)

Apply func to all key => value pairs of vnt, in place if possible.

func should accept a pair of VarName and value, and return the new value to be set.

map_values!!(func, vnt::VarNamedTuple)

Apply func to elements of vnt, in place if possible.

subset(vnt::VarNamedTuple, vns)

Create a new VarNamedTuple containing only the variables subsumed by ones in vns.

NoTemplate

A singleton struct representing the fact that there is no template for a top-level variable. When NoTemplate is used, several things happen:

• If you attempt to call make_leaf with an Index optic, this will error.
• When recursing into substructures, NoTemplate will be propagated.

Collectively this means that you can only set values for variables that only have Property optics, unless a template is provided.

It might seem more idiomatic to use nothing or missing for this. However, this causes a bug with BangBang.setindex!!: https://github.com/JuliaFolds2/BangBang.jl/issues/43 so we use a dedicated struct instead.

SkipTemplate{N}(value)

A struct representing the fact that value is the template for the variable N levels down from the top-level variable. In other words, SkipTemplate{0}(value) is equivalent to just value, and SkipTemplate{1}(value) means that value is the template for a when setting the variable @vn(x.a).

templated_setindex!!(vnt, value, vn, template; allow_new=Val(true))

Assign value to the location in vnt specified by vn.

The argument template must be provided in order to guide the creation of PartialArrays, as well as to concretise any dynamic indices in vn. It must be an object that has the shape of the top-level symbol in vn. For example:

vnt = VarNamedTuple()
templated_setindex!!(vnt, 10, @vn(x[1]), rand(2, 2))

Here, rand(2, 2) is the template for the top-level symbol x, which tells setindex!! that x should be a PartialArray that is backed by a matrix.

The actual data inside template is not needed, and template is never mutated by this call.

templated_setindex_no_overwrite!!(vnt, value, vn, template)

This is the same as templated_setindex!!, but throws an error if the location specified by vn already has a value in vnt. This is useful for ensuring that values in a VarNamedTuple are not set multiple times.

densify!!(vnt::VarNamedTuple)

Convert any PartialArrays in vnt where all elements are filled to just normal arrays.

skeleton(vnt::VarNamedTuple)

Create a skeleton of the given VarNamedTuple, which holds enough template information to reconstruct the original vnt from an ordered set of (VarName, value) pairs.

Specifically, we require that for any input orig_vnt that the following holds true:

skel = skeleton(orig_vnt)
# Check roundtrip reconstruction
new_vnt = VarNamedTuple()
for (vn, val) in pairs(orig_vnt)
    top_sym = getsym(vn)
    template = get(skel.data, top_sym, NoTemplate())
    new_vnt = templated_setindex!!(new_vnt, val, vn, template)
end
@assert new_vnt == orig_vnt

This is accomplished by recursively traversing vnt and dropping any elements that do not contain PartialArrays. PartialArrays on the other hand are replaced with arrays of the same shape and array type, but with the elements replaced with nothing (since that is space-efficient). This does not cause any issues with reconstruction type stability, since templated_setindex!! does not use the array's element type when creating a new entry (it infers the element type to use from the value that is being set).

Draw{P<:VarNamedTuple,S<:NamedTuple}

A struct that bundles a set of parameter values stored as a VarNamedTuple, along with a NamedTuple of extra information associated with this set of values (for example, this can be used to store log-probability values, etc.).

In VarNames.jl, this struct is not used at all; however, it is defined here so that other packages can dispatch on it easily.

VarNames.parameters(draw::Draw)::VarNamedTuple

Returns the parameters of the draw as a VarNamedTuple.

VarNames.extras(draw::Draw)::NamedTuple

Returns the extra information of the draw as a NamedTuple.