Aggregation
Index
Mill.AbstractAggregationMill.AggregationStackMill.BagCountMill.SegmentedLSEMill.SegmentedMaxMill.SegmentedMeanMill.SegmentedPNormMill.SegmentedSumMill.SegmentedMeanMaxMill.SegmentedPNormLSE
API
Mill.AbstractAggregation — TypeAbstractAggregationSupertype for any aggregation operator.
See also: AggregationStack, SegmentedSum, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
Mill.AggregationStack — TypeAggregationStack{T <: Tuple{Vararg{AbstractAggregation}}} <: AbstractAggregationA container that implements a concatenation of one or more AbstractAggregations.
Construct with e.g. AggregationStack(SegmentedMean([t::Type, ]d)) for single operators and with e.g. SegmentedPNormLSE([t::Type, ]d) for concatenations. With these calls all parameters inside operators are initialized randomly as Float32 arrays, unless type t is further specified. Another option is to vcat two operators together.
Nested stacks are flattened into a single-level structure upon construction, see examples.
Intended to be used as a functor:
(a::AggregationStack)(x, bags[, w])where x is either AbstractMatrix or missing, bags is AbstractBags structure and optionally w is an AbstractVector of weights.
Examples
julia> a = AggregationStack(SegmentedMean(2), SegmentedMax(2))
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0])
+
+julia> a(Float32[0 1 2; 3 4 5], bags([1:1, 2:3]))
+4×2 Matrix{Float32}:
+ 0.0 1.5
+ 3.0 4.5
+ 0.0 2.0
+ 3.0 5.0
+
+julia> a = AggregationStack(SegmentedMean(2), AggregationStack(SegmentedMax(2)))
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0])
+
+julia> a = SegmentedMeanMax(Float32, 2)
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0])
+
+julia> vcat(SegmentedMean(2), SegmentedMax(2))
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0])
+See also: AbstractAggregation, SegmentedSum, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
Mill.SegmentedSum — TypeSegmentedSum{V <: AbstractVector{<:Number}} <: AbstractAggregationAbstractAggregation implementing segmented sum aggregation:
$f(\{x_1, \ldots, x_k\}) = \sum_{i = 1}^{k} x_i$
Stores a vector of parameters ψ that are filled into the resulting matrix in case an empty bag is encountered.
See also: AbstractAggregation, AggregationStack, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
Mill.SegmentedMax — TypeSegmentedMax{V <: AbstractVector{<:Number}} <: AbstractAggregationAbstractAggregation implementing segmented max aggregation:
$f(\{x_1, \ldots, x_k\}) = \max_{i = 1, \ldots, k} x_i$
Stores a vector of parameters ψ that are filled into the resulting matrix in case an empty bag is encountered.
See also: AbstractAggregation, AggregationStack, SegmentedMean, SegmentedSum, SegmentedPNorm, SegmentedLSE.
Mill.SegmentedMean — TypeSegmentedMean{V <: AbstractVector{<:Number}}} <: AbstractAggregationAbstractAggregation implementing segmented mean aggregation:
$f(\{x_1, \ldots, x_k\}) = \frac{1}{k} \sum_{i = 1}^{k} x_i$
Stores a vector of parameters ψ that are filled into the resulting matrix in case an empty bag is encountered.
See also: AbstractAggregation, AggregationStack, SegmentedMax, SegmentedSum, SegmentedPNorm, SegmentedLSE.
Mill.SegmentedPNorm — TypeSegmentedPNorm{V <: AbstractVector{<:AbstractFloat}} <: AbstractAggregationAbstractAggregation implementing segmented p-norm aggregation:
$f(\{x_1, \ldots, x_k\}; p, c) = \left(\frac{1}{k} \sum_{i = 1}^{k} \vert x_i - c \vert ^ {p} \right)^{\frac{1}{p}}$
Stores a vector of parameters ψ that are filled into the resulting matrix in case an empty bag is encountered, and vectors of parameters p and c used during computation.
See also: AbstractAggregation, AggregationStack, SegmentedMax, SegmentedMean, SegmentedSum, SegmentedLSE.
Mill.SegmentedLSE — TypeSegmentedLSE{V <: AbstractVector{<:AbstractFloat}} <: AbstractAggregationAbstractAggregation implementing segmented log-sum-exp (LSE) aggregation:
$f(\{x_1, \ldots, x_k\}; r) = \frac{1}{r}\log \left(\frac{1}{k} \sum_{i = 1}^{k} \exp({r\cdot x_i})\right)$
Stores a vector of parameters ψ that are filled into the resulting matrix in case an empty bag is encountered, and a vector of parameters r used during computation.
See also: AbstractAggregation, AggregationStack, SegmentedMax, SegmentedMean, SegmentedSum, SegmentedPNorm.
Mill.SegmentedMeanMax — FunctionSegmentedMeanMax([t::Type, ]d::Int)Construct AggregationStack consisting of SegmentedMean and SegmentedMax operators.
Examples
julia> SegmentedMeanMax(4)
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0, 0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0, 0.0, 0.0])
+
+julia> SegmentedMeanMax(Float64, 2)
+AggregationStack:
+ SegmentedMean(ψ = [0.0, 0.0])
+ SegmentedMax(ψ = [0.0, 0.0])See also: AbstractAggregation, AggregationStack, SegmentedSum, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
Mill.SegmentedPNormLSE — FunctionSegmentedPNormLSE([t::Type, ]d::Int)Construct AggregationStack consisting of SegmentedPNorm and SegmentedLSE operators.
See also: AbstractAggregation, AggregationStack, SegmentedSum, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
Mill.BagCount — TypeBagCount{T <: AbstractAggregation}A wrapper type that when called applies the AbstractAggregation stored in it, and appends one more element containing bag size after $x ↦ \log(x + 1)$ transformation to the result.
Used as a functor:
(bc::BagCount)(x, bags[, w])where x is either AbstractMatrix or missing, bags is AbstractBags structure and optionally w is an AbstractVector of weights.
Examples
julia> x = Float32[0 1 2; 3 4 5]
+2×3 Matrix{Float32}:
+ 0.0 1.0 2.0
+ 3.0 4.0 5.0
+
+julia> b = bags([1:1, 2:3])
+AlignedBags{Int64}(UnitRange{Int64}[1:1, 2:3])
+
+julia> a = vcat(SegmentedMean(2), SegmentedMax(2))
+AggregationStack:
+ SegmentedMean(ψ = Float32[0.0, 0.0])
+ SegmentedMax(ψ = Float32[0.0, 0.0])
+
+julia> a(x, b)
+4×2 Matrix{Float32}:
+ 0.0 1.5
+ 3.0 4.5
+ 0.0 2.0
+ 3.0 5.0
+
+julia> BagCount(a)(x, b)
+5×2 Matrix{Float32}:
+ 0.0 1.5
+ 3.0 4.5
+ 0.0 2.0
+ 3.0 5.0
+ 0.693147 1.09861See also: AbstractAggregation, AggregationStack, SegmentedSum, SegmentedMax, SegmentedMean, SegmentedPNorm, SegmentedLSE.
+ 
+ 
+
+
+
+
+
![\"Iris](\"../assets/iris.svg\")
\n![\"Iris](\"../assets/iris-dark.svg\")
","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"where the input sample bmx is a vector (or generally speaking any tensor) of a fixed dimension containing various measurements of the specimen.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"Most of the time, a skilled botanist is able to identify a specimen not by making use of any measuring device, but by visual or tactile inspection of its stem, leaves and blooms. For different species, different parts of the flower may need to be examined for indicators. At the same time, many species may have nearly identical-looking leaves or blooms, therefore, one needs to step back, consider the whole picture, and appropriately combine lower-level observations into high-level conclusions about the given specimen.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"If we want to use such more elaborate description of the Iris flower using fixed size structures, we will have a hard time, because every specimen can have a different amounts of leaves or blooms (or they may be completely missing). This means that to use the usual fixed dimension paradigm, we have to either somehow select a single leaf (blossom) and extract features from them, or design procedures for aggregating such features over whole sets, so that the output has fixed dimension. This is clearly undesirable. Mill.jl a framework that seamlessly deals with these challenges in data representation.","category":"page"},{"location":"motivation/#Hierarchical-Multiple-Instance-Learning","page":"Motivation","title":"Hierarchical Multiple Instance Learning","text":"","category":"section"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"In Hierarchical Multiple Instance Learning (HMIL) the input may consists of not only sets, but also sets of sets and Cartesian Products of these structures. Returning to the previous Iris flower example, a specimen can be represented like this for HMIL:","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"![\"Iris](\"../assets/iris2.svg\")
\n![\"Iris](\"../assets/iris2-dark.svg\")
","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"The only stem is represented by vector bmx_s encoding its distinctive properties such as shape, color, structure or texture. Next, we inspect all blooms. Each of the blooms may have distinctive discriminative signs, therefore, we describe all three in vectors bmx_b_1 bmx_b_2 bmx_b_3, one vector for each bloom, and group them to a set. Finally, bmx_u represents the only flower which has not blossomed. Likewise, we could describe all leaves of the specimen if any were present. Here we assume that each specimen of the considered species has only one stem, but may have multiple flowers or leaves. Hence, all blooms and buds are represented as unordered sets of vectors as opposed to stem representation, which consists of only one vector.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"How does MIL models cope with variability in numbers of flowers and leaves? Each MIL model consists of two feed-forward neural networks with an element-wise aggregation operator like mean (or maximum) sandwiched between them. Denoting those feed-forward networks (FFNs) as f_1 and f_2, the output of the model applied to a bag is calculated for example as f_2 left(frac1lsum_i=1^l f_1(x_i) right) if we use mean as an aggregation function.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"The HMIL model corresponding to the Iris example above would comprise two FFNs and an aggregation to convert set of leafs to a single vector, and another two FFNs and an aggregation to convert set of blossoms to a single vector. These two outputs would be concatenated with a description of a stem, which would be fed to yet another FFN providing the final output. Since the whole scheme is differentiable, we can compute gradients and use any available gradient-based method to optimize the whole model at once using only labels on the level of output[2].","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"[2]: Some methods for MIL problems require instance-level labels as well, which are not always available.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"The Mill.jl library simplifies implementation of machine learning problems using (H)MIL representation. In theory, it can represent any problem that can be represented in JSONs. That is why we have created a separate tool, JsonGrinder.jl, which helps with processing JSON documents for learning.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"In [6], authors have further extended the Universal approximation theorem to MIL problems, their Cartesian products, and nested MIL problems, i.e. a case where instances of one bag are in fact bags again.","category":"page"},{"location":"motivation/#Relation-to-Graph-Neural-Networks","page":"Motivation","title":"Relation to Graph Neural Networks","text":"","category":"section"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"HMIL problems can be seen as a special subset of general graphs. They differ in two important ways:","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"In general graphs, vertices are of a small number of semantic type, whereas in HMIL problems, the number of semantic types of vertices is much higher (it is helpful to think about HMIL problems as about those for which JSON is a natural representation).\nThe computational graph of HMIL is a tree, which introduces assumption that there exist an efficient inference. Contrary, in general graphs (with loops) there is no efficient inference and one has to resort to message passing (Loopy belief propagation).\nOne update message in loopy belief propagation can be viewed as a MIL problem, as it has to produce a vector based on infomation inthe neighborhood, which can contain an arbitrary number of vertices.","category":"page"},{"location":"motivation/#Differences-from-sequence-based-modeling","page":"Motivation","title":"Differences from sequence-based modeling","text":"","category":"section"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"The major difference is that instances in bag are not ordered in any way. This means that if a sequence (abc) should be treated as a set, then the output of a function f should be the same for any permutation, i.e. f(abc) = f(cba) = f(bac) = ldots.","category":"page"},{"location":"motivation/","page":"Motivation","title":"Motivation","text":"This property has a dramatic implication on the computational complexity. Sequences are typically modeled using Recurrent Neural Networks (RNNs), where the output is calculated roughly as f(abc) = g(a g(b g(c))). During optimization, a gradient of g needs to be calculated recursively, giving raise to infamous vanishing / exploding gradient problems. In constrast, (H)MIL models calculate the output as f(frac13(g(a) + g(b) + g(c))) (slightly abusing notation again), which means that the gradient of g can be calculated in parallel and not recurrently. ","category":"page"},{"location":"examples/dag/#DAGs","page":"DAGs","title":"DAGs","text":"","category":"section"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"using Mill, Flux, Zygote, ChainRulesCore, Graphs","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"Imagine a data/knowledge base represented in a form of a directed acyclic graph (DAG), where a vertex would be modelled based on its parents (and their parents), but not on its descendants. We will make one assumption (common in graphical models) that two children are independent given their parent or, in other words, once we have access to the data or inferred values of the parent, we do not have to inspect its other children.","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"For example, in the graph below, when we infer some value for vertex e, we ignore vertices d, g, and h:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"(Image: )","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"Firstly, we define a new type for our graph that would be able to store a structure of the graph together with vertex features:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"struct DagGraph{G <: SimpleDiGraph,T}\n g::G\n vertex_features::T\nend\n\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"Then, we specify a model type:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"struct DagModel{M}\n m::M\n od::Int\nend\n\nFlux.@layer :ignore DagModel\n\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"Here, m is a function realizing one step of the message passing procedure, and od is output dimension.","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"In the course of calculating the value of vertex e, it may happen that for some vertices we will have to compute their value multiple times (for example for c, once when reached from e and once from f). Ideally, we would like to have a cache of already calculated values, which is difficult to do when autodifferentiating with Zygote as it does not support setindex operation. However, since the cache is assigned only once this can be realized through Zygote.buffer. We begin by initializing the cache:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"initcache(g, k) = [Zygote.Buffer(zeros(Float32, k, 1)) for _ in 1:nv(g)]\nChainRulesCore.@non_differentiable initcache(g, k)\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"To get the already computed value of a vertex when using the model, we just delegate the question to cache as ","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"function (model::DagModel)(g::DagGraph, i)\n cache = initcache(g.g, model.od)\n ArrayNode(getfromcache!(cache, g, model, i))\nend\n\n(model::DagModel)(g::SimpleDiGraph, vertex_features, i) = model(DagGraph(g, vertex_features), i)\n\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"which means that the getfromcache! will do all the heavy lifting. It turns out that this function just has to check if the value in cache has been already calculated. If not, it will calculate the value (applying model on millvertex!) and freeze the calculated item in cache:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"function getfromcache!(cache, g::DagGraph, model::DagModel, i::Int)\n cache[i].freeze && return(copy(cache[i]))\n ds = millvertex!(cache, g, model, i)\n cache[i][:] = model.m(ds)\n return(copy(cache[i]))\nend\n\nfunction getfromcache!(cache, g::DagGraph, model::DagModel, ii::Vector{Int}) \n reduce(catobs, [getfromcache!(cache, g, model, i) for i in ii])\nend\n\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"And what does millvertex! function do? It just takes the representation of ancestors (from cache) and put them together:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"function millvertex!(cache, g::DagGraph, model::DagModel, i)\n ProductNode(neighbours = millneighbors!(cache, g, model, i), \n vertex = vertex_features[i]\n )\nend\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"The last missing piece is millneighbors! definition:","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"function millneighbors!(cache, g::DagGraph, model::DagModel, ii::Vector{Int})\n isempty(ii) && return(BagNode(missing, [0:-1]))\n xs = [getfromcache!(cache, g, model, i) for i in ii]\n BagNode(ArrayNode(reduce(catobs, xs)), [1:length(xs)])\nend\n\nmillneighbors!(cache, g::DagGraph, model::DagModel, i::Int) = millneighbors!(cache, g, model, inneighbors(g.g, i))\nChainRulesCore.@non_differentiable inneighbors(g, i)\nnothing # hide","category":"page"},{"location":"examples/dag/","page":"DAGs","title":"DAGs","text":"Note that this recursive approach is not the most efficient way to implement this. It would be better to spent a little time with graphs to identify sets of vertices that can be processed in parallel and for which all ancestors are known. But this was a fun little exercise.","category":"page"},{"location":"examples/jsons/","page":"Processing JSONs","title":"Processing JSONs","text":"![\"JsonGrinder.jl](\"https://raw.githubusercontent.com/CTUAvastLab/JsonGrinder.jl/master/docs/src/assets/logo.svg\"){kind=logo}
\n ![\"JsonGrinder.jl](\"https://raw.githubusercontent.com/CTUAvastLab/JsonGrinder.jl/master/docs/src/assets/logo-dark.svg\"){kind=logo}
\n ![\"Bag](\"../../assets/bagmodel.svg\")
\n![\"Bag](\"../../assets/bagmodel-dark.svg\")
","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"Three instances of the BagNode are represented by red subtrees are first mapped with instance model im, aggregated (aggregation operator here is a concatenation of two different operators a_1 and a_2), and the results of aggregation are transformed with bag model bm.","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"ukn: Musk example\nAnother handy feature of Mill.jl models is that they are completely differentiable and therefore fit in the Flux.jl framework. Nodes for processing arrays and bags are sufficient to solve the classical Musk problem.","category":"page"},{"location":"manual/nodes/#[ProductNode](@ref)s-and-[ProductModel](@ref)s","page":"Nodes","title":"ProductNodes and ProductModels","text":"","category":"section"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"ProductNode can be thought of as a Cartesian Product or a Dictionary. It holds a Tuple or NamedTuple of nodes (not necessarily of the same type). For example, a ProductNode with the BagNode and the ArrayNode from above and two more nodes as children would look like this:","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"PN = tuple(\n ArrayNode(randn(Float32, 2, 3)),\n BagNode(ArrayNode(zeros(Float32, 4, 4)), [1:1, 2:2, 3:4]),\n BN,\n AN\n ) |> ProductNode","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"Analogically, the ProductModel contains a Tuple or NamedTuple of (sub)models processing each of its children (stored in ms field standing for models), as well as one more (sub)model m:","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"ms = tuple(\n ArrayModel(Dense(2, 2)),\n BagModel(ArrayModel(Dense(4, 6)), SegmentedMean(6), Dense(6, 5)),\n BM,\n AM);\nm = Dense(14, 9);\nPM = ProductModel(ms, m)","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"Again, since the library is based on the property that the output of each model is an array, the product model applies models from ms to appropriate children and vertically concatenates the output, which is then processed by model m. An example of model processing the above sample would be:","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"y = PM.m(vcat([PM.ms[i](PN.data[i]) for i in 1:4]...))","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"which is equivalent to:","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"PM(PN) == y","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"Application of this product model can be schematically visualized as follows:","category":"page"},{"location":"manual/nodes/","page":"Nodes","title":"Nodes","text":"![\"Product](\"../../assets/productmodel.svg\")
\n![\"Product](\"../../assets/productmodel-dark.svg\")
","category":"page"},{"location":"citation/#Citation","page":"Citation","title":"Citation","text":"","category":"section"},{"location":"citation/","page":"Citation","title":"Citation","text":"Kindly cite our work with the following entries if you find it interesting, please:","category":"page"},{"location":"citation/","page":"Citation","title":"Citation","text":"JsonGrinder.jl: automated differentiable neural architecture for embedding arbitrary JSON data\n@article{Mandlik2021,\n author = {Šimon Mandlík and Matěj Račinský and Viliam Lisý and Tomáš Pevný},\n title = {JsonGrinder.jl: automated differentiable neural architecture for embedding arbitrary JSON data},\n journal = {Journal of Machine Learning Research},\n year = {2022},\n volume = {23},\n number = {298},\n pages = {1--5},\n url = {http://jmlr.org/papers/v23/21-0174.html}\n}\nMalicious Internet Entity Detection Using Local Graph Inference (practical Mill.jl application)\n@article{Mandlik2024,\n author = {Mandlík, Šimon and Pevný, Tomáš and Šmídl, Václav and Bajer, Lukáš},\n journal = {IEEE Transactions on Information Forensics and Security},\n title = {Malicious Internet Entity Detection Using Local Graph Inference},\n year = {2024},\n volume = {19},\n pages = {3554-3566},\n doi = {10.1109/TIFS.2024.3360867}\n}\nthis implementation (fill in the used version)\n@software{Mill,\n author = {Tomas Pevny and Simon Mandlik},\n title = {Mill.jl framework: a flexible library for (hierarchical) multi-instance learning},\n url = {https://github.com/CTUAvastLab/Mill.jl},\n version = {...},\n}","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"using Mill, Flux","category":"page"},{"location":"manual/reflectin/#Model-reflection","page":"Model reflection","title":"Model reflection","text":"","category":"section"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Since constructions of large models can be a tedious and error-prone process, Mill.jl provides reflectinmodel function that helps to automate it. The simplest definition accepts only one argument, a sample ds, and returns a compatible model:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"ds = BagNode(ProductNode((BagNode(randn(Float32, 4, 10),\n [1:2, 3:4, 5:5, 6:7, 8:10]),\n randn(Float32, 3, 5),\n BagNode(BagNode(randn(Float32, 2, 30),\n [i:i+1 for i in 1:2:30]),\n [1:3, 4:6, 7:9, 10:12, 13:15]),\n randn(Float32, 2, 5))),\n [1:1, 2:3, 4:5]);\nprinttree(ds)\n\nm = reflectinmodel(ds, d -> Dense(d, 2));\nprinttree(m)\n\nm(ds)","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"The sample ds serves here as a specimen needed to specify a structure of the problem and calculate dimensions.","category":"page"},{"location":"manual/reflectin/#Optional-arguments","page":"Model reflection","title":"Optional arguments","text":"","category":"section"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"To have better control over the topology, reflectinmodel accepts up to two more optional arguments and four keyword arguments:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"The first optional argument expects a function that returns a layer (or a set of layers) given input dimension d (defaults to d -> Flux.Dense(d, 10)).\nThe second optional argument is a function returning aggregation function for BagModel nodes (defaults to BagCount ∘ SegmentedMeanMax).","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Compare the following example to the previous one:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"m = reflectinmodel(ds, d -> Dense(d, 5, relu), SegmentedMax);\nprinttree(m)\n\nm(ds)","category":"page"},{"location":"manual/reflectin/#Keyword-arguments","page":"Model reflection","title":"Keyword arguments","text":"","category":"section"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"The reflectinmodel allows even further customization. To index into the sample (or model), we can use printtree(ds; trav=true) from HierarchicalUtils.jl that prints the sample together with identifiers of individual nodes:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"using HierarchicalUtils","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"printtree(ds; trav=true)","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"These identifiers can be used to override the default construction functions. Note that the output, i.e. the last feed-forward network of the whole model is always tagged with an empty string \"\", which simplifies putting linear layer with an appropriate output dimension on the end. Dictionaries with these overrides can be passed in as keyword arguments:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"fsm overrides constructions of feed-forward models\nfsa overrides construction of aggregation functions.","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"For example to specify just the last feed forward neural network:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"reflectinmodel(ds, d -> Dense(d, 5, relu), SegmentedMeanMax;\n fsm = Dict(\"\" => d -> Chain(Dense(d, 20, relu), Dense(20, 12)))) |> printtree","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Both keyword arguments in action:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"reflectinmodel(ds, d -> Dense(d, 5, relu), SegmentedMeanMax;\n fsm = Dict(\"\" => d -> Chain(Dense(d, 20, relu), Dense(20, 12))),\n fsa = Dict(\"Y\" => SegmentedMean, \"g\" => SegmentedPNorm)) |> printtree","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"There are even more ways to modify the reflection behavior, see the reflectinmodel api reference.","category":"page"},{"location":"manual/reflectin/#Float-precision","page":"Model reflection","title":"Float precision","text":"","category":"section"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Mill.jl is built on top of Flux.jl, which by default uses 32-bit precision for model parameters:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Dense(2, 2).weight |> eltype","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"If you attempt to process Float64 data with model using lower precision, you get a warning:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"x = randn(2, 2)\neltype(x)\nm = Dense(2, 2)\nm(x)","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Unless additional arguments are provided, reflectinmodel also instantiates all Dense layers using 32-bit precision:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"x = randn(Float32, 2, 2) |> ArrayNode\neltype(Mill.data(x))\nm = reflectinmodel(x)\nm.m.weight |> eltype","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Because reflectinmodel evaluates (sub)models on parts of the input when building the model, if some Float64 values are passed in, the warning is shown during construction as well as during the evaluation:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"x = randn(2, 2) |> ArrayNode\neltype(Mill.data(x))\nm = reflectinmodel(x)\nm(x)","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"To prevent this from happening, we recommend making sure that the same precision is used for input data and for reflectinmodel parameters. For example:","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"x32 = randn(Float32, 2, 2) |> ArrayNode\nm = reflectinmodel(x32)\n\nx64 = randn(2, 2) |> ArrayNode\nm = reflectinmodel(x64, d -> f64(Dense(d, 5)), d -> f64(SegmentedMean(d)))","category":"page"},{"location":"manual/reflectin/","page":"Model reflection","title":"Model reflection","text":"Functions Flux.f64 and Flux.f32 may come in handy.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"using Mill, Flux","category":"page"},{"location":"manual/aggregation/#Bag-aggregation","page":"Bag aggregation","title":"Bag aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Aggregation operators in Mill.jl are all subtypes of AbstractAggregation. These structures are responsible for mapping of vector representations of multiple instances into a single vector. They all operate element-wise and independently of dimension and thus the output has the same size as representations on the input, unless the Concatenation of multiple operators is used.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Some setup:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"d = 2\nX = Float32.([1 2 3 4; 8 7 6 5])\nbags = AlignedBags([1:1, 2:3, 4:4])","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Different choice of operator, or their combinations, are suitable for different problems. Nevertheless, because the input is interpreted as an unordered bag of instances, every operator is invariant to permutation and also does not scale when increasing size of the bag.","category":"page"},{"location":"manual/aggregation/#Non-parametric-aggregation","page":"Bag aggregation","title":"Non-parametric aggregation","text":"","category":"section"},{"location":"manual/aggregation/#Max-aggregation","page":"Bag aggregation","title":"Max aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"SegmentedMax implements a simple max and is the most straightforward operator defined in one dimension as follows:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_max(x_1 ldots x_k) = max_i = 1 ldots k x_i","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"where x_1 ldots x_k are all instances of the given bag. In Mill.jl, the operator is constructed this way:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_max = SegmentedMax(d)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"The application is straightforward:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_max(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Since we have three bags, we have three columns in the output, each storing the maximal element over all instances of the given bag.","category":"page"},{"location":"manual/aggregation/#Mean-aggregation","page":"Bag aggregation","title":"Mean aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"SegmentedMean implements mean function, defined as:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamemean(x_1 ldots x_k) = frac1k sum_i = 1^k x_i","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"and used the same way:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_mean = SegmentedMean(d)\na_mean(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"ukn: Sufficiency of the mean operator\nIn theory, mean aggregation is sufficient for approximation as proven in [6], but in practice, a combination of multiple operators performes better.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"The max aggregation is suitable for cases when one instance in the bag may give evidence strong enough to predict the label. On the other side of the spectrum lies the mean aggregation function, which detects well trends identifiable globally over the whole bag.","category":"page"},{"location":"manual/aggregation/#Sum-aggregation","page":"Bag aggregation","title":"Sum aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"The last non-parametric operator is SegmentedSum, defined as:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamemean(x_1 ldots x_k) = sum_i = 1^k x_i","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"and used the same way:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_sum = SegmentedSum(d)\na_sum(X, bags)","category":"page"},{"location":"manual/aggregation/#Parametric-aggregation","page":"Bag aggregation","title":"Parametric aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Whereas non-parametric aggregations do not use any parameter, parametric aggregations represent an entire class of functions parametrized by one or more real vectors of parameters, which can be even learned during training.","category":"page"},{"location":"manual/aggregation/#Log-sum-exp-(LSE)-aggregation","page":"Bag aggregation","title":"Log-sum-exp (LSE) aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"SegmentedLSE (log-sum-exp) aggregation ([8]) is parametrized by a vector of positive numbers bmr in (mathbbR^+)^d m that specifies one real parameter for computation in each output dimension:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamelse(x_1 ldots x_k r) = frac1rlog left(frac1k sum_i = 1^k exp(rcdot x_i)right)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"With different values of r, LSE behaves differently and in fact both max and mean operators are limiting cases of LSE. If r is very small, the output approaches simple mean, and on the other hand, if r is a large number, LSE becomes a smooth approximation of the max function. Naively implementing the definition above may lead to numerical instabilities, however, the Mill.jl implementation is numerically stable.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_lse = SegmentedLSE(d)\na_lse(X, bags)","category":"page"},{"location":"manual/aggregation/#p-norm-aggregation","page":"Bag aggregation","title":"p-norm aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"(Normalized) p-norm operator ([9]) is parametrized by a vector of real numbers bmp in (mathbbR^+)^d, where forall i in 1 ldots m colon p_i geq 1, and another vector bmc in (mathbbR^+)^d. It is computed with formula:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamepnorm(x_1 ldots x_k p c) = left(frac1k sum_i = 1^k vert x_i - c vert ^ p right)^frac1p","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Again, the Mill.jl implementation is numerically stable.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_pnorm = SegmentedPNorm(d)\na_pnorm(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Because all parameter constraints are included implicitly (field ρ in both types is a real number that undergoes appropriate transformation before being used), both parametric operators are easy to use and do not require any special treatment. Replacing the definition of aggregation operators while constructing a model (either manually or with reflectinmodel) is enough.","category":"page"},{"location":"manual/aggregation/#Concatenation","page":"Bag aggregation","title":"Concatenation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"To use a concatenation of two or more operators, one can use the AggregationStack constructor:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a = AggregationStack(a_mean, a_max)\na(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"For the most common combinations, Mill.jl provides some convenience definitions:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"SegmentedMeanMax(d)\nSegmentedPNormLSE(d)","category":"page"},{"location":"manual/aggregation/#Weighted-aggregation","page":"Bag aggregation","title":"Weighted aggregation","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Sometimes, different instances in the bag are not equally important and contribute to output to a different extent. For instance, this may come in handy when performing importance sampling over very large bags. SegmentedMean and SegmentedPNorm have definitions taking weights into account:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamemean((x_i w_i)_i=1^k) = frac1sum_i=1^k w_i sum_i = 1^k w_i cdot x_i","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_operatornamepnorm(x_i w_i_i=1^k p c) = left(frac1sum_i=1^k w_i sum_i = 1^k w_icdotvert x_i - c vert ^ p right)^frac1p","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"This is done in Mill.jl by passing an additional parameter:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"w = Float32.([1.0, 0.2, 0.8, 0.5])\na_mean(X, bags, w)\na_pnorm(X, bags, w)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"For SegmentedMax and SegmentedLSE it is possible to pass in weights, but they are ignored during computation:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_max(X, bags, w) == a_max(X, bags)","category":"page"},{"location":"manual/aggregation/#Weighted-nodes","page":"Bag aggregation","title":"Weighted nodes","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"WeightedBagNode is used to store instance weights into a dataset. It accepts weights in the constructor:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"wbn = WeightedBagNode(X, bags, w)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"and passes them to aggregation operators:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"m = reflectinmodel(wbn, d -> Dense(d, 3))\nm(wbn)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Otherwise, WeightedBagNode behaves exactly like the standard BagNode.","category":"page"},{"location":"manual/aggregation/#Bag-count","page":"Bag aggregation","title":"Bag count","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"For some problems, it may be beneficial to use the size of the bag directly and feed it to subsequent layers. To do this, wrap an instance of AbstractAggregation or AggregationStack in the BagCount type.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"In the aggregation phase, bag count appends one more element which stores the bag size to the output after all operators are applied. Furthermore, Mill.jl, performs a mapping x mapsto log(x) + 1 on top of that:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"a_mean_bc = BagCount(a_mean)\na_mean_bc(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"The matrix now has three rows, the last one storing the size of the bag.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Model reflection adds BagCount after each aggregation operator by default.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"bn = BagNode(X, bags)\nbm = reflectinmodel(bn, d -> Dense(d, 3))","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"Note that the bm (sub)model field of the BagNode has size of (7, 3), 3 for each of two aggregation outputs and 1 for sizes of bags.","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"bm(bn)","category":"page"},{"location":"manual/aggregation/#Default-aggregation-values","page":"Bag aggregation","title":"Default aggregation values","text":"","category":"section"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"When all aggregation operators are printed, one may notice that all of them store one additional vector ψ. This is a vector of default parameters, initialized to all zeros, that are used for empty bags:","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"bags = AlignedBags([1:1, 0:-1, 2:3, 0:-1, 4:4])\na_mean(X, bags)","category":"page"},{"location":"manual/aggregation/","page":"Bag aggregation","title":"Bag aggregation","text":"That's why the dimension of input is required in the constructor. See Missing data page for more information.","category":"page"},{"location":"","page":"Home","title":"Home","text":"![\"Mill.jl](\"assets/logo.svg\"){kind=logo}
\n![\"Mill.jl](\"assets/logo-dark.svg\"){kind=logo}
","category":"page"},{"location":"","page":"Home","title":"Home","text":"Mill.jl (Multiple Instance Learning Library) is a library built on top of Flux.jl aimed to flexibly prototype hierarchical multiple instance learning models as described in [1], [2] and [3]. It is developed to be:","category":"page"},{"location":"","page":"Home","title":"Home","text":"flexible and versatile\nas general as possible\nfast \nand dependent on only handful of other packages","category":"page"},{"location":"","page":"Home","title":"Home","text":"Watch our introductory talk from JuliaCon 2021.","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Run the following in REPL:","category":"page"},{"location":"","page":"Home","title":"Home","text":"] add Mill","category":"page"},{"location":"","page":"Home","title":"Home","text":"Julia v1.9 or later is required.","category":"page"},{"location":"#Getting-started","page":"Home","title":"Getting started","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"For the quickest start, see the Musk example.","category":"page"},{"location":"","page":"Home","title":"Home","text":"Motivation: a brief introduction into the philosophy of Mill.jl\nManual: a brief tutorial into Mill.jl\nExamples: some examples of Mill.jl use\nExternal tools: examples of integration with other packages\nPublic API: extensive API reference\nReferences: related literature\nCitation: preferred citation entries","category":"page"},{"location":"api/switches/#General","page":"Switches","title":"General","text":"","category":"section"},{"location":"api/switches/#Index","page":"Switches","title":"Index","text":"","category":"section"},{"location":"api/switches/","page":"Switches","title":"Switches","text":"Pages = [\"switches.md\"]","category":"page"},{"location":"api/switches/#API","page":"Switches","title":"API","text":"","category":"section"},{"location":"api/switches/","page":"Switches","title":"Switches","text":"Mill.emptyismissing\nMill.emptyismissing!\n\nMill.string_start_code\nMill.string_start_code!\n\nMill.string_end_code\nMill.string_end_code!","category":"page"},{"location":"api/switches/#Mill.emptyismissing","page":"Switches","title":"Mill.emptyismissing","text":"Mill.emptyismissing()\n\nGet the current value of the emptyismissing parameter.\n\nSee also: Mill.emptyismissing!.\n\n\n\n\n\n","category":"function"},{"location":"api/switches/#Mill.emptyismissing!","page":"Switches","title":"Mill.emptyismissing!","text":"Mill.emptyismissing!(::Bool; persist=false)\n\nSet the new value to the emptyismissing parameter.\n\nSet persist=true to persist this setting between sessions.\n\nSee also: Mill.emptyismissing.\n\n\n\n\n\n","category":"function"},{"location":"api/switches/#Mill.string_start_code","page":"Switches","title":"Mill.string_start_code","text":"Mill.string_start_code()\n\nGet the current value of the string_start_code parameter used as a code point of the abstract string-start character. The default value of the parameter is 0x02, which corresponds to the STX character in ASCII encoding.\n\nSee also: Mill.string_start_code!, Mill.string_end_code, Mill.string_end_code!.\n\n\n\n\n\n","category":"function"},{"location":"api/switches/#Mill.string_start_code!","page":"Switches","title":"Mill.string_start_code!","text":"Mill.string_start_code!(c::Integer; persist=false)\n\nSet the new value to the string_start_code parameter used as a code point of the abstract string-start character to c. The default value of the parameter is 0x02, which corresponds to the STX character in ASCII encoding.\n\nc should fit into UInt8.\n\nSet persist=true to persist this setting between sessions.\n\nSee also: Mill.string_start_code, Mill.string_end_code, Mill.string_end_code!.\n\n\n\n\n\n","category":"function"},{"location":"api/switches/#Mill.string_end_code","page":"Switches","title":"Mill.string_end_code","text":"Mill.string_end_code()\n\nGet the current value of the string_end_code parameter used as a code point of the abstract string-end character. The default value of the parameter is 0x03, which corresponds to the ETX character in ASCII encoding.\n\nSee also: Mill.string_end_code!, Mill.string_start_code, Mill.string_start_code!.\n\n\n\n\n\n","category":"function"},{"location":"api/switches/#Mill.string_end_code!","page":"Switches","title":"Mill.string_end_code!","text":"Mill.string_end_code!(c::Integer; persist=false)\n\nSet the new value to the string_end_code parameter used as a code point of the abstract string-end character to c. The default value of the parameter is 0x03, which corresponds to the ETX character in ASCII encoding.\n\nc should fit into UInt8.\n\nSet persist=true to persist this setting between sessions.\n\nSee also: Mill.string_end_code, Mill.string_start_code, Mill.string_start_code!.\n\n\n\n\n\n","category":"function"}]
+}
diff --git a/v2.10.6/siteinfo.js b/v2.10.6/siteinfo.js
new file mode 100644
index 00000000..05ab3cb6
--- /dev/null
+++ b/v2.10.6/siteinfo.js
@@ -0,0 +1 @@
+var DOCUMENTER_CURRENT_VERSION = "v2.10.6";
diff --git a/v2.10.6/tools/hierarchical/index.html b/v2.10.6/tools/hierarchical/index.html
new file mode 100644
index 00000000..5b2e2cc8
--- /dev/null
+++ b/v2.10.6/tools/hierarchical/index.html
@@ -0,0 +1,78 @@
+
+