GitHub

Clusters

PortfolioOptimisers.Clusters — Type
struct Clusters{__T_res, __T_S, __T_D, __T_P, __T_k} <: AbstractClusteringResult

Carries a clustering of the asset universe together with the matrices it was computed from.

Holds the result of either family, because res takes both shapes clusterise produces: a Clustering.Hclust from a hierarchical algorithm and a Clustering.ClusteringResult from a non-hierarchical one. k is the number of clusters selected by an AbstractOptimalNumberClustersEstimator, and assignments is what reads a cluster label per asset off the pair — cutting the tree at k on the hierarchical branch, reading the field on the other.

D is what the clustering saw, unless P is present

S and D are the similarity and distance matrices of the universe, and an ordinary ClustersEstimator clusters D itself and leaves P as nothing. A NetworkClustersEstimator instead accumulates a pseudo-distance matrix out of the network structure and clusters that, so it fills P and res is the clustering of P. Both matrices are kept because a consumer wants the network's clustering and the universe's own distances.

Fields

  • res: Clustering result.
  • S: Similarity matrix.
  • D: Distance matrix.
  • P: Pseudo-distance matrix the clustering was run on, nothing when the clustering ran on D itself. A NetworkClustersEstimator builds it by accumulating the network structure out of the distance or similarity matrix; see clusterise.
  • k: Optimal number of clusters.

Constructors

Clusters(;    res::ClTypes,    S::MatNum,    D::MatNum,    P::Option{<:MatNum} = nothing,    k::Integer) -> Clusters

Keywords correspond to the struct's fields.

Validation

  • !isempty(S).
  • !isempty(D).
  • size(S) == size(D).
  • If P is not nothing, !isempty(P) and size(S) == size(P).
  • k >= 1.

Related

source
PortfolioOptimisers.clusterise — Method
clusterise(cle::AbstractClusteringResult, args...; kwargs...)

Return the clustering result cle unchanged.

Identity pass-through, so that every consumer takes an estimator or a precomputed result through one call. A ClustersEstimator reaching this function has already been run; a result reaching it is not run again.

Arguments

  • cle: Clustering result.
  • args...: Additional positional arguments, ignored.
  • kwargs...: Additional keyword arguments, ignored.

Returns

  • cle::AbstractClusteringResult: The input, unchanged.

Related

source
PortfolioOptimisers.SecondOrderDifference — Type
struct SecondOrderDifference{__T_alg} <: AbstractOptimalNumberClustersAlgorithm

Picks the number of clusters at which the within-cluster dispersion curve bends most sharply.

The two-difference gap statistic. It reads the second-order difference of the within-cluster dispersion across cluster counts, so it finds the elbow of that curve without the Monte Carlo simulation the original gap statistic needs.

Mathematical definition

\[\begin{align} c^{\star} &= \underset{c}{\arg\max} \left(W_{c+2} + W_{c} - 2 W_{c+1}\right)\,,\\ &\mathrm{s.t.} \quad 1 \leq c \leq \sqrt{N}\,,\\ W_{c} &= \sum_{j=1}^{c} g\left(\left\{d_{uv} : u, v \in \mathcal{C}_{j},\, u < v\right\}\right)\,. \end{align}\]

Where:

  • $c^{\star}$: Selected number of clusters.
  • $W_{c}$: Within-cluster dispersion of a cut into $c$ clusters.
  • $\mathcal{C}_{j}$: Set of assets in the $j$-th cluster of that cut.
  • $d_{uv}$: Entry of the distance matrix the clustering ran on.
  • $g$: Vector-to-scalar measure alg, applied to one cluster's pairwise distances. A cluster of one asset has no pairwise distance and contributes $0$.
  • $N$: Number of assets.

The selected $c$ is the left end of the triple, not its centre. That is the source's own statement of the problem, and the code maximises it as written.

The measure decides what "dispersion" means, and the default is not the source's

The source takes $W_{c}$ as the mean of a cluster's pairwise distances, which is alg = MeanValue(). The default alg = StandardisedValue() divides that mean by the corrected standard deviation of the same distances, which is a different statistic and selects a different $c$: on a 400x40 sample the two answer 6 and 4.

Note

A cluster of exactly two assets carries one pairwise distance, and a single value has no corrected standard deviation. StandardisedValue divides by $1$ in that case, so such a cluster contributes its mean pairwise distance to $W_{c}$. The gap series stays finite, and the selected $c$ is a real maximiser. The two measures still differ on every larger cluster.

Fields

  • alg: The measure used to evaluate clustering quality.

Constructors

SecondOrderDifference(;    alg::Num_VecToScaM = StandardisedValue()) -> SecondOrderDifference

Keywords correspond to the struct's fields.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

  • alg: Recursively updated via factory.

Examples

julia> SecondOrderDifference()SecondOrderDifference  alg ┼ StandardisedValue      │   mv ┼ MeanValue      │      │   w ┴ nothing      │   sv ┼ StdValue      │      │           w ┼ nothing      │      │   corrected ┴ Bool: true

Related

References

  • [4] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 12.2.1.2, Equation 12.15.
  • [47] S. Yue, X. Wang and M. Wei. Application of two-order difference to gap statistic. Transactions of Tianjin University 14, 217–221 (2008).
  • [48] R. Tibshirani, G. Walther and T. Hastie. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63, 411–423 (2001).
source
PortfolioOptimisers.SilhouetteScore — Type
struct SilhouetteScore{__T_alg} <: AbstractOptimalNumberClustersAlgorithm

Picks the number of clusters whose assets sit best inside their own cluster.

Each asset gets a silhouette, which compares how far it sits from its own cluster against how far it sits from the nearest other one. alg reduces the whole vector of them to one number per cluster count, and the count with the largest number wins.

Mathematical definition

\[\begin{align} s_{i} &= \frac{b_{i} - a_{i}}{\max\left(a_{i},\, b_{i}\right)}\,,\\ a_{i} &= \sum_{j \in \mathcal{C}_{I},\, j \neq i} \frac{d_{ij}}{\left|\mathcal{C}_{I}\right| - 1}\,,\\ b_{i} &= \underset{J \neq I}{\min} \sum_{j \in \mathcal{C}_{J}} \frac{d_{ij}}{\left|\mathcal{C}_{J}\right|}\,,\\ c^{\star} &= \underset{c}{\arg\max} \; g\left(\boldsymbol{s}\right)\,, \quad 1 \leq c \leq \sqrt{N}\,. \end{align}\]

Where:

  • $s_{i}$: Silhouette of asset $i$, in $\left[-1,\, 1\right]$. It is $1$ when the asset is clustered well and $-1$ when it is clustered wrongly.
  • $a_{i}$: Mean distance from asset $i$ to the other members of its own cluster $\mathcal{C}_{I}$.
  • $b_{i}$: Smallest mean distance from asset $i$ to the members of any other cluster.
  • $d_{ij}$: Entry of the distance matrix the clustering ran on.
  • $\boldsymbol{s}$: Vector of silhouettes over all assets, one per asset.
  • $g$: Vector-to-scalar measure alg.
  • $c^{\star}$: Selected number of clusters.
  • $N$: Number of assets.

The default reduction is the source's standardised score

alg = StandardisedValue() divides the mean of $\boldsymbol{s}$ by its corrected standard deviation, which is the source's quality measure term for term. The standardisation is what makes scores comparable across cluster counts, so it is the reduction the selection wants; the field is a knob only because a caller may want the plain mean instead. The two disagree in general, and the standardised form is the one this default computes.

Unlike SecondOrderDifference, the reduction here runs over one vector of length $N$, never over one cluster at a time, so no cluster size makes it undefined.

Fields

  • alg: The measure used to evaluate clustering quality.

Constructors

SilhouetteScore(;    alg::Num_VecToScaM = StandardisedValue()) -> SilhouetteScore

Keywords correspond to the struct's fields.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

  • alg: Recursively updated via factory.

Examples

julia> SilhouetteScore()SilhouetteScore  alg ┼ StandardisedValue      │   mv ┼ MeanValue      │      │   w ┴ nothing      │   sv ┼ StdValue      │      │           w ┼ nothing      │      │   corrected ┴ Bool: true

Related

References

  • [4] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 12.2.1.1, Equations 12.13-12.14.
  • [49] P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, 53–65 (1987).
  • [50] M. López de Prado and M. J. Lewis. Detection of false investment strategies using unsupervised learning methods. Quantitative Finance 19, 1555–1565 (2019).
source
PortfolioOptimisers.OptimalNumberClusters — Type
struct OptimalNumberClusters{__T_max_k, __T_alg} <: AbstractOptimalNumberClustersEstimator

Decides how many clusters to cut a dendrogram or a partition into.

Pairs a selection rule with a ceiling on the answer. alg is either an AbstractOptimalNumberClustersAlgorithm, which chooses the count from the data, or an Integer, which states it outright.

A stated k is a request, not a guarantee

max_k caps every branch, and the cap is itself capped: the ceiling in force is min(floor(Int, sqrt(N)), max_k), so a max_k above $\sqrt{N}$ buys nothing. A stated k above that ceiling is lowered to it.

A stated k is also checked against the tree on the hierarchical branch. A cut into k clusters that no node of the dendrogram supports is not honoured; optimal_number_clusters searches upward and downward for the nearest k that is, and takes the nearer of the two. The data-driven branches apply the same test through valid_k_clusters, which walks the scoring array from its largest entry down until one passes.

Fields

  • max_k: Maximum number of clusters to consider. If nothing, computed as the floor(Int, sqrt(assets)).
  • alg: Algorithm for selecting the optimal number of clusters. If an integer, defines the number of clusters directly.

Constructors

OptimalNumberClusters(;    max_k::Option{<:Integer} = nothing,    alg::Int_ONC = SecondOrderDifference()) -> OptimalNumberClusters

Keywords correspond to the struct's fields.

Validation

  • If max_k is not nothing, max_k >= 1.
  • If alg is an Integer, alg >= 1.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

  • alg: Recursively updated via factory.

Examples

julia> OptimalNumberClusters(; max_k = 10)OptimalNumberClusters  max_k ┼ Int64: 10    alg ┼ SecondOrderDifference        │   alg ┼ StandardisedValue        │       │   mv ┼ MeanValue        │       │      │   w ┴ nothing        │       │   sv ┼ StdValue        │       │      │           w ┼ nothing        │       │      │   corrected ┴ Bool: true

Related

References

  • [4] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 12.2.1.
source
PortfolioOptimisers.HClustAlgorithm — Type
struct HClustAlgorithm{__T_linkage} <: AbstractHierarchicalClusteringAlgorithm

Builds a dendrogram by merging the two nearest clusters until one remains.

linkage is the criterion that says how near two clusters are once they hold more than one asset. :single takes the smallest distance between their members and :complete the largest; :average takes the mean over all cross pairs; :ward merges the pair that raises the total within-cluster variance least. The symbol is passed straight to Clustering.hclust, so every criterion that package accepts is available.

Fields

  • linkage: Linkage method for hierarchical clustering from Clustering.jl.

Constructors

HClustAlgorithm(;    linkage::Symbol = :ward) -> HClustAlgorithm

Keywords correspond to the struct's fields.

Examples

julia> HClustAlgorithm(; linkage = :average)HClustAlgorithm  linkage ┴ Symbol: :average

Related

References

  • [4] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 12.1.1, Equations 12.6-12.12.
  • [51] D. Müllner. Modern hierarchical, agglomerative clustering algorithms. arXiv preprint arXiv:1109.2378 (2011).
source
PortfolioOptimisers.ClustersEstimator — Type
struct ClustersEstimator{__T_ce, __T_de, __T_alg, __T_onc} <: AbstractClustersEstimator

Turns a return matrix into a clustering of the asset universe.

Holds the four steps in the order they run: ce estimates the covariance, de turns it into a distance matrix, alg clusters that matrix, and onc decides how many clusters to keep. clusterise runs them and returns a Clusters.

The universe is clustered, not the observations

Every step reads an assets x assets matrix, so the number of observations leaves the picture at ce. That is why alg and onc carry no @fprop tag: an ObsWeights weights observations, and there is no observation left for it to weight once de has run. ce and de are tagged, and they are where a weight belongs.

Fields

  • ce: Covariance estimator.
  • de: Distance matrix estimator.
  • alg: Clustering algorithm.
  • onc: Optimal number of clusters estimator.

Constructors

ClustersEstimator(;    ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),    de::AbstractDistanceEstimator = Distance(; alg = CanonicalDistance()),    alg::AbstractClustersAlgorithm = HClustAlgorithm(),    onc::AbstractOptimalNumberClustersEstimator = OptimalNumberClusters()) -> ClustersEstimator

Keywords correspond to the struct's fields.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

  • ce: Recursively updated via factory.
  • de: Recursively updated via factory.

Examples

julia> ClustersEstimator()ClustersEstimator   ce ┼ PortfolioOptimisersCovariance      │   ce ┼ Covariance      │      │    me ┼ SimpleExpectedReturns      │      │       │   w ┴ nothing      │      │    ce ┼ GeneralCovariance      │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)      │      │       │    w ┴ nothing      │      │   alg ┴ FullMoment()      │   mp ┼ MatrixProcessing      │      │     pdm ┼ Posdef      │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton      │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()      │      │      dn ┼ nothing      │      │      dt ┼ nothing      │      │     alg ┼ nothing      │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)   de ┼ Distance      │   power ┼ nothing      │     alg ┴ CanonicalDistance()  alg ┼ HClustAlgorithm      │   linkage ┴ Symbol: :ward  onc ┼ OptimalNumberClusters      │   max_k ┼ nothing      │     alg ┼ SecondOrderDifference      │         │   alg ┼ StandardisedValue      │         │       │   mv ┼ MeanValue      │         │       │      │   w ┴ nothing      │         │       │   sv ┼ StdValue      │         │       │      │           w ┼ nothing      │         │       │      │   corrected ┴ Bool: true

Related

References

  • [4] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 12.1.1.
source
PortfolioOptimisers.ClTypes — Type
const ClTypes = Union{<:Clustering.ClusteringResult, <:Clustering.Hclust}

Alias for clustering result types from the Clustering.jl package.

Matches either a Clustering.ClusteringResult or Clustering.Hclust. Used internally to accept output from the Clustering.jl library for both flat and hierarchical clustering results.

Related

source

References

[4]
D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[47]
S. Yue, X. Wang and M. Wei. Application of two-order difference to gap statistic. Transactions of Tianjin University 14, 217–221 (2008).
[48]
R. Tibshirani, G. Walther and T. Hastie. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63, 411–423 (2001).
[49]
P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, 53–65 (1987).
[50]
M. López de Prado and M. J. Lewis. Detection of false investment strategies using unsupervised learning methods. Quantitative Finance 19, 1555–1565 (2019).
[51]
D. Müllner. Modern hierarchical, agglomerative clustering algorithms, arXiv preprint arXiv:1109.2378 (2011).