Base Phylogeny

PortfolioOptimisers.HopCount — Type
struct HopCount{__T_n} <: AbstractSeparationAlgorithm

Separation measured as the number of graph edges between two assets.

The separation between two assets is the number of edges on the shortest path between them. The weights of the edges have no effect. The budget is n edges. The power sum sum(A^i for i in 0:n) of phylogeny_matrix and the power sums of the two clusterise methods read this n.

The budget is a field and not an argument, because its unit is the hop, and only this member uses that unit. PathLength measures the same structure in the units of the distance estimator and holds its budget in those units. So a caller does not have to know the unit.

The power sum is the range connection matrix of the source

The matrix $\mathbf{P} = \mathbb{1}_{x \geq 1}\left(\sum_{i=0}^{n} \mathbf{A}^{i}\right) - \mathbf{I}$ of phylogeny_matrix is the range connection matrix $\mathbf{B}_{1,n}$ of the source, with one indicator in place of n indicators. The source builds a connection matrix for each walk length, $\mathbf{B}_{k} = \mathbb{1}_{x \geq 1}(\mathbf{A}^{k} + \mathbf{I}) - \mathbf{I}$, and then applies the indicator to their sum.

The library adds the powers first. The sum then has the term $\mathbf{A}^{0} = \mathbf{I}$, and the last $- \mathbf{I}$ removes it. The two forms are equal for every graph and every n. Off the diagonal, each one is 1 when a walk of length 1 to n joins the pair. On the diagonal, each one is 0.

The original, [117], prints the indicator as $\mathbb{1}_{x > 1}$. With that indicator, a pair that one walk joins is not in $\mathbf{B}_{k}$, and Figure 9 of the same source shows that it is. Equations 13.1 and 13.2 of [12] print $\mathbb{1}_{x \geq 1}$, which gives that figure. The library builds this form.

The budget can be a rule

n is a CountOf of hops, or a FractionOf of the reachable pairs that the budget relates. A caller who cannot state the budget in advance gives the fraction. This applies when the universe is a cross-validation fold or a subproblem of a meta optimiser. HopCount(; n = FractionOf(0.25)) asks for the hop budget that relates about a quarter of the reachable pairs. resolve_separation measures the hop separations over the structure that the consumer built, takes their 0.25 quantile over the reachable pairs with separation_quantile, and rounds it to the nearest Int. A plain Integer is a count, and any other plain real number is a fraction, through count_or_fraction.

A stated count keeps the number of hops constant, and the number of related pairs changes with the graph. A fraction keeps the number of related pairs near the fraction, and the number of hops changes. The strength of a downstream constraint follows the number of related pairs. A hop count is an Integer and a quantile is not, so the share of related pairs is near the fraction but not equal to it. On a small graph the shells are coarse, and the share can be far from it. The smallest off-diagonal hop separation is 1, so the rounded quantile is never below one.

n also accepts a HopCountAlgorithm or a bare Function. resolve_separation calls it as n(nte, X, g; dims = dims, kwargs...), where g is the structure that the consumer built.

After resolution, n is a CountOf, and its val is an Integer. Three readers use 0:n.val as a count of matrix powers, and 0:1.5 drops a power with no error. So resolve_separation checks the return value of a rule.

Fields

  • n: Separation budget in hops. A pair whose shortest path has more than n.val edges is unrelated. A CountOf states the hops, and the library uses it as it is. A FractionOf states the share of the reachable pairs to relate, and resolve_separation turns it into a count of hops. A plain number is read through count_or_fraction. A HopCountAlgorithm or a Function is a rule. resolve_separation calls it as n(nte, X, g; dims = dims, kwargs...), where g is the structure that the consumer built, and the rule must return an Integer.

Constructors

HopCount(;    n::HopCountValue = CountOf(1)) -> HopCount

Keywords correspond to the struct's fields. The constructor reads a plain number through count_or_fraction.

Validation

  • A plain n passes count_or_fraction. If n is a CountOf, n.val <= RESOURCE_LIMITS[].max_hop_count. Three readers sum A^i over i in 0:n.val, so the compute cost is linear in n.val. See RESOURCE_LIMITS. resolve_separation resolves a fraction or a rule, and the constructor does not check the value that it gives.
A search grid can turn a plain count into a fraction

A field that takes a CountOf or a FractionOf also takes a plain number, and count_or_fraction reads an Integer as a count and any other real number as a fraction. Julia promotes the elements of a vector literal to one type before the library reads them. So ["opt.ret.ucs.alg.method.active" => [3, 5, 8], "r.alpha" => [0.05, 0.1]] is a Vector{Pair{String, Vector{Float64}}}, and the counts arrive as 3.0, 5.0 and 8.0. The field then reads them as fractions, and the check of FractionOf refuses them. State the grid in the wrappers, such as [CountOf(3), CountOf(5), FractionOf(0.2)]. A wrapper keeps its type in every container.

Examples

julia> HopCount()HopCount  n ┼ CountOf    │   val ┴ Int64: 1julia> HopCount(; n = FractionOf(0.25))HopCount  n ┼ FractionOf    │   val ┴ Float64: 0.25

Related

References

  • [117] D. Cajas. A Graph Theory Approach to Portfolio Optimization. Available at SSRN 4602019 (2023). Section 2.2, Equations 1-2.
  • [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 13.1.2, Equations 13.1-13.2, for the indicator $\mathbb{1}_{x \geq 1}$.
source
PortfolioOptimisers.PathLength — Type
struct PathLength{__T_dmax} <: AbstractSeparationAlgorithm

Separation measured as the length of the shortest weighted path between two assets.

The separation between two assets is the sum of the distances along the shortest path that joins them in the network. The budget dmax has the same units. It is the graded counterpart of HopCount. Both measure how far apart two assets are in the same structure. HopCount counts the edges, and PathLength adds their lengths.

No published source states this separation. The library adds it. HopCount has a source, the range connection matrix of a walk length. That literature counts edges only, and states no budget in the units of a distance estimator.

The path uses distances on both branches

The path uses the distance matrix on the edges of the structure alone, through calc_distance_weighted_graph. This is true for each branch that builds the structure. On the tree branch, the weights of the graph are these distances. On the PMFG branch, the similarity selects the structure. The distance then weights its edges, and the similarity is a strictly decreasing function of that distance.

A path over the similarities of the PMFG is wrong, not a second convention. A shortest path over similarities has the smallest total similarity, so it goes through the weakest links. The error is hard to see. Over the four similarity algorithms, the wrong answer has a Pearson correlation of 0.95 to 0.97 with the correct one.

It can replace a hop count, but its values are not comparable

PathLength and HopCount satisfy the same contract. Proximity and phylogeny_matrix read a separation through that contract, so each one accepts the two members. Their outputs are not comparable as values. The budgets have different units and the supports are different. Under LinearDecay, the scales are also different.

The two clusterise methods refuse PathLength at dispatch. They do not read the separation through the contract. They use sep.n as the count of matrix powers, and a radius has no equivalent of that count.

On real data the two separations agree more than this suggests, because the distance selects both structures. Over twenty assets, the Pearson correlation of the two separation matrices is 0.99 on a minimum spanning tree and 0.97 to 0.98 on a PMFG. On the tree 0.16% of the pairs of pairs are in the opposite order, and on the PMFG none are. This agreement is a measurement, not a guarantee. It comes from the graphs that an AbstractDistanceEstimator usually gives, not from a property of the two separations.

The budget

The default dmax = nothing means the whole connected component. The library uses the observed diameter, which is the largest finite entry of the separation matrix. A caller cannot easily choose a summed path length in the units of an AbstractDistanceEstimator, such as dmax = 0.37. The default reads the whole component and lets the decay make the fall-off. A caller who needs the same radius in each fold states a number.

separation_budget clamps a stated dmax to the observed diameter. No pair is further apart than the diameter, so the clamp removes no pair. It changes only the top of the scale of LinearDecay, which is the one decay that reads the budget. With no clamp, a dmax far above the diameter makes the scores of LinearDecay almost constant and excludes no pair.

phylogeny_matrix uses the budget to select pairs, not to shape a fall-off. There the default relates every reachable pair. So the phylogeny matrix of NetworkEstimator(; sep = PathLength()) has ones at each entry off the diagonal. This is the opposite of the default n = 1 of HopCount, which gives the smallest neighbourhood. State a numeric dmax to select fewer pairs.

The budget can be a rule

dmax also accepts a FractionOf of the reachable pairs. A fraction helps a caller who cannot state a number. PathLength(; dmax = FractionOf(0.25)) gives the radius that relates a quarter of the reachable pairs. resolve_separation measures the path separations over the structure that the consumer built, and takes their 0.25 quantile over the reachable pairs with separation_quantile, with no rounding. A caller can choose that fraction, and it means the same in each fold of a cross-validation and in each subproblem of a meta optimiser. A plain number is a radius, so the field takes no plain shorthand for a fraction, and a CountOf is no radius.

A fixed dmax and a fraction keep different quantities constant. A fixed dmax keeps the radius constant, and the number of related pairs changes with the graph. A fraction keeps the number constant, and the radius changes. Neither keeps both constant across folds, because each fold fits a new graph. For SemiDefinitePhylogeny and IntegerPhylogeny, the strength of the constraint is the number of related pairs.

A hop budget moves through a small number of shell cardinalities, and it cannot stop between them. A fraction of the path length is continuous, so FractionOf(0.3) asks for a cardinality, and the result is close to it. This is the one real gain of the radius ball over a hop count.

dmax also accepts a PathLengthAlgorithm or a bare Function. resolve_separation calls it as dmax(nte, X, g; dims = dims, kwargs...).

Fields

  • dmax: Separation budget, in the units of the separation. nothing means the observed diameter of the structure. A FractionOf states the share of the reachable pairs to relate, and resolve_separation turns it into a radius. A plain number is a radius, not a fraction. A PathLengthAlgorithm or a Function is a rule. resolve_separation calls it as dmax(nte, X, g; dims = dims, kwargs...), where g is the structure that the consumer built, and the rule must return a Number.

Constructors

PathLength(;    dmax::Option{PathLengthValue} = nothing) -> PathLength

Keywords correspond to the struct's fields.

Validation

  • If dmax is a Number, dmax > 0. resolve_separation resolves a fraction or a rule, and the constructor does not check the value that it gives.

Examples

julia> PathLength()PathLength  dmax ┴ nothingjulia> PathLength(; dmax = 0.5)PathLength  dmax ┴ Float64: 0.5julia> PathLength(; dmax = FractionOf(0.3))PathLength  dmax ┼ FractionOf       │   val ┴ Float64: 0.3

Related

source
PortfolioOptimisers.LinearDecay — Type
struct LinearDecay <: AbstractSeparationDecayAlgorithm

Separation decay that falls linearly to the edge of the budget.

Mathematical definition

\[\begin{align} f(d) &= d_{\mathrm{max}} + 1 - d\,, \end{align}\]

Where:

  • $d$: Separation between two assets.
  • $d_{\mathrm{max}}$: Separation budget in scope.

This is the default member, and the only member that reads the budget. A direct neighbour scores $d_{\mathrm{max}}$, and the asset itself scores $d_{\mathrm{max}} + 1$.

The budget truncates, and the decay does not, so the decay needs no max(0, ⋅) floor. On 0 <= d <= dmax the score is strictly positive, and its smallest value is 1.

Examples

julia> separation_decay.(Ref(LinearDecay()), 0:3, 3)4-element Vector{Int64}: 4 3 2 1

Related

source
PortfolioOptimisers.ExponentialDecay — Type
struct ExponentialDecay{__T_rate} <: AbstractSeparationDecayAlgorithm

Separation decay that falls exponentially.

Mathematical definition

\[\begin{align} f(d) &= e^{-\lambda d}\,, \end{align}\]

Where:

  • $d$: Separation between two assets.
  • $\lambda$: Rate of the fall-off, rate.

The score of the asset itself is f(0) = 1. rate sets the contrast between an asset and its neighbour, and the budget has no effect on it. A large rate makes the score fall fast. The budget states only how far apart a related pair can be.

The parameter is a rate, not a retention per step. $\rho^d$ and $e^{-\lambda d}$ are the same function when $\lambda = -\log\rho$. A retention per step applies to integer steps, but the separation is a real number. ExponentialSimilarity and ExpGerberIQDecay also use a rate. The rate needs rate > 0 for a monotone decay, and a finite rate, because Inf * 0 is NaN at the separation 0. A large finite rate gives each neighbour a score near 0, and the budget sets which neighbours the consumer reads.

Fields

  • rate: Rate of the exponential fall-off, exp(-rate * d). A larger rate makes the score fall faster. The retention form ratio^d is the same function at rate = -log(ratio).

Constructors

ExponentialDecay(;    rate::Number = 1.0) -> ExponentialDecay

Keywords correspond to the struct's fields.

Validation

  • isfinite(rate) && rate > 0.

Examples

julia> ExponentialDecay()ExponentialDecay  rate ┴ Float64: 1.0julia> separation_decay.(Ref(ExponentialDecay()), 0:3, 3)4-element Vector{Float64}: 1.0 0.36787944117144233 0.1353352832366127 0.049787068367863944

Related

source
PortfolioOptimisers.ReciprocalDecay — Type
struct ReciprocalDecay{__T_power} <: AbstractSeparationDecayAlgorithm

Separation decay that falls as a power of the separation.

Mathematical definition

\[\begin{align} f(d) &= \left(1 + d\right)^{-p}\,, \end{align}\]

Where:

  • $d$: Separation between two assets.
  • $p$: Exponent of the fall-off, power.

Its tail is heavier than the tail of ExponentialDecay, so a distant asset keeps a small score.

The 1 + keeps the classical inverse-distance weight finite at $d = 0$. It also sets f(0) = 1, which is the top of the scale of ExponentialDecay.

The library does not use the form $(1 + d^p)^{-1}$. That form gives $f(1) = 1/2$ for every p. A larger p raises the score of a pair nearer than $d = 1$, and lowers the score of a pair further away. From p = 1 to p = 3, the score at $d = 0.5$ goes from 0.6667 to 0.8889, and the score at $d = 2$ goes from 0.3333 to 0.1111. So in that form, p cannot make the decay steeper below $d = 1$. In the form above, a larger p lowers the score at every $d > 0$.

Fields

  • power: Exponent of the reciprocal fall-off, inv((1 + d)^power). A larger power makes the score fall faster.

Constructors

ReciprocalDecay(;    power::Number = 1.0) -> ReciprocalDecay

Keywords correspond to the struct's fields.

Validation

  • power > 0.

Examples

julia> ReciprocalDecay()ReciprocalDecay  power ┴ Float64: 1.0julia> separation_decay.(Ref(ReciprocalDecay()), 0:3, 3)4-element Vector{Float64}: 1.0 0.5 0.3333333333333333 0.25

Related

source
PortfolioOptimisers.factory — Method
factory(
    alg::AbstractPhylogenyAlgorithm,
    args...;
    kwargs...
) -> AbstractClustersAlgorithm

Return the phylogeny algorithm alg unchanged.

A caller that rebuilds an estimator with factory reaches this method for a phylogeny algorithm that has no nearer method. A phylogeny algorithm has no field that depends on a prior, so the method keeps every field and replaces none.

Algorithm

  1. Return alg itself. The method rebuilds no field of it, and it discards args and kwargs.

Arguments

  • alg: Phylogeny algorithm.
  • args...: Optional arguments. The method does not read them.
  • kwargs...: Optional keyword arguments. The method does not read them.

Returns

  • alg::AbstractPhylogenyAlgorithm: The original phylogeny algorithm.

Related

source
PortfolioOptimisers.separation_decay — Function
separation_decay(dk::LinearDecay, d::Number, dmax::Number)
separation_decay(dk::ExponentialDecay, d::Number, dmax::Number)
separation_decay(dk::ReciprocalDecay, d::Number, dmax::Number)
separation_decay(::Nothing, d::Number, dmax::Number)

Score a separation under a decay algorithm.

This function is the extension contract of AbstractSeparationDecayAlgorithm. A new member is a struct and one method of this function.

The method the call selects

This function selects a method and runs no step of its own. It dispatches on dk, and the selected method evaluates one closed form. The # Mathematical definition of each member states its form, and this docstring does not repeat it.

dkThe fall-off it selectsReads dmax
LinearDecayLinear to the edge of the budgetyes
ExponentialDecayExponential in rateno
ReciprocalDecayA power of 1 + d, set by powerno
nothingFlat, $f(d) = 1$no

LinearDecay is the only member that reads the budget. So the other three methods leave their third argument unnamed.

No decay

nothing does not fall off at all, and it is the only decay that is not strictly decreasing. The contract asks for a monotone non-increasing function, and a constant is one. A pair outside the budget still scores 0, because the budget truncates and the decay does not. So Proximity with decay = nothing gives the indicator of the neighbourhood that the budget selects, not a matrix of ones. Under HopCount it is the indicator of the n-hop neighbourhood, and under PathLength it is the indicator of the ball of radius dmax.

Arguments

  • dk: Separation decay algorithm, or nothing for no decay.
  • d: Separation between two assets, d >= 0. It is a real number, so it can be a weighted path length or a hop count.
  • dmax: Separation budget in scope. Only LinearDecay reads it, and the other members ignore it.

Validation

This function does not check the contract, because it runs inside an assets × assets loop. A caller probes the decay once before the loop with assert_separation_decay.

Returns

  • f::Number: Score for the separation. It is non-negative for 0 <= d <= dmax. Above the budget the contract does not set the sign, and LinearDecay is negative there. The consumer tests the budget first, so it does not call this function there.

Examples

julia> separation_decay(LinearDecay(), 2, 3)2julia> separation_decay(ExponentialDecay(; rate = 2.0), 2, 3)0.01831563888873418julia> separation_decay(ReciprocalDecay(; power = 2.0), 2, 3)0.1111111111111111julia> separation_decay(nothing, 2, 3)1julia> separation_decay.(nothing, 0:3, 3)4-element Vector{Int64}: 1 1 1 1

Related

source

References

[12]
D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[117]
D. Cajas. A Graph Theory Approach to Portfolio Optimization. Available at SSRN 4602019 (2023).