GitHub

Linear Constraints: private API

PortfolioOptimisers.AbstractParsingResult — Type
abstract type AbstractParsingResult <: AbstractConstraintResult

Abstract supertype for all equation parsing result types.

All concrete and/or abstract types representing parsing results should be subtypes of AbstractParsingResult. Every member carries one parsed equation in canonical form — the variable names, their coefficients, the comparison operator and the right-hand side — so that the stages after parse_equation read one shape whatever the equation was written in.

Related

source
PortfolioOptimisers.allowed_functions — Constant
allowed_functions = Dict{Symbol, Function}(:+ => +, :- => -, :* => *, :/ => /,
                                           :^ => ^, :sqrt => sqrt, :cbrt => cbrt,
                                           :exp => exp, :exp2 => exp2, :exp10 => exp10,
                                           :log => log, :log2 => log2, :log10 => log10,
                                           :abs => abs, :min => min, :max => max)

Enumerated table of the functions permitted in equation parsing, mapping each allowed name directly to its function object. Evaluating constraint/view strings crosses a trust boundary (config files, spreadsheets, UI), so the parser must be able to call only these 16 mathematical functions. Using an explicit Symbol => Function table — rather than resolving a name against Base with getfield(Base, fname) — bounds that capability to exactly this table: a name absent from the keys fails closed with a Meta.ParseError, and the set of callable functions cannot drift from the set of allowed names, because they are the same list. See docs/adr/0025-enumerated-parser-allowlist.md.

The prior(...) marker is deliberately absent from this table: it names assets/groups (not numbers) and is expanded structurally by eval_numeric_functions/replace_group_by_assets, never evaluated numerically.

Related

source
PortfolioOptimisers.merge_partial_linear_constraints — Function
merge_partial_linear_constraints(
    ps
) -> Union{Nothing, PartialLinearConstraint{var"#s185", <:AbstractVector{var"#s137"}} where {var"#s137"<:(Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}), var"#s185"<:AbstractMatrix{var"#s137"}, var"#s137"<:(Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar})}}

Concatenate the rows of the same half of several PartialLinearConstraints, skipping the absent ones.

Algorithm

  1. Collect the entries of ps that are not nothing, giving kept.
  2. Return nothing when kept is empty, because the half is absent from every input.
  3. Read the row width of the first entry of kept, giving N, and check every other entry against it.
  4. Stack the A matrices of kept in input order, and stack their B vectors the same way.
  5. Return the PartialLinearConstraint built from the two stacks.

Arguments

Validation

  • Every kept half is written over the same number of variables, size(p.A, 2) == N. A DimensionMismatch is thrown otherwise.

Returns

Related

source
PortfolioOptimisers.merge_linear_constraints — Function
merge_linear_constraints(
    lcs::AbstractVector{<:LinearConstraint}
) -> LinearConstraint

Combine several LinearConstraints into the single one that holds all their rows.

A LinearConstraint is a block of rows, and applying two blocks is the same as applying the block that stacks them — the inequality halves concatenate, the equality halves concatenate, and an absent half contributes nothing. This is exactly what generation already does when it is handed several estimators at once: centrality_constraints over a vector of CentralityConstraints appends every row into one result rather than returning one result per estimator.

That equivalence is what this function exists to preserve. A caller that computes its constraints separately — a Pipeline running one step per estimator — can merge them here and reach the optimiser with the value it would have had from the vector form.

Algorithm

  1. Return the one element unchanged when lcs holds a single constraint.
  2. Merge the ineq half of every element with merge_partial_linear_constraints, giving the inequality half of the result.
  3. Merge the eq half of every element the same way, giving the equality half.
  4. Return the LinearConstraint built from the two halves.

Arguments

  • lcs: The constraints to merge.

Validation

  • lcs is non-empty.
  • Every merged half is written over the same number of variables.

Returns

  • lc::LinearConstraint: One constraint carrying every row, in input order.

Examples

julia> lc1 = LinearConstraint(; ineq = PartialLinearConstraint(; A = [1.0 0.0], B = [0.5]));julia> lc2 = LinearConstraint(; ineq = PartialLinearConstraint(; A = [0.0 1.0], B = [0.25]));julia> PortfolioOptimisers.merge_linear_constraints([lc1, lc2])LinearConstraint  ineq ┼ PartialLinearConstraint       │   A ┼ 2×2 Matrix{Float64}       │   B ┴ Vector{Float64}: [0.5, 0.25]    eq ┴ nothing

Related

source
PortfolioOptimisers.get_linear_constraints — Function
get_linear_constraints(lcs::PR_VecPR, sets::UniverseSets,
                       key::Option{<:AbstractString} = nothing;
                       datatype::DataType = Float64, strict::Bool = false,
                       rr::Option{<:AbstractLoadingsRegressionResult} = nothing)

Convert parsed linear constraint equations into a LinearConstraint object.

get_linear_constraints takes one or more ParsingResult objects (as produced by parse_equation), expands variable names using the provided UniverseSets, and assembles the corresponding constraint matrices and right-hand side vectors. The result is a LinearConstraint object containing both equality and inequality constraints, suitable for use in portfolio optimisation routines.

A row takes one of two shapes. Without rr it runs over the universe the names resolve against. With rr it runs over the assets, because the loadings re-base each term as the row is assembled and what leaves the function is an ordinary asset-space row.

A row is the unit of a drop. A row is a joint statement over several names with one right-hand side, so a name this function cannot resolve takes the whole row with it rather than only its own term: a + c == 0.05 assembled without c would fit a == 0.05, a different and stronger claim than the caller wrote. What the name's failure was decides only whether the drop is reported. A name on the counterpart axis — read with counterpart_axis_names, and in practice the Non-Investable Axis a door minted — is dropped in silence under both settings of strict, because it was a correct name over the universe the caller was handed and the data moved it; the departure is announced once, by the door. A name on neither axis is a typo, and is reported exactly as before.

Algorithm

  1. Take k as key, or sets.xkey when key is nothing, read the universe nx from sets.dict under it, name the axis with universe_axis, and read the counterpart axis with counterpart_axis_names.
  2. Take N, the row length, from constraint_row_length, and allocate the working row At of that length.
  3. Zero At for each parsing result, and start that result not dropped.
  4. Build the indicator of each variable name of the result over nx. A name that matches no entry marks the row dropped, and is reported through strict_diagnostic unless it names the counterpart axis. Every name of the row is still visited, so a row carrying two typos names both.
  5. Add the contribution constraint_row_term gives for the name and its coefficient to At. With rr the contribution arrives already projected, so At is asset-length while it is accumulated.
  6. Move to the next result when the row was marked dropped.
  7. Report the row through strict_diagnostic and drop it when At is still zero. Every name resolved to get here, so the message says the row was annihilated — by the loadings under rr, by its own cancelling coefficients otherwise, or by there being no name in it at all — and never that a name was mistyped.
  8. Read the sign and the inequality flag of the operator from comparison_sign_ineq_flag, and scale the row and its right-hand side by the sign. That negates a >= row, so both senses of an inequality are written in the <= sense, which is the convention LinearConstraint states.
  9. Append the row to the inequality accumulator when the flag is true, and to the equality accumulator when it is false.
  10. Reshape each accumulator that holds a row into a matrix of N columns, and build the PartialLinearConstraint of that half.
  11. Return the LinearConstraint holding the halves that were built, or nothing when neither half holds a row.

Arguments

  • lcs: A single ParsingResult or a vector of such objects, representing parsed constraint equations.
  • sets: A UniverseSets object specifying the universes and groupings.
  • key: Key naming the universe the variables resolve against. Defaults to sets.xkey; a re-based constraint passes sets.tfkey.
  • datatype: Numeric type for coefficients and right-hand side.
  • strict: If true, throws an error if a variable or group is not found in sets; if false, issues a warning.
  • rr: Loadings to re-base through, or nothing for an ordinary asset-space constraint. See ExposureConstraintEstimator — callers do not pass this directly.
  • ledger: The door's ledger of departure casualties, or nothing when nobody is collecting. A row dropped for a name on the counterpart axis is recorded into it through record_non_investable_drop!.

Validation

  • lcs is non-empty, when it is a vector.
  • A variable name that matches no entry of the universe and none of the counterpart axis raises when strict is true, and issues a warning otherwise. The row is dropped either way.
  • A variable name on the counterpart axis drops its row in silence, under both settings of strict.
  • A row whose terms all fall away raises when strict is true, and issues a warning otherwise. The row is dropped either way.
  • Each op is one of "==", "<=" or ">=", which comparison_sign_ineq_flag enforces.

Returns

  • lcs::LinearConstraint: An object containing the assembled equality and inequality constraints, or nothing if no constraints are present.

Related

source
PortfolioOptimisers.prefixed_sets_keys — Function
prefixed_sets_keys(
    dict::AbstractDict,
    prefix::AbstractString
) -> Vector{String}

Collect the dict keys that start with prefix, as the candidate pool of a suggest_declared_key suggestion inside UniverseSets.

A missing partition key is reported by the group that asked for it, so the whole key set is the wrong pool: the nearest neighbour of nx_sector in Dict("ux_sector" => …) is ux_sector, the very key under validation, and the caller would be told to rename the one thing that is correct. Narrowing the pool to the prefix the missing key must carry leaves only keys that could genuinely have been meant.

Algorithm

  1. Return the keys of dict that start with prefix, as strings, in the order dict iterates in.

Arguments

  • dict: The UniverseSets dictionary being validated.
  • prefix: The axis prefix the missing key must carry, xkey, tfkey or cfkey.

Returns

  • candidates::Vector{String}: The keys of dict that start with prefix.

Related

source
PortfolioOptimisers.unclaimed_sets_keys — Function
unclaimed_sets_keys(
    dict::AbstractDict,
    claimed
) -> Vector{String}

Collect the dict keys that no axis in claimed has taken, as the candidate pool of the missing-xkey suggestion inside UniverseSets.

The counterpart of prefixed_sets_keys for the one key with no prefix of its own. The asset universe is whichever key holds the asset names, so it cannot be found by a prefix; what can be ruled out is every key another declared axis already speaks for. Without that, a dict carrying only a feature axis answers a mistyped xkey with the feature key, which is a different axis and never the right fix.

Algorithm

  1. Return the keys of dict that start with no entry of claimed, as strings, in the order dict iterates in.

Arguments

  • dict: The UniverseSets dictionary being validated.
  • claimed: The other declared axis prefixes, uxkey, tfkey, utfkey, cfkey and ucfkey.

Returns

  • candidates::Vector{String}: The keys of dict that start with no entry of claimed.

Related

source
PortfolioOptimisers.assert_factor_partition — Function
assert_factor_partition(dict::AbstractDict, k::AbstractString, fkey::AbstractString,
                        axis::AbstractString) -> Nothing

Assert that the factor partition k names a declared factor axis fkey, and that the two agree on how many factors there are.

UniverseSets carries two factor axes, and both obey this one rule, so the rule is written once and called twice. axis names the axis in both messages — a caller who declared the time-series axis and wrote a cross-sectional partition is told which of the two is missing, which the key value alone does not say.

Arguments

  • dict: The UniverseSets dictionary being validated.
  • k: The fkey-prefixed key under validation.
  • fkey: The factor axis key the prefix belongs to, tfkey or cfkey.
  • axis: Names the axis in both diagnostic messages, for example "time-series factor".

Validation

  • haskey(dict, fkey). A KeyError naming axis is thrown otherwise.
  • length(dict[k]) == length(dict[fkey]). A DimensionMismatch naming axis is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_factor_unique_group — Function
assert_factor_unique_group(dict::AbstractDict, k::AbstractString, fkey::AbstractString,
                           ufkey::AbstractString, axis::AbstractString) -> Nothing

Assert that the unique-entry factor group k names a declared factor axis fkey, that the partition it draws its entries from exists, and that the partition has the length of the axis.

The sibling of assert_factor_partition, and written for the same reason: UniverseSets carries two factor axes and both obey this one rule, so the rule is written once and called twice.

Arguments

  • dict: The UniverseSets dictionary being validated.
  • k: The ufkey-prefixed key under validation.
  • fkey: The factor axis key the group summarises, tfkey or cfkey.
  • ufkey: The unique-entry prefix k carries, utfkey or ucfkey.
  • axis: Names the axis in every diagnostic message, for example "cross-sectional factor".

Validation

  • haskey(dict, fkey). A KeyError naming axis is thrown otherwise.
  • haskey(dict, fkey * chopprefix(k, ufkey)). A KeyError carrying a spelling suggestion is thrown otherwise.
  • length(dict[fkey * chopprefix(k, ufkey)]) == length(dict[fkey]). A DimensionMismatch is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.universe_axis — Function
universe_axis(sets::UniverseSets, key::AbstractString) -> String

Name of the axis the universe stored under key belongs to, read off the key itself: "factor" for anything carrying the tfkey prefix, "asset" otherwise. It exists only so unknown_variable_msg and empty_row_msg can name the axis the user wrote in.

The key is the evidence, for both callers, and the reason is that both resolve names against sets.dict[key] and nothing else: whatever axis that universe belongs to is the axis a failed lookup failed on. get_black_litterman_views takes the key from the estimator that owns the views, and get_linear_constraints from the constraint space — FactorSpace resolving at sets.tfkey. Reading it off the re-basis instead would be a second encoding of the same fact, and a worse one: a wrapped estimator carrying its own key overrides the space's, so a re-based row can legitimately resolve against a universe the loadings are not written in, and the message must name the universe that was searched.

The prefix rather than equality is what makes a factor group key ("nf_sector") resolve as the factor axis too, and the disjoint-prefix rule UniverseSets enforces at construction is what makes that unambiguous.

Algorithm

  1. Return "factor" when key starts with sets.tfkey.
  2. Return "asset" in every other case.

Arguments

  • sets: The UniverseSets whose tfkey names the factor axis.
  • key: The key the names were resolved against.

Returns

  • axis::String: "factor" or "asset", the word a diagnostic message uses to name the axis.

Related

source
PortfolioOptimisers.constraint_row_length — Function
constraint_row_length(rr, nx::VecStr) -> Int

Length of the assembled constraint row. Without a re-basis this is the size of the universe the names resolve against; with one it is the number of assets the loadings project onto, because the projection is applied while the row is assembled and what leaves is an ordinary asset-space row.

Algorithm

The method that Julia selects is the algorithm, and the re-basis selects it.

  1. rr is nothing: return the length of nx, the universe the names resolve against.
  2. rr is a regression result: return the number of rows of rr.M, which is the number of assets the loadings project onto.

Arguments

  • rr: Loadings to re-base through, or nothing for an ordinary asset-space row.
  • nx: The universe the names resolve against.

Returns

  • N::Int: The number of entries one assembled row has.

Related

source
PortfolioOptimisers.constraint_row_term — Function
constraint_row_term(::Nothing, Ai, c)
constraint_row_term(rr::AbstractLoadingsRegressionResult, Ai, c)

Contribution of one matched variable to a constraint row.

Without a re-basis the contribution is the indicator Ai scaled by the coefficient c. With one it is the columns of the loadings that Ai selects, summed and scaled. The columns are summed rather than indexed by findfirst, so a factor universe carrying a duplicated name contributes every column bearing it, matching how the asset path treats a duplicated asset name.

rr.M is used, never rr.L: M's columns are the named original factors, and a constraint must be written in names a user can put in an equation. Risk decomposition wants L and is right to.

Mathematical definition

\[\begin{align} \boldsymbol{a}^\intercal \boldsymbol{w}_f &= \boldsymbol{a}^\intercal \mathbf{M}^\intercal \boldsymbol{w}_a = (\mathbf{M} \boldsymbol{a})^\intercal \boldsymbol{w}_a\,. \end{align}\]

Where:

  • $\boldsymbol{a}$: A constraint row written in factor names.
  • $\boldsymbol{w}_f$: The factor weights that row is written against.
  • $\boldsymbol{w}_a$: The asset weights the optimiser holds.
  • $\mathbf{M}$: The factor loadings, one column per named factor and one row per asset.

The identity is what lets a row written in factor names bind asset weights with no change of variables: the re-based row is an ordinary asset-space row over $\boldsymbol{w}_a$.

Algorithm

The method that Julia selects is the algorithm, and the re-basis selects it.

  1. rr is nothing: return Ai scaled by c, one entry per name of the universe.
  2. rr is a regression result: sum the columns of rr.M that Ai selects, scale the sum by c, and return it. The value is asset-length whatever the row was written in.

Arguments

  • rr: Loadings to re-base through, or nothing for an ordinary asset-space row.
  • Ai: The indicator of the matched name over the universe the names resolve against.
  • c: The coefficient the matched name carries.

Returns

Related

source
PortfolioOptimisers.name_to_val! — Function
name_to_val!(nx::VecStr, sdict::AbstractDict, key::Any, val::Number,
             arr::VecNum, strict::Bool, nxkey::AbstractString,
             other::VecStr = String[])

Set values in a vector for the asset or the group of assets that key names.

name_to_val! resolves key through resolve_axis_name — an asset name resolves to itself, a group name expands to its members — maps the result to indices in the asset universe nx, and sets the corresponding entries of arr to val. If key names neither, the function either throws an error or issues a warning, depending on the strict flag. Every diagnostic message names the size of the universe and never the universe itself or the input value dictionary, because each is routed through a shared message builder in 01_Base/06_Messages.jl.

other is the counterpart axis: the asset names that this call is not resolving against, but that the same UniverseSets declares. A name found there is skipped in silence, under strict or not, and that is the whole of what strict gives up. strict exists to catch a caller's typo, and a name on the counterpart axis is the opposite of a typo: it was a correct name over the universe the caller was given, and the data moved it. A caller cannot know in advance which asset a prior will fail to estimate, so refusing them — or even warning, once per constraint, per window of a walk-forward — reports something no one can act on. The departure itself is announced once, by the door that derived the mask.

The two asset axes are counterparts of each other, and the relation is symmetric. Resolving on the asset universe, other is the Non-Investable Axis, so a bound stated for an asset that left is dropped. Resolving on the Non-Investable Axis — which is how a forced-liquidation rate is priced — other is the asset universe, so a liquidation rate stated for an asset that stayed is dropped by the same rule. A factor axis has no counterpart, and other is then empty.

Algorithm

  1. Resolve key through resolve_axis_name, giving members. An asset name resolves to itself, and a group name expands to a copy of its member list. An asset name takes precedence over a group name of the same spelling.
  2. Return in silence when members is nothing and key names an entry of other, because the name is on the counterpart axis: it is known-good, and this axis has no entry to write it into.
  3. Report through strict_diagnostic and return when members is nothing, because key names neither an asset nor a group. The suggestion pool is widened from nx to nx together with the keys of sdict, because a missing name may be a mistyped asset or a mistyped group.
  4. Map members to positions in nx with axis_name_indices, giving idx. Members that miss the universe are dropped. Those on other are struck from the report by the same rule as step 2, and any that remain are reported once through strict_diagnostic — so a group whose departed members are all accounted for is silent, and one holding a genuine typo still names it.
  5. Set the entries of arr at idx to val.

Arguments

  • nx: Vector of asset names.
  • sdict: Dictionary mapping group names to vectors of asset names. It is never modified, because resolve_axis_name returns a copy of the member list.
  • key: Name of the asset or the group of assets to set values for.
  • val: The value to assign.
  • arr: The array to be modified in-place.
  • strict: If true, throws an error if key resolves to nothing; if false, issues a warning.
  • nxkey: Name of the asset-universe key in sets.dict (e.g. "nx"), used only to name the universe in the diagnostic message — see unknown_variable_msg / missing_group_assets_msg.
  • other: The counterpart asset axis, whose names are skipped in silence rather than reported.

Validation

  • key names an asset of nx, a group of sdict, or an entry of other. An ArgumentError is thrown when strict is true, and a warning is issued otherwise.
  • Every member of a resolved group names an entry of nx or of other. A member that misses both is dropped, and the drop raises when strict is true and issues a warning otherwise.

Returns

  • nothing. The operation is performed in-place on arr.

Related

source
PortfolioOptimisers._parse_equation — Function
_parse_equation(lhs, opstr::AbstractString, rhs; datatype::DataType = Float64)

Parse and canonicalise a linear constraint equation from Julia expressions.

_parse_equation takes the left-hand side (lhs) and right-hand side (rhs) of a constraint equation, both as Julia expressions, and a comparison operator string (opstr). It evaluates numeric functions, moves all terms to the left-hand side, collects coefficients and variables, and returns a ParsingResult with the canonicalised equation.

Algorithm

  1. Fold the constant subexpressions of both sides with eval_numeric_functions, giving lexpr and rexpr, and check each with rethrow_parse_error.
  2. Build diff_expr, the expression lexpr - (rexpr). This moves every term of the equation to the left-hand side.
  3. Walk diff_expr with _collect_terms, giving terms, one (coefficient, variable) pair per term.
  4. Accumulate terms into varmap, which holds the summed coefficient of each variable name, and into constant, the sum of the coefficients that carry no variable.
  5. Read variables and coefficients off varmap, and take rhs_val as the negated constant. This moves the constant to the right-hand side.
  6. Render each pair with format_term, join the renderings with +, and fold + - into -, giving the canonical string formatted.
  7. Return the ParsingResult built from variables, coefficients, opstr, rhs_val and formatted.

Arguments

  • lhs: Left-hand side of the equation as a Julia expression.
  • opstr: Comparison operator as a string.
  • rhs: Right-hand side of the equation as a Julia expression.
  • datatype: Numeric type for coefficients and right-hand side.

Returns

  • res::ParsingResult: Structured result with canonicalised variables, coefficients, operator, right-hand side, and formatted equation. The order of vars is the order the variable map iterates in, and it is not the order the equation was written in.

Related

source
PortfolioOptimisers.rethrow_parse_error — Function
rethrow_parse_error(expr; side = :lhs)

Internal utility for error handling during equation parsing.

rethrow_parse_error is used to detect and handle incomplete or invalid expressions encountered while parsing constraint equations. It is called on both sides of an equation during parsing to ensure that the expressions are valid and complete. If an incomplete expression is detected, a Meta.ParseError is thrown; otherwise, the function returns nothing. The parser fails closed on an empty side rather than assuming zero, because a silently assumed zero is a constraint the author never wrote. A caller who means zero writes it.

Algorithm

The method that Julia selects is the algorithm, and one method answers each shape a parsed side can take.

  1. expr is Nothing, which is what an empty side gives: raise, and name side in the message.
  2. expr is an Expr: raise when its head is :incomplete, and return nothing otherwise.
  3. expr is anything else, a number or a symbol among them: return nothing.

Arguments

  • expr: The parsed Julia expression to check. Can be an Expr, Nothing, or any other type.
  • side: Symbol indicating which side of the equation is being checked (:lhs or :rhs). Used for error messages.

Validation

  • expr is not Nothing. A Meta.ParseError naming side is thrown otherwise.
  • expr.head != :incomplete. A Meta.ParseError naming side and the expression is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.format_term — Function
format_term(coeff, var)

Format a single term in a linear constraint equation as a string.

format_term takes a coefficient and a variable name and returns a string representation suitable for display in a canonicalised linear constraint equation. Handles special cases for coefficients of 1 and -1 to avoid redundant notation.

Algorithm

  1. Return the variable name alone when coeff is one.
  2. Return the variable name behind a minus sign when coeff is minus one.
  3. Return the coefficient, a *, and the variable name, in every other case.

Arguments

  • coeff: Numeric coefficient for the variable.
  • var: Variable name as a string.

Returns

  • term_str::String: The formatted term as a string.

Related

source
PortfolioOptimisers.collect_terms! — Function
collect_terms!(expr, coeff, terms)

Recursively collect and expand terms from a Julia expression for linear constraint parsing.

collect_terms! traverses a Julia expression tree representing a linear equation, expanding and collecting all terms into a vector of (coefficient, variable) pairs. It handles numeric constants, variables, and arithmetic operations (+, -, *, /), supporting canonicalisation of linear constraint equations for further processing.

Algorithm

  1. Append (coeff * expr, nothing) when expr is a Number, so a constant carries the coefficient and no variable.
  2. Append (coeff, string(expr)) when expr is a Symbol, so a bare variable carries the coefficient it arrived with.
  3. For a multiplication a * b, recurse into the side that is not a number, with coeff multiplied by the side that is. A product of two non-numeric sides is opaque, so append it whole as (coeff, string(expr)).
  4. For a division a / b, recurse into a with coeff divided by b, when b is a number. A division by a denominator that is not a number is opaque, so append it whole.
  5. For an addition, recurse into every argument with coeff unchanged.
  6. For a subtraction, recurse into every argument but the last with coeff, and into the last with -coeff. A unary minus holds no argument but the last, so this negates its one operand.
  7. Append any other expression whole, as (coeff, string(expr)). This is what makes a term such as sqrt(x) opaque: it becomes one variable named by its own text, and the row builder resolves that text against the universe like any other name.

Arguments

  • expr: The Julia expression to traverse.
  • coeff: The current numeric coefficient to apply.
  • terms: A vector to which (coefficient, variable) pairs are appended in-place. Each pair is of the form (typeof(coeff), Option{<:String}), where Nothing indicates a constant term.

Returns

  • nothing. The function modifies terms in-place.

Related

source
PortfolioOptimisers._collect_terms — Function
_collect_terms(expr::Union{Symbol, Expr, <:Number}, datatype::DataType = Float64)

Expand and collect all terms from a Julia expression representing a linear constraint equation.

_collect_terms takes a Julia expression (such as the left-hand side of a constraint equation), recursively traverses its structure, and returns a vector of (coefficient, variable) pairs. It supports numeric constants, variables, and arithmetic operations (+, -, *, /), and is used to canonicalise linear constraint equations for further processing.

The starting coefficient is one(datatype), so every coefficient the walk builds is of that type. The caller asked for the numeric domain the optimiser works in, and the coefficients belong to it as much as the right-hand side does.

Algorithm

  1. Open an empty vector terms.
  2. Walk expr with collect_terms!, from the starting coefficient one(datatype). The walk appends one pair to terms per term it reaches: a constant as (coefficient, nothing), and anything else as (coefficient, name).
  3. Return terms.

Arguments

  • expr: The Julia expression to expand.
  • datatype: Numeric type of the coefficients the walk builds.

Returns

  • terms::Vector{Tuple{datatype, Option{<:String}}}: A vector of (coefficient, variable) pairs, where variable is a string for variable terms or nothing for constant terms.

Related

source
PortfolioOptimisers.eval_numeric_functions — Function
eval_numeric_functions(expr, datatype::DataType = Float64)

Recursively evaluate numeric functions and constants in a Julia expression.

eval_numeric_functions traverses a Julia expression tree and evaluates any sub-expressions that are purely numeric, including standard mathematical functions and constants (such as Inf). This is used to simplify constraint equations before further parsing and canonicalisation.

When an allowlisted function is actually evaluated (all its arguments are numeric), its arguments are coerced to datatype (a float type) first, so the arithmetic happens in the same numeric domain the optimiser will use rather than in machine Int64. This prevents integer literals from combining and wrapping — e.g. 2^64 yields 1.8446744073709552e19 rather than silently wrapping to 0, and 2^-1 yields 0.5 rather than a DomainError. Numeric literals that survive inside an unevaluated (nonlinear) subexpression are left untouched, so 2^z still renders as 2 ^ z.

Only the functions enumerated in allowed_functions may be evaluated; any other call head fails closed with a Meta.ParseError. The prior(...) marker is handled structurally (see replace_group_by_assets) and throws a Meta.ParseError if given purely numeric arguments.

Algorithm

  1. Fold every argument of expr first, by applying this function to each, when expr is an Expr. A node whose head is not :call is rebuilt from its folded arguments and returned.
  2. Rebuild a :call node whose head is prior from its folded arguments, and return it. The marker names assets or groups, so it is never folded to a number.
  3. Look the head of any other :call node up in allowed_functions, giving f. A head that the table does not hold raises.
  4. Rebuild the call and return it when any folded argument is not a Number, so a nonlinear subexpression keeps its own literals untouched.
  5. Coerce every folded argument to datatype, apply f to them, and return the value that comes back.
  6. Return the value Inf when expr is the symbol :Inf. Return expr itself in every other case, so a number stands and a variable name survives as a symbol.

Arguments

  • expr: The Julia expression to evaluate. Can be a Number, Symbol, or Expr.
  • datatype: Float type into which numeric arguments are coerced before an allowlisted function is evaluated.

Validation

  • The head of a :call node is a key of allowed_functions, or the prior marker. A Meta.ParseError naming the head is thrown otherwise.
  • prior(...) carries at least one argument that is not a number. A Meta.ParseError is thrown otherwise.

Returns

  • The evaluated expression, with all numeric sub-expressions replaced by their computed values. Non-numeric or symbolic expressions are returned in their original or partially simplified form.

Related

source
PortfolioOptimisers.has_invalid_plus — Function
has_invalid_plus(expr)

Check whether a Julia expression contains an invalid + operator in a constraint context.

Internal helper used during linear constraint parsing to detect unsupported + operator usage in constraint expressions. It is the Expr counterpart of the ++ check the string form of parse_equation runs on the raw text.

Algorithm

  1. Return false when expr is not a call, because only a call can carry the head this function refuses.
  2. Return true when the head of the call is the ++ operator.
  3. Apply this function to every argument of the call that is itself an expression, and return true when any of them does.

Arguments

  • expr: Julia expression to check.

Returns

  • Bool: true if the expression contains an invalid +, false otherwise.

Related

source
PortfolioOptimisers.factor_universe — Function
factor_universe(sets::UniverseSets, key::AbstractString, K::Integer,
                need::AbstractString, source::AbstractString) -> VecStr

Read the declared factor universe sets.dict[key], checking that it exists and that it agrees with source — the observations × factors matrix whose K columns it must name — on how many factors there are.

A factor axis is optional on UniverseSets but is not optional for a consumer written against it, so the failure has to be diagnosed at the point of need. Both messages name key and the matrix, because the two are what a caller has to reconcile: a user arriving from the pre-declaration shape put the factor names under xkey and would otherwise be told about an asset universe they never wrote in.

key is stated rather than read off sets, because UniverseSets declares two factor axes and this helper cannot tell which one a caller means. A caller that holds a loadings result reads the key from it with factor_axis_key; a caller written against the returns data's own F states sets.tfkey, the axis those columns live on.

One helper therefore serves every consumer of either axis, and none of them re-encodes the checks.

Arguments

  • sets: The UniverseSets whose factor axis is read.
  • key: The factor axis key to read, sets.tfkey or sets.cfkey.
  • K: The number of columns of source, which the declared axis must name.
  • need: Names the consumer in both diagnostic messages, for example "a FactorSpace constraint".
  • source: Names the matrix in both diagnostic messages, for example "rr.M" or "F".

Validation

  • haskey(sets.dict, key). A KeyError naming need is thrown otherwise.
  • length(sets.dict[key]) == K. A DimensionMismatch naming source is thrown otherwise.

Returns

  • nf::VecStr: The declared factor names, in the column order of source.

Related

source
PortfolioOptimisers.factor_axis_key — Function
factor_axis_key(sets::UniverseSets, rr::Regression) -> AbstractString
factor_axis_key(sets::UniverseSets, rr::CrossSectionalFactorModel) -> AbstractString
factor_axis_key(sets::UniverseSets,
                re::AbstractTimeSeriesRegressionEstimator) -> AbstractString

Return the UniverseSets key naming the factor axis that rr's loadings are written on.

UniverseSets declares two factor axes, so a consumer that resolves factor names has to say which one it means. It never says so by hand. The key follows the block that carries M: a Regression is fitted per asset over the observations, so its columns are the columns of rd.F and it answers sets.tfkey; a CrossSectionalFactorModel is fitted per observation across the assets, so its columns are the exposures the fit was built from and it answers sets.cfkey. A caller therefore cannot name the wrong axis, and no consumer gains a field to state it in.

The third method serves a consumer that holds an unfitted specification rather than a result. Only the time-series family names a specification here, because RegE_Reg admits an AbstractTimeSeriesRegressionEstimator and no other estimator, and every result that family fits is a Regression. The three methods therefore cover RegE_Reg exactly.

There is deliberately no fallback on AbstractLoadingsRegressionResult. A future member of the root would silently inherit whichever axis the fallback named, and half the time that is the wrong list of names with the right length — a constraint written against it would still solve and would constrain the wrong factors. A missing method is a MethodError that names the type.

Arguments

  • sets: The UniverseSets whose factor axis key is read.
  • rr / re: The loadings result, or the specification whose verb produces one.

Returns

  • key::AbstractString: sets.tfkey for the time-series family, sets.cfkey for the cross-sectional one.

Related

source
PortfolioOptimisers._expr_depth_exceeds — Function
_expr_depth_exceeds(x, limit::Integer) -> Bool

Return true if the expression tree x is deeper than limit.

Guards the Expr form of parse_equation against a deeply nested AST that no string length cap covers. The check itself recurses at most limit + 1 frames deep and short-circuits the moment the limit is breached, so it cannot exhaust the stack it protects. docs/adr/0027-cap-equation-parser-recursion.md owns the limit this function is called with.

Algorithm

  1. Return true when limit is negative, because the walk has already gone one level past the cap.
  2. Return false when x is not an expression, because a leaf adds no depth.
  3. Apply this function to every argument of x, with limit lowered by one, and return true when any of them does. The scan stops at the first argument that answers true.

Arguments

  • x: The expression tree to measure.
  • limit: The greatest depth the tree may have.

Returns

  • Bool: true when the tree is deeper than limit, false otherwise.

Related

source
PortfolioOptimisers.assert_investable_constraint_width — Function
assert_investable_constraint_width(lcs::Nothing, N::Integer, slot::AbstractString)
assert_investable_constraint_width(lc::LinearConstraint, N::Integer,
                                   slot::AbstractString)
assert_investable_constraint_width(lcs::VecLc, N::Integer, slot::AbstractString)

Refuse a precomputed LinearConstraint whose rows are wider than the investable universe, and say why.

A name-keyed estimator survives a reduction to the Investable Mask: it resolves against the UniverseSets the door hands it, and a name that left resolves on the Non-Investable Axis instead of being refused. A precomputed constraint cannot. Its A is a matrix, and position is the only link between a column and an asset, so there is no name to re-resolve and no honest way to narrow it — dropping a column silently changes what Ax ≤ B means, and port_opt_view(::LinearConstraint, i) is deliberately the identity for that reason.

So the row survives the door at its original width and meets a shorter weight vector. Left alone that surfaces inside the model as a bare DimensionMismatch between two numbers, with nothing to connect either to the asset that delisted. This says it once, at the seam, in terms of what the caller did and what they can do instead.

The repair is always the same: state the constraint as a LinearConstraintEstimator. A name-keyed constraint is re-resolved over whatever universe the door leaves, which is the whole point of stating it by name.

Algorithm

  1. Return when there is nothing to check: a nothing slot, or a nothing half of a LinearConstraint.
  2. Otherwise compare size(A, 2) of each half against N and throw a DimensionMismatch naming the slot, the two widths and the repair when they disagree.

Arguments

  • lcs: The resolved constraint, a vector of them, or nothing.
  • N: The number of investable assets the optimisation runs over.
  • slot: Names the field the constraint came from in the message, for example "lcse".

Validation

  • Every half of every precomputed constraint has one column per investable asset.

Returns

  • nothing.

Related

source
PortfolioOptimisers.non_investable_sets — Function
non_investable_sets(sets::Nothing, ni) -> Nothing
non_investable_sets(sets::UniverseSets, ni::VecStr) -> UniverseSets

Mint the Non-Investable Axis on sets: declare ni, the names the Investable Mask left out, under sets.nikey.

This is the only way the axis comes to exist. A door calls it after it has reduced an optimiser to the Investable Mask, so a UniverseSets that carries the axis was reduced by exactly one door, for exactly one problem, and port_opt_view drops it rather than pass it to a sub-problem that did not earn it.

A caller may still declare the axis by hand, and outside a door that is the only way to resolve a forced-liquidation rate — fees_constraints called directly, with no optimisation around it. Inside a door the mask is the truth, so a hand-authored entry is overwritten here rather than merged: the two can only disagree, and the mask is the one derived from the data.

An empty ni returns sets untouched. Nothing left the universe, so nothing is owed, and declaring an empty axis would make fees_constraints resolve a carrier that prices no position.

Algorithm

  1. Return sets unchanged when ni is empty.
  2. Otherwise copy sets.dict, write ni under sets.nikey, and rebuild the UniverseSets from it and the seven unchanged key prefixes, which revalidates uniqueness and disjointness over the minted axis.

Arguments

  • sets: The UniverseSets to mint the axis on, or nothing.
  • ni: The names the Investable Mask left out, in the order the complement of the mask visits them.

Returns

  • sets: The UniverseSets carrying the Non-Investable Axis, or nothing.

Related

source
PortfolioOptimisers.non_investable_names — Function
non_investable_names(nx::Nothing, imsk::BitVector) -> VecStr
non_investable_names(nx::VecStr, imsk::BitVector) -> VecStr

Read the names the Investable Mask leaves out, in the order the complement of the mask visits them.

The order is the whole point. A forced-liquidation carrier is sliced to the complement of the mask by index, and the rate that prices it is resolved against these names by position, so the two must walk the complement the same way. Both do: this indexes nx with .!imsk, which is ascending, and port_opt_view(::Fees, i, X) takes the complement of i over the width of X, which is ascending too.

Unnamed returns data answers an empty vector rather than throwing. Names are what the axis is made of, so a problem with no names has no Non-Investable Axis to mint — and no name-keyed constraint to resolve against one either.

It lives here, beside non_investable_sets, rather than beside the optimisation door that first needed it, because a wrapping prior mints the axis at its own entry too and loads seven directories earlier. The vocabulary of the Non-Investable Axis is therefore one file, and no layer reaches it by a back reference.

Arguments

  • nx: The asset names of the unreduced returns data, or nothing.
  • imsk: The Investable Mask the optimisation reduced on: true at every asset whose prior moments were finite. It is nothing when every asset was investable, and that sentinel is what skips both the reduction and the expansion. investable_mask derives it once from the full-universe prior result, and the result carries it, because the reduced prior can no longer yield it.

Returns

  • ni::VecStr: The names the mask leaves out, or an empty vector.

Related

source
PortfolioOptimisers.record_non_investable_drop! — Function
record_non_investable_drop!(ledger::Nothing, what::AbstractString) -> Nothing
record_non_investable_drop!(ledger::AbstractVector, what::AbstractString) -> Nothing

Record, for the door to report, one thing a departure cost.

A departed name is dropped where it is met — a view row here, a group member there — and each of those places is far from the door that derived the mask and knows the departure happened as an event. Reporting at the site would say the same thing once per row per window of a walk-forward, and that repetition is refused. Reporting nothing leaves a caller who wrote three views and got one fitted with no way to learn it. So the site writes what it dropped into a ledger, and the door reads the ledger once and says both things together, through announce_non_investable.

A nothing ledger is the no-collection path, and it is the default everywhere: a caller who assembles constraints outside a door has no door to report to, and pays nothing for the ledger it does not keep. The branch is dispatch rather than a condition, as it is throughout the reduction machinery.

what is a noun phrase naming the casualty, not a sentence: the door joins them into one message and supplies the verb.

Arguments

  • ledger: The door's ledger, or nothing when nobody is collecting.
  • what: A noun phrase naming what was dropped, for example $"the view row `a + c == 0.05`"$.

Returns

  • nothing. A vector ledger is appended to in place.

Related

source
PortfolioOptimisers.record_group_shed! — Function
record_group_shed!(ledger::Option{<:AbstractVector}, group::AbstractString,
                   shed::Integer, kept::Integer, eqn::AbstractString) -> Nothing

Record what a group shed to a departure, for the door to report through announce_non_investable.

A group that loses some of its members still describes the rest, so its row survives at a coefficient spread over the survivors; a group that loses all of them describes nothing, and its row goes with it. The two are different news to a caller, so they are phrased differently, and this is the one place either sentence is written. replace_group_by_assets is the only caller, at each of its four expansion branches.

Counts, not names: the departed assets are named once by the door, and repeating them per group would make the message longer than what it reports.

A shed of nothing records nothing, so the all-investable path costs one comparison.

Arguments

  • ledger: The door's ledger, or nothing when nobody is collecting.
  • group: The group name as the caller wrote it, or the pair "(a, b)" for a correlation view.
  • shed: How many members the group lost.
  • kept: How many members survived.
  • eqn: The row the group appears in, as the caller wrote it.

Returns

  • nothing.

Related

source
PortfolioOptimisers.announce_non_investable — Function
announce_non_investable(ni::VecStr, drops::VecStr = String[],
                        process::AbstractString = "optimisation",
                        consequence::AbstractString = "…";
                        warn::Bool = false) -> Nothing

Announce, once per door, the assets that left the investable universe and what their leaving cost.

The door is the only place that knows a departure happened as an event rather than as a shape. Downstream, a departed asset is simply absent: a bound stated for it resolves on the Non-Investable Axis and is skipped, a view row naming it is dropped whole. Reporting each of those where it happens would say the same thing once per row per window of a walk-forward, so each site writes its casualty into a ledger with record_non_investable_drop! and the door says everything once, here.

process names the work the departure is excluded from, because more than one kind of door mints the axis: an optimisation reduces at its entry, and a wrapping prior reduces at its own before it builds a view. Hard-coding "optimisation" made the message wrong for the second. consequence states what a departure means to this door — a forced-liquidation carrier is priced by an optimisation and by nothing else — and both are ordinary defaults, so the optimisation door reads as it always did.

It is @info by default, not a warning and not a strict_diagnostic. Nothing is wrong: the data moved, and the work is proceeding correctly over what is left. Making it raise under strict would put back the refusal this whole path exists to remove. warn raises it to @warn for the one case that is not routine — a departure that took the last of something the caller asked for, such as the final view of a view set, because handing back the unconditioned answer changes the result and the caller has no other way to learn it.

An empty ni says nothing at all, which is the all-investable path and the unnamed-data path alike. An empty drops says who left and stops there, which is the door that has not yet resolved anything over them.

Arguments

  • ni: The names the Investable Mask left out.
  • drops: The ledger of casualties, as record_non_investable_drop! filled it.
  • process: Noun phrase naming the work, for example "optimisation" or "entropy pooling fit".
  • consequence: Sentence stating what a departure means to this door.
  • warn: Raise the message to @warn, for a departure that changed the model rather than trimming it.

Returns

  • nothing.

Related

source
PortfolioOptimisers.counterpart_axis_names — Function
counterpart_axis_names(sets::UniverseSets, nxkey::AbstractString) -> VecStr

Return the asset axis that nxkey is the counterpart of, or an empty vector when it has none.

UniverseSets declares two axes over assets: the investable universe under xkey, and the Non-Investable Axis under nikey, which a door mints from the complement of the Investable Mask. A name-keyed estimator resolves against one of them, and a name it does not find there may still be a perfectly good name on the other — a bound stated for an asset that has since left, or a forced-liquidation rate stated for one that stayed. name_to_val! needs that list to tell such a name from a typo, and this is where the pairing is written down.

The relation is symmetric and covers only these two. A factor axis names factors, so no asset name is ever a departed factor and the answer is empty; a caller-supplied key naming some other list is treated the same way.

An axis that sets does not declare answers empty, which is the common case: a problem in which every asset is investable carries no nikey entry at all.

Arguments

  • sets: The UniverseSets whose axes are read.
  • nxkey: The key of the axis being resolved against.

Returns

  • other::VecStr: The counterpart axis, or an empty vector.

Related

source
PortfolioOptimisers.shed_departed_members — Function
shed_departed_members(members::AbstractVector, other::VecStr,
                      ledger::Option{<:AbstractVector}, group::AbstractString,
                      eqn::AbstractString) -> AbstractVector
shed_departed_members(members1::AbstractVector, members2::AbstractVector,
                      other::VecStr, ledger::Option{<:AbstractVector},
                      group::AbstractString, eqn::AbstractString) -> Tuple

Strike from a group's member list the names that sit on the counterpart axis, and tell the door's ledger what went.

A group is a description the data resolves, not a term the caller chose: "tech" means the technology assets of this problem, and when one of them delists the description still names the rest. replace_group_by_assets therefore sheds the departed members before it spreads the group's coefficient, so a Black–Litterman mean divides by the surviving count and an entropy pooling sum runs over the survivors — the row still computes what its right-hand side asserts. Striking a member afterwards would leave k - 1 legs of c/k against an unchanged target. That is why a group differs from a written-out name, which takes its row with it.

A group that loses every member keeps the first of them rather than answering empty, because a group that describes nobody is a row naming a departed asset, and saying so is what makes it drop by the counterpart rule one door later — whole, and in silence. Answering empty would leave a row with no variable in it, which is what a caller writing 1 == 0.004 produces, and that one still has to be diagnosed.

The second method is the pair form, for a correlation view written over two groups. The two lists are walked together, and a position is kept only when both of its names survived: a pair is one correlation, so a pair that has lost either side has nothing left to measure, and shedding jointly is also what keeps the two lists the same length, which replace_group_by_assets has already checked. The all-lost case keeps the first pair, on the same reasoning.

An empty other is the all-investable path, and both methods then return their arguments untouched and record nothing, so a problem with no departure pays one comparison and no allocation.

The recording lives here rather than at the four call sites so that replace_group_by_assets spends one line per branch on the whole of it. The four branches are otherwise identical, and four copies of the shed, the record and the all-lost fallback is where they would drift.

Arguments

  • members / members1, members2: The group's member names, as sets.dict holds them.
  • other: The counterpart axis, read with counterpart_axis_names. Usually the Non-Investable Axis.
  • ledger: The door's ledger, or nothing when nobody is collecting.
  • group: The group name as the caller wrote it, for the ledger.
  • eqn: The row the group appears in, for the ledger.

Returns

  • members::AbstractVector: The members that are not on other, in their original order, or the first departed member when none survived.
  • (members1, members2)::Tuple: The pair form, restricted to the positions both lists survived, or the first pair when none did.

Related

source