GitHub

Linear Constraints (b): private API

PortfolioOptimisers.get_linear_constraints — Function
get_linear_constraints(lcs::PR_VecPR, sets::UniverseSets,
                       key::Option{<:AbstractString} = nothing;
                       datatype::DataType = Float64, strict::Bool = false,
                       rr::Option{<:AbstractLoadingsRegressionResult} = nothing,
                       ledger::Option{<:AbstractVector} = nothing)

Convert parsed linear constraint equations into a LinearConstraint object.

The function resolves the variable names of one or more ParsingResults, as parse_equation returns them, against the universe of the UniverseSets. It assembles the coefficient matrices and the right-hand sides, and returns a LinearConstraint that holds the equality rows and the inequality rows.

A row takes one of two shapes. Without rr, it runs over the universe that the names resolve against. With rr, it runs over the assets, because the loadings re-base each term during the assembly and the result is an ordinary asset-space row.

The function drops whole rows and never single terms. A row is one statement over several names with one right-hand side, so a name that the function cannot resolve drops the whole row. a + c == 0.05 without c fits a == 0.05, which is a different and stronger statement than the caller wrote. The cause of the failure decides only whether the function reports the drop. The function drops a row that names the counterpart axis in silence, whatever strict is. counterpart_axis_names reads that axis, which in practice is the Non-Investable Axis that a door wrote. The name was correct over the universe that the caller had, and the data moved it, so the door reports the departure once. A name on neither axis is a typo, and the function reports it.

Algorithm

  1. Take k as key, or sets.xkey when key is nothing, read the universe nx from sets.dict under it, name the axis with universe_axis, and read the counterpart axis with counterpart_axis_names.
  2. Take N, the row length, from constraint_row_length, and allocate the working row At of that length.
  3. Zero At for each parsing result, and start that result not dropped.
  4. Build the indicator of each variable name of the result over nx. A name that matches no entry marks the row dropped, and is reported through strict_diagnostic unless it names the counterpart axis. Every name of the row is still visited, so a row carrying two typos names both.
  5. Add the contribution that constraint_row_term gives for the name and its coefficient to At. With rr the contribution is already projected, so At has one entry per asset.
  6. Move to the next result when the row was marked dropped.
  7. Report the row through strict_diagnostic and drop it when At is still zero. Every name resolved before this step, so the message says that the row sums to zero, because of the loadings under rr, because its coefficients cancel, or because it holds no name. It never says that a name has a typo.
  8. Read the sign and the inequality flag of the operator from comparison_sign_ineq_flag, and scale the row and its right-hand side by the sign. That negates a >= row, so both senses of an inequality are written in the <= sense, which is the convention LinearConstraint states.
  9. Append the row to the inequality accumulator when the flag is true, and to the equality accumulator when it is false.
  10. Reshape each accumulator that holds a row into a matrix of N columns, and build the PartialLinearConstraint of that half.
  11. Return the LinearConstraint holding the halves that were built, or nothing when neither half holds a row.

Arguments

  • lcs: A single ParsingResult or a vector of such objects, representing parsed constraint equations.
  • sets: A UniverseSets object specifying the universes and groupings.
  • key: Key naming the universe the variables resolve against. Defaults to sets.xkey; a re-based constraint passes the factor axis key that factor_axis_key reads off the loadings.
  • datatype: Numeric type for coefficients and right-hand side.
  • strict: If true, a variable name that the universe does not hold throws. If false, it warns.
  • rr: Loadings to re-base through, or nothing for an ordinary asset-space constraint. Callers do not pass it directly, and ExposureConstraintEstimator passes it.
  • ledger: The door's ledger of departure casualties, or nothing when nobody is collecting. A row dropped for a name on the counterpart axis is recorded into it through record_non_investable_drop!.

Validation

  • lcs is non-empty, when it is a vector.
  • A variable name that matches no entry of the universe and none of the counterpart axis raises when strict is true, and issues a warning otherwise. The row is dropped either way.
  • A variable name on the counterpart axis drops its row in silence, under both settings of strict.
  • A row whose terms all fall away raises when strict is true, and issues a warning otherwise. The row is dropped either way.
  • Each op is one of "==", "<=" or ">=", which comparison_sign_ineq_flag enforces.

Returns

  • lcs::LinearConstraint: An object containing the assembled equality and inequality constraints, or nothing if no constraints are present.

Related

source
PortfolioOptimisers.universe_axis — Function
universe_axis(sets::UniverseSets, key::AbstractString) -> String

Return the name of the axis of the universe under key, read from the key itself.

A key that starts with tfkey or cfkey gives "factor", and any other key gives "asset". unknown_variable_msg and empty_row_msg read it to name the axis that the user wrote in.

Both callers resolve names against sets.dict[key] alone, so the axis of that universe is the axis where a lookup failed, and the key is the right source. get_black_litterman_views takes the key from the estimator that owns the views, and get_linear_constraints takes it from the constraint space. FactorSpace resolves at the factor axis that factor_axis_key reads from the loadings, sets.tfkey for the time-series family and sets.cfkey for the cross-sectional family. The re-basis is the wrong source. A wrapped estimator with its own key replaces the key of the space, so a re-based row can resolve against a universe that the loadings do not use, and the message must name the universe that the lookup searched.

The test is on the prefix and not on equality, so a factor group key such as "nf_sector" or "ncf_sector" also gives the factor axis. UniverseSets refuses a key prefix that starts with another, so the answer is unique.

Algorithm

  1. Return "factor" when key starts with sets.tfkey or with sets.cfkey.
  2. Return "asset" in every other case.

Arguments

  • sets: The UniverseSets whose tfkey and cfkey name the two factor axes.
  • key: The key the names were resolved against.

Returns

  • axis::String: "factor" or "asset", the word a diagnostic message uses to name the axis.

Related

source
PortfolioOptimisers.constraint_row_length — Function
constraint_row_length(rr, nx::VecStr) -> Int

Return the length of an assembled constraint row.

Without a re-basis, the length is the size of the universe that the names resolve against. With a re-basis, it is the number of assets that the loadings project onto, because the assembly of the row applies the projection and gives an ordinary asset-space row.

Algorithm

The method that Julia selects is the algorithm, and the re-basis selects it.

  1. When rr is nothing, return the length of nx, the universe the names resolve against.
  2. When rr is a regression result, return the number of rows of rr.M, which is the number of assets the loadings project onto.

Arguments

  • rr: Loadings to re-base through, or nothing for an ordinary asset-space row.
  • nx: The universe the names resolve against.

Returns

  • N::Int: The number of entries one assembled row has.

Related

source
PortfolioOptimisers.constraint_row_term — Function
constraint_row_term(::Nothing, Ai, c)
constraint_row_term(rr::AbstractLoadingsRegressionResult, Ai, c)

Return the contribution of one matched variable to a constraint row.

Without a re-basis, the contribution is the indicator Ai times the coefficient c. With a re-basis, it is the sum of the columns of the loadings that Ai selects, times c. The function sums the columns and does not take only the first match, so a duplicated name in a factor universe contributes every column with that name, as a duplicated asset name does on the asset path.

The function reads rr.M and never rr.L. The columns of M are the named original factors, and a constraint must use names that a user can write in an equation. Risk decomposition reads L instead.

Mathematical definition

\[\begin{align} \boldsymbol{a}^\intercal \boldsymbol{w}_f &= \boldsymbol{a}^\intercal \mathbf{M}^\intercal \boldsymbol{w}_a = (\mathbf{M} \boldsymbol{a})^\intercal \boldsymbol{w}_a\,. \end{align}\]

Where:

  • $\boldsymbol{a}$: A constraint row written in factor names.
  • $\boldsymbol{w}_f$: The factor weights that row is written against.
  • $\boldsymbol{w}_a$: The asset weights the optimiser holds.
  • $\mathbf{M}$: The factor loadings, one column per named factor and one row per asset.

The identity lets a row written in factor names bind the asset weights with no change of variables. The re-based row is an ordinary asset-space row over $\boldsymbol{w}_a$.

Algorithm

The method that Julia selects is the algorithm, and the re-basis selects it.

  1. When rr is nothing, return Ai times c, one entry per name of the universe.
  2. When rr is a regression result, sum the columns of rr.M that Ai selects, multiply the sum by c, and return it. The value has one entry per asset, whatever names the row uses.

Arguments

  • rr: Loadings to re-base through, or nothing for an ordinary asset-space row.
  • Ai: The indicator of the matched name over the universe the names resolve against.
  • c: The coefficient the matched name carries.

Returns

Related

source
PortfolioOptimisers._parse_equation — Function
_parse_equation(lhs, opstr::AbstractString, rhs, datatype::DataType = Float64)

Parse and canonicalise a linear constraint equation from Julia expressions.

The function takes both sides of an equation as expressions, and the comparison operator as a string. It evaluates the numeric calls, moves every term to the left-hand side, sums the coefficient of each variable, and returns the equation in canonical form as a ParsingResult.

Algorithm

  1. Fold the constant subexpressions of both sides with eval_numeric_functions, giving lexpr and rexpr, and check each with rethrow_parse_error.
  2. Build diff_expr, the expression lexpr - (rexpr). This moves every term of the equation to the left-hand side.
  3. Walk diff_expr with _collect_terms, giving terms, one (coefficient, variable) pair per term.
  4. Accumulate terms into varmap, which holds the summed coefficient of each variable name, and into constant, the sum of the coefficients that carry no variable.
  5. Read variables and coefficients off varmap, and take rhs_val as the negated constant. This moves the constant to the right-hand side.
  6. Render each pair with format_term, join the renderings with +, and fold + - into -, giving the canonical string formatted.
  7. Return the ParsingResult built from variables, coefficients, opstr, rhs_val and formatted.

Arguments

  • lhs: Left-hand side of the equation as a Julia expression.
  • opstr: Comparison operator as a string.
  • rhs: Right-hand side of the equation as a Julia expression.
  • datatype: Numeric type for coefficients and right-hand side.

Returns

  • res::ParsingResult: The variables, coefficients, operator, right-hand side and formatted string of the equation in canonical form. The order of vars is the order the variable map iterates in, and it is not the order the equation was written in.

Related

source
PortfolioOptimisers.has_invalid_plus — Function
has_invalid_plus(expr)

Check whether a Julia expression contains the operator ++ anywhere in its tree.

It is the Expr counterpart of the ++ check that the string form of parse_equation runs on the raw text.

Algorithm

  1. Return false when expr is not a call, because only a call can carry the head this function refuses.
  2. Return true when the head of the call is the ++ operator.
  3. Apply this function to every argument of the call that is itself an expression, and return true when any of them does.

Arguments

  • expr: Julia expression to check.

Returns

  • Bool: true if the tree holds a ++ call, false otherwise.

Related

source
PortfolioOptimisers._expr_depth_exceeds — Function
_expr_depth_exceeds(x, limit::Integer) -> Bool

Return true if the expression tree x is deeper than limit.

It protects the Expr form of parse_equation from a deeply nested tree that no string length limit covers. The depth counts the levels of Expr along the deepest path, so :(a + b) has depth one. The check recurses at most limit + 1 frames deep and stops at the first branch that passes the limit, so it cannot use up the stack that it protects.

Algorithm

  1. Return true when limit is negative, because the walk has already gone one level past the cap.
  2. Return false when x is not an expression, because a leaf adds no depth.
  3. Apply this function to every argument of x, with limit lowered by one, and return true when any of them does. The scan stops at the first argument that answers true.

Arguments

  • x: The expression tree to measure.
  • limit: The greatest depth the tree may have.

Returns

  • Bool: true when the tree is deeper than limit, false otherwise.

Related

source
PortfolioOptimisers.assert_investable_constraint_width — Function
assert_investable_constraint_width(lcs::Nothing, N::Integer, slot::AbstractString)
assert_investable_constraint_width(lc::LinearConstraint, N::Integer,
                                   slot::AbstractString)
assert_investable_constraint_width(lcs::VecLc, N::Integer, slot::AbstractString)

Refuse a precomputed LinearConstraint whose rows are wider than the investable universe, with a message that states why.

A name-keyed estimator stays valid after a reduction to the Investable Mask. It resolves against the UniverseSets that the door gives it, and a name that left resolves on the Non-Investable Axis and is not refused. A precomputed constraint cannot do this. Its A is a matrix, and the position is the only link between a column and an asset, so there is no name to resolve again and no correct way to narrow the matrix. A dropped column changes what Ax ≤ B means, and for that reason port_opt_view(::LinearConstraint, i) is the identity.

The row therefore passes the door at its original width and meets a shorter weight vector. Without this check, the model raises a bare DimensionMismatch between two numbers, and nothing links either number to the asset that delisted. This function raises once, before the model, and its message states what the caller did and what the caller can do instead.

The repair is always the same. State the constraint as a LinearConstraintEstimator, which resolves its names again over the universe that the door leaves.

Algorithm

  1. Return when there is nothing to check, which is a nothing slot or a nothing half of a LinearConstraint.
  2. Otherwise compare size(A, 2) of each half against N and throw a DimensionMismatch naming the slot, the two widths and the repair when they disagree.

Arguments

  • lcs: The resolved constraint, a vector of them, or nothing.
  • N: The number of investable assets the optimisation runs over.
  • slot: Names the field the constraint came from in the message, for example "lcse".

Validation

  • Every half of every precomputed constraint has one column per investable asset.

Returns

  • nothing.

Related

source