Propagatable: private API

Utility functions

PortfolioOptimisers.PROP_TAG_NAMES — Constant
PROP_TAG_NAMES

The propagation tag set of @propagatable, as data.

One entry per field tag. The code that recognises a tag comes from this tuple, and no tag is written out in that code. The macro names (PROP_TAG_MACRO_NAMES), the lookup (prop_tag), the test (is_prop_tag_call), the peeler (peel_prop_tags) and the parser (propagatable_parse_body) all read it. A new tag is one row here, one branch in prop_tag_expr, one entry in a channel of PROP_TAG_CHANNELS and one stub macro. check_prop_tag_macros refuses to load the module when a row has no stub macro, no channel or no branch.

Related

source
PortfolioOptimisers.PROP_TAG_MACRO_NAMES — Constant
PROP_TAG_MACRO_NAMES

The macro name of every tag of PROP_TAG_NAMES, in the same order.

The tuple comes from the tag names, so a row of the table has no second spelling. prop_tag matches the head of a :macrocall against this tuple.

Algorithm

  1. Map each tag of PROP_TAG_NAMES to Symbol("@", tag), which is the name that Julia gives the macro of that tag.

The map keeps the order, so the $k$-th entry here is the macro name of the $k$-th tag there. prop_tag and check_prop_tag_macros both walk the two tuples together, and they need this shared order to pair each tag with its macro name.

Related

source
PortfolioOptimisers.PROP_TAG_CHANNELS — Constant
PROP_TAG_CHANNELS

The propagation channels of @propagatable, with their tag precedence as data.

One entry per generated method. Each entry has two tuples of tags:

  • gate: the tags that make @propagatable emit the method.
  • precedence: the order in which the channel reads the tags of a field. The first tag of this tuple that the field carries gives the transform of the field. The channel ignores the other tags of that field.

The factory channel prefers @fprop to @wprop. The prior channel prefers @pprop, then @cprop, then @wprop, then @fprop, so @pprop wins over @fprop on one field. prop_channel_pairs reads the precedence for every channel.

A tag means what its channel says it means. The obs channel reads the same @wprop and @fprop tags as factory, and it gives them different transforms. factory replaces a @wprop field with an incoming ObsWeights value, and obs indexes the value that the field holds. For this reason, prop_tag_expr takes the channel as well as the tag. A weights field joins obs_weights_view through its @wprop tag, and it needs no second tag.

The obs channel opens on a @wprop, a @fprop or an @oprop field. Every reader in a fit reads the observations of the same sample, so a composed child that holds weights deep in its tree must take the cut too, see shared_series_estimator. A struct whose only tags are @fprop therefore gets an obs_weights_view method that recurses into its children. @oprop marks a child that the obs channel alone reads, for a child that factory must not reach.

Related

source
PortfolioOptimisers.PROPAGATABLE_CONTRACTS — Constant
PROPAGATABLE_CONTRACTS

Every type declared with @propagatable, with the names of its @pprop fields.

The macro adds one entry immediately after the struct that it declares. When this module finishes its load, the list holds every type of this package, and check_propagatable_contracts reads it at that point. An external package adds its types when it loads, after that check, and it must call check_propagatable_contracts at the end of its own module.

Related

source
PortfolioOptimisers.prop_tag — Function
prop_tag(x) -> Union{Nothing, Symbol}

Return the tag of PROP_TAG_NAMES that x names, or nothing.

x is the head of a :macrocall node. It is a bare Symbol in a struct body written by hand, and a GlobalRef after another macro expanded around it, so the function resolves both. A name outside PROP_TAG_MACRO_NAMES gives nothing. The callers use this to tell a tag from any other macro. A name never resolves to a tag that it does not name.

Algorithm

  1. Read name from x. A GlobalRef gives its name field, a Symbol gives itself, and any other value returns nothing at once.
  2. Walk PROP_TAG_NAMES and PROP_TAG_MACRO_NAMES together. Return the tag whose macro name is identical to name.
  3. No macro name matches: return nothing.

The comparison is === on a Symbol, so only an identical name matches. A macro with a similar name never matches.

Arguments

  • x: The first argument of a :macrocall expression.

Returns

  • tag::Symbol: The tag name, without the @.
  • nothing: If x names no tag.

Related

source
PortfolioOptimisers.is_prop_tag_call — Function
is_prop_tag_call(x) -> Bool

Return true if x is a macro call to any tag of PROP_TAG_NAMES.

peel_prop_tags and propagatable_parse_body use it to find the tagged fields of a struct body.

Algorithm

  1. Return true when all three hold: x is an Expr, its head is :macrocall, and prop_tag of its first argument is not nothing.
  2. Return false otherwise.

Arguments

  • x: Any expression appearing in a struct body.

Related

source
PortfolioOptimisers.prop_tag_expr — Function
prop_tag_expr(
    channel::Symbol,
    tag::Symbol,
    fname::Symbol,
    xf,
    mod::Module,
    thread
) -> Expr

Return the expression that a tag substitutes for one field, inside a generated method.

This function gives the field transform of each tag. It has one branch per tag of PROP_TAG_NAMES, and prop_channel_pairs calls it for every channel. A tag of the table with no branch here raises an error, so it cannot take the transform of another tag without a sign.

Algorithm

The function reads the channel first, and then the tag, because one tag has two transforms.

  1. channel is :obs:
    1. tag is :wprop: return obs_weights_index(xf, thread...). The field is the weights, so the method indexes it at the selected observations. An index keeps the AbstractWeights subtype, and a view does not. obs_weights_index states the rule for a mask of the full sample.
    2. tag is :oprop or :fprop: return obs_weights_view(xf, thread...). The field is a composed child, so the verb recurses into it.
  2. channel is any other channel:
    1. tag is :fprop: return factory(xf, thread..., args...; kwargs...). The traversal owns the identity on a leaf, so a field that holds data or nothing comes back with no change.
    2. tag is :vprop: return view_child(xf, thread..., args...), which calls port_opt_view on every value but a precomputed optimisation result. This channel forwards no keywords.
    3. tag is :pprop: return sel(xf, getproperty(pr, fname)). The prior result gives the property of the same name, so the field name is an argument.
    4. tag is :cprop: return sel(xf, _ctx(args...)), which reads the context from the threaded arguments and not from the prior.
    5. tag is :wprop: return _wprop(xf, args...; kwargs...), which replaces the field with an incoming ObsWeights.
  3. No branch matched: raise an error that names the tag and the channel, and asks for a branch here.

Steps 1.1 and 2.5 are the same tag with two transforms, so the channel decides what @wprop means. Every emitted name is qualified with mod, because the expansion is escaped into the module of the caller.

Arguments

  • tag::Symbol: A tag of PROP_TAG_NAMES.
  • fname::Symbol: The field name. @pprop needs it to name the prior property.
  • xf: The expression that reads the field from the incoming struct.
  • mod::Module: The module that defines @propagatable. Every emitted name is qualified with it, because the expansion is escaped into the module of the caller.
  • thread: The positional arguments that the channel threads before args....

Returns

  • expr::Expr: The value of the field in the generated constructor call.

Related

source
PortfolioOptimisers.prop_channel_active — Function
prop_channel_active(
    channel::Symbol,
    tagged::AbstractDict
) -> Any

Return true if a channel of PROP_TAG_CHANNELS must emit a method.

A channel is active when at least one field carries a tag of the gate of the channel.

Algorithm

  1. Read the gate tuple of the channel from PROP_TAG_CHANNELS.
  2. Return true when at least one tag of gate has a non-empty entry in tagged, and false otherwise.

This function does not read the precedence tuple. So a tag that a channel reads, but does not gate on, never makes that channel emit a method alone. The prior channel reads @wprop and @fprop, but it gates on @pprop and @cprop, so a type with @fprop alone gets no prior method.

Arguments

Related

source
PortfolioOptimisers.prop_channel_pairs — Function
prop_channel_pairs(
    channel::Symbol,
    tagged::AbstractDict,
    all_fields::AbstractVector{Symbol},
    obj::Symbol,
    mod::Module,
    thread
) -> Vector{Any}

Return the keyword pairs of the constructor call that one channel generates.

Every declared field gets one pair, in declaration order. The function reads the tags of a field in the precedence order of the channel. The first match gives the value through prop_tag_expr. A field with no tag of the channel keeps its value.

Algorithm

  1. Read the precedence tuple of the channel from PROP_TAG_CHANNELS.
  2. For each field name fname of all_fields, in declaration order:
    1. Build xf, the expression obj.fname that reads the field from the incoming struct.
    2. Find idx, the position of the first tag of precedence that fname carries.
    3. When idx is nothing, the field carries no tag of this channel, and the value is xf.
    4. Otherwise, the value is prop_tag_expr of that tag, in this channel.
    5. Push Expr(:kw, fname, value) onto pairs.
  3. Return pairs.

In step 2.2, the precedence decides the transform of one field. A field with @pprop and @fprop takes the @pprop transform on the prior channel and the @fprop transform on the factory channel, because the two channels put the tags in different orders.

Arguments

  • channel::Symbol: A channel name of PROP_TAG_CHANNELS.
  • tagged::AbstractDict: Map from a tag name to the names of the fields that carry it.
  • all_fields::AbstractVector{Symbol}: Every declared field, in declaration order.
  • obj::Symbol: The struct from which the generated method reads the fields.
  • mod::Module: The module that defines @propagatable.
  • thread: The positional arguments that the channel threads before args....

Returns

  • pairs::Vector{Any}: One Expr(:kw, field, value) per declared field.

Related

source
PortfolioOptimisers.check_prop_tag_macros — Function
check_prop_tag_macros()
check_prop_tag_macros(tags)
check_prop_tag_macros(tags, macro_names)
check_prop_tag_macros(tags, macro_names, channels)
check_prop_tag_macros(
    tags,
    macro_names,
    channels,
    mod::Module
)

Check that every tag of PROP_TAG_NAMES is complete.

A tag row is complete when it has a stub macro, a channel of PROP_TAG_CHANNELS that names it, and a field transform in prop_tag_expr for every channel that names it. A row without one of the three is a tag that parses but never propagates, and the table exists to stop that failure. A tag means what its channel says it means, so the function probes each channel. That probe finds a tag that a second channel names without a transform there. The function collects all the violations and reports them together.

The module calls this function once, at its end. It throws an ArgumentError that lists every violation, so the package does not precompile with a tag that never propagates.

The three tables are arguments, and each one has the table of the module as its default. The tables of the module are complete, so the call of the module never reports a violation. A caller can pass a table of its own to make each of the three clauses fail and read the message that it gives.

Algorithm

  1. Make violations, an empty vector of strings.
  2. For each tag of PROP_TAG_NAMES, with its macro name from PROP_TAG_MACRO_NAMES:
    1. The macro name is not defined in mod: push a message that the tag declares no stub macro.
    2. The tag is in the precedence of no channel of channels: push a message that the tag is in no channel.
    3. For each channel whose precedence names the tag, call prop_tag_expr with the probe name :probe. When that call raises an error, push a message that names the tag and the channel.
  3. violations is not empty: throw an ArgumentError that lists all of them.
  4. Return nothing.

Step 2.3 probes each channel that names the tag, not the tag alone. This probe finds a tag that a second channel names with no transform there, and a probe of the tag alone does not.

Arguments

  • tags: The tag names to check.
  • macro_names: The stub macro name of each tag, in the order of tags.
  • channels: The channel table whose precedence tuples the function reads.
  • mod::Module: The module in which the function finds the stub macros. It also qualifies the names that prop_tag_expr emits.

Returns

  • nothing: Every row of tags is complete.

Related

source
PortfolioOptimisers.is_doc_macro — Function
is_doc_macro(x) -> Union{Missing, Bool}

Return true if x is a reference to Julia's @doc macro (bare Symbol or GlobalRef).

propagatable_parse_body uses it to find the fields of a struct body that have a docstring.

Algorithm

  1. Return true when x is a GlobalRef whose name is Symbol("@doc").
  2. Return true when x is equal to Symbol("@doc").
  3. Return false otherwise.

The function needs both spellings for the same reason as prop_tag. A struct body written by hand has the bare Symbol, and a struct body that another macro expanded has the GlobalRef.

Related

source
PortfolioOptimisers._ctx — Function
_ctx(args...)

Find the one optimiser value that a prior factory call threads, a solver of type Slv_VecSlv, in the variadic tail of the call, and return nothing when the tail holds none. The @cprop tag emits this call as the source argument of sel. The compiler unrolls the scan of the tuple, so the call is type-stable and does not allocate.

Related

source
PortfolioOptimisers.extract_field_name — Function
extract_field_name(expr) -> Symbol

Extract the field name Symbol from a bare field or field::Type expression.

The function raises an error that names expr when expr is neither a bare Symbol nor a field::Type annotation. Only those two forms are valid after a tag such as @fprop.

Algorithm

  1. expr is a Symbol: return it.
  2. expr is an Expr whose head is :(::): return its first argument, which is the field name.
  3. expr is any other expression: raise an error that names the expression.

Step 3 is the difference from try_field_name, which returns nothing in the same case. A tag states that the node is a field. So a node that is not a field is a defect in the struct body, and the function does not skip it.

Arguments

  • expr: A Symbol, an Expr with head :(::), or any other expression, which raises an error.

Returns

  • name::Symbol: The field name.

Related

source
PortfolioOptimisers.propagatable_find_struct — Function
propagatable_find_struct(
    expr
) -> Union{Tuple{Expr, typeof(identity)}, Tuple{Expr, Union{PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, typeof(identity)}, PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, rebuild}} where rebuild}}}

Find the innermost :struct node of a chain of macro calls, through a recursion.

The function returns (struct_node, rebuild_fn). rebuild_fn(new_struct) builds the original macro chain again, with new_struct in place of the original struct. So @propagatable can put a changed struct definition back inside any wrapping macro, such as @concrete.

Algorithm

  1. expr is not an Expr: raise an error that names its type.
  2. expr has head :struct: return expr and identity, which rebuilds nothing.
  3. expr has head :macrocall: take inner, its last argument, and call this function again on it. That call gives struct_node and rebuild. Read prefix, every argument of expr except the last. Return struct_node and the function s -> Expr(:macrocall, prefix..., rebuild(s)).
  4. expr has any other head: raise an error that names the head.

Step 3 builds the chain again from the inside out. A struct inside several macros comes back inside the same macros, in the same order, with the same arguments. The prefix holds the arguments of the macro and its LineNumberNode, so the rebuilt call loses nothing.

Arguments

  • expr: A :struct expression, or a :macrocall expression around one.

Returns

  • struct_node::Expr: The innermost :struct expression.
  • rebuild_fn::Function: A function that takes a replacement :struct and returns the full macro chain, with the replacement in place of the original.

Related

source
PortfolioOptimisers.propagatable_bare_name — Function
propagatable_bare_name(n) -> Symbol

Get the plain struct name, as a Symbol, from a name expression that can have type parameters and a supertype.

The function accepts the forms Name, Name{T, ...} and Name{T, ...} <: SuperType. It peels the :curly and :<: nodes, through a recursion, until it gets a bare Symbol.

Algorithm

  1. n is a Symbol: return it.
  2. n has head :curly: call this function again on its first argument, which drops the type parameters.
  3. n has head :<:: call this function again on its first argument, which drops the supertype.
  4. n is any other expression: raise an error that names the expression.

Steps 2 and 3 work together, so Name{T} <: Super loses the supertype first and then the parameters.

Arguments

  • n: A Symbol, or an Expr with head :curly or :<:.

Returns

  • name::Symbol: The plain struct name.

Related

source
PortfolioOptimisers.try_field_name — Function
try_field_name(expr) -> Symbol

Return the field name, as a Symbol, of a plain field declaration, or nothing for a node that is not a field.

The function accepts a bare Symbol field and a field::Type annotation. It returns nothing for a LineNumberNode, an inner constructor and every other expression that does not declare one named field.

Algorithm

  1. expr is a Symbol: return it.
  2. expr has head :(::) and its first argument is a Symbol: return that argument.
  3. expr is any other expression: return nothing.

Step 2 tests the first argument as well as the head, and extract_field_name does not. So a node such as a.b::Int, whose first argument is not a Symbol, gives nothing here.

Arguments

  • expr: Any expression appearing in a struct body.

Returns

  • name::Symbol: The field name, if expr is a plain field declaration.
  • nothing: If expr is not a plain field declaration.

Related

source
PortfolioOptimisers.peel_prop_tags — Function
peel_prop_tags(expr) -> Tuple{Set{Symbol}, Any}

Peel the tag macrocalls off a field expression, and record the tags.

A field can have a stack of tags in any order, such as @pprop @fprop field. The stack parses as nested :macrocall nodes. This function removes all of them and returns the bare field expression. It finds each tag with prop_tag, so a macro that is not a tag stops the peel.

Algorithm

  1. Make tags, an empty Set{Symbol}.
  2. While is_prop_tag_call of expr holds, push prop_tag of its first argument onto tags, and replace expr with its last argument, which is the expression the tag wraps.
  3. Return tags and the peeled expr.

tags is a set, so a tag written two times on one field is recorded once. The loop stops at the first node that is not a tag call, and a macro between two tags that is not a tag hides the tags below it.

Arguments

  • expr: A field expression, with or without tag macrocalls around it.

Returns

  • tags::Set{Symbol}: The tags of PROP_TAG_NAMES that expr carries.
  • stripped: The field expression with all tags removed.

Related

source
PortfolioOptimisers.propagatable_parse_body — Function
propagatable_parse_body(
    body
) -> Tuple{Dict{Symbol, Vector{Symbol}}, Vector{Symbol}, Expr}

Walk a struct body. Collect the names of the tagged fields and of all fields, and remove the tags from the body.

The function accepts a bare tagged field such as @fprop field, a stack of tags in any order such as @pprop @fprop field, and a field with a docstring such as "doc" \n @fprop field. It keeps a node that is not a field, such as a line number or an inner constructor, with no change. The tags are the rows of PROP_TAG_NAMES, so a new tag needs no change here.

Algorithm

  1. Make tagged, one empty vector per tag of PROP_TAG_NAMES. Make all_fields, an empty vector, and new_args, an empty vector for the stripped body.
  2. For each node arg of the struct body, in declaration order, take one of three branches:
    1. arg is a @doc macrocall, which is how a field with a docstring parses. Peel the tags off inner, its last argument.
      1. The field carries at least one tag: record the field name under each of its tags and in all_fields. Then push a rebuilt @doc node whose last argument is the stripped field.
      2. The field carries no tag: record its name in all_fields when try_field_name finds one, and push arg with no change.
    2. arg is a tag macrocall with no docstring: peel the tags, record the field name under each of them and in all_fields, and push the stripped field expression.
    3. arg is any other node, such as a LineNumberNode, an untagged field or an inner constructor: record its name in all_fields when try_field_name finds one, and push arg with no change.
  3. Return tagged, all_fields, and the new body as one :block expression.

all_fields holds every declared field, with or without a tag. With it, the generated constructor call names every keyword. The returned body has no tag, so the wrapped macros and Julia never see one.

Arguments

  • body::Expr: The :block expression of the struct body.

Returns

  • tagged::Dict{Symbol, Vector{Symbol}}: One entry per tag of PROP_TAG_NAMES, which holds the names of the fields that carry the tag, in declaration order.
  • all_fields::Vector{Symbol}: The names of every declared field, with or without a tag.
  • new_body::Expr: The struct body with no tags.

Related

source
PortfolioOptimisers.propagatable_register! — Function
propagatable_register!(
    T::Type,
    pprops::Tuple{Vararg{Symbol}}
)

Record the type T and the names of its @pprop fields in PROPAGATABLE_CONTRACTS.

@propagatable calls it at the declaration. The function only records. The outer keyword constructor is written below the struct, so it does not exist yet, and the function cannot check it. check_propagatable_contracts does the check when the module is complete.

Algorithm

  1. Push the pair (T, pprops) onto PROPAGATABLE_CONTRACTS.
  2. Return nothing.

T has @nospecialize, so one method serves every registered type, and the registration costs no compilation.

Related

source
PortfolioOptimisers.propagatable_keywords — Function
propagatable_keywords(T::Type) -> Vector{Symbol}

Return the keyword names that the outer constructors of T accept, as the union over its methods.

The function drops a kwargs... slurp and does not count it. A slurp accepts field = value and then discards it without an error. The contract check exists to find that failure, so a slurp must not satisfy the contract.

Algorithm

  1. Make kws, an empty vector of symbols.
  2. For each method m of the constructor T, append Base.kwarg_decl(m) to kws. The union is over every outer constructor, so a keyword that any one of them names counts.
  3. Remove the repeats from kws.
  4. Remove every name whose string ends in ..., because Base.kwarg_decl gives a slurp in that form.
  5. Return kws.

Step 4 is the reason for the function. A constructor with kwargs... gives the slurp as a keyword name. If the function counted it, every field would satisfy the contract.

Related

source
PortfolioOptimisers.propagatable_contract_violations — Function
propagatable_contract_violations(
    T::Type,
    pprops,
    pool
) -> Vector{String}

Return the broken clauses of the @propagatable contract of one type, as messages.

The function checks two clauses. Both come from the code that the macro emits:

  • Every field name is a keyword of the outer constructor. Each generated method builds the struct again with StructName(; field = …) over all fields, with or without a tag. One field that the keyword constructor does not name causes a MethodError at the first call of factory or port_opt_view.
  • Every @pprop field is a property of a prior result. The generated factory(x, pr::AbstractPriorResult, args...) reads getproperty(pr, :field). So a name that is not in prior_result_property_pool throws when a prior is threaded.

Each message has a suggestion from suggest_declared_key, so a field name with a typing error names the nearest valid name.

Algorithm

  1. Make msgs, an empty vector of strings.
  2. Read kws, the keywords of the outer constructors of T, with propagatable_keywords.
  3. For each field name of T that is not in kws, push a message that names the type, the field and the suggest_declared_key suggestion from kws.
  4. For each name of pprops that is not in pool, push a message that names the type, the field and the suggestion from pool.
  5. Return msgs.

The function collects every clause, and no clause stops the walk, so one call reports every violation of one type. An empty result means that the contract of the type holds.

Related

source
PortfolioOptimisers.check_propagatable_contracts — Function
check_propagatable_contracts()
check_propagatable_contracts(contracts)
check_propagatable_contracts(contracts, pool)

Check the @propagatable contract of every type in PROPAGATABLE_CONTRACTS.

The module calls it once, at its end. So the library enforces the contract of every generated method where the structs are declared, not at the first call of factory. The function collects the violations of every type and reports them together, because a run that stops at the first violation hides the others. See propagatable_contract_violations for the two clauses.

It throws an ArgumentError that lists every violation. So the package does not precompile with a type whose generated methods throw at their first use. A package that declares its own @propagatable types calls this function at the end of its own module to get the same check.

Algorithm

  1. Read pool, the property names that a prior result can carry, with prior_result_property_pool.
  2. Make msgs, an empty vector of strings.
  3. For each pair (T, pprops) of PROPAGATABLE_CONTRACTS, append the messages that propagatable_contract_violations reports for that type.
  4. msgs is not empty: throw an ArgumentError naming the count and listing every message.
  5. Return nothing.

Step 3 collects and never stops, so one run reports the violations of every registered type. propagatable_register! fills the registry at each declaration, so this function must run after the last declaration. For this reason, the module calls it at its end.

The registry and the pool are arguments, and each has the value of the module as its default. A caller can pass a registry of its own to read the message of a broken contract, and register no broken type.

Arguments

  • contracts: The pairs to check, each a type and the names of its @pprop fields.
  • pool: The property names that a prior result can carry.

Returns

  • nothing: Every pair of contracts satisfies the contract.

Related

source