Propagatable: private API
Utility functions
PortfolioOptimisers.PROP_TAG_NAMES — Constant
PROP_TAG_NAMESThe propagation tag set of @propagatable, as data.
One entry per field tag. The code that recognises a tag comes from this tuple, and no tag is written out in that code. The macro names (PROP_TAG_MACRO_NAMES), the lookup (prop_tag), the test (is_prop_tag_call), the peeler (peel_prop_tags) and the parser (propagatable_parse_body) all read it. A new tag is one row here, one branch in prop_tag_expr, one entry in a channel of PROP_TAG_CHANNELS and one stub macro. check_prop_tag_macros refuses to load the module when a row has no stub macro, no channel or no branch.
Related
PortfolioOptimisers.PROP_TAG_MACRO_NAMES — Constant
PROP_TAG_MACRO_NAMESThe macro name of every tag of PROP_TAG_NAMES, in the same order.
The tuple comes from the tag names, so a row of the table has no second spelling. prop_tag matches the head of a :macrocall against this tuple.
Algorithm
- Map each tag of
PROP_TAG_NAMEStoSymbol("@", tag), which is the name that Julia gives the macro of that tag.
The map keeps the order, so the $k$-th entry here is the macro name of the $k$-th tag there. prop_tag and check_prop_tag_macros both walk the two tuples together, and they need this shared order to pair each tag with its macro name.
Related
PortfolioOptimisers.PROP_TAG_CHANNELS — Constant
PROP_TAG_CHANNELSThe propagation channels of @propagatable, with their tag precedence as data.
One entry per generated method. Each entry has two tuples of tags:
gate: the tags that make@propagatableemit the method.precedence: the order in which the channel reads the tags of a field. The first tag of this tuple that the field carries gives the transform of the field. The channel ignores the other tags of that field.
The factory channel prefers @fprop to @wprop. The prior channel prefers @pprop, then @cprop, then @wprop, then @fprop, so @pprop wins over @fprop on one field. prop_channel_pairs reads the precedence for every channel.
A tag means what its channel says it means. The obs channel reads the same @wprop and @fprop tags as factory, and it gives them different transforms. factory replaces a @wprop field with an incoming ObsWeights value, and obs indexes the value that the field holds. For this reason, prop_tag_expr takes the channel as well as the tag. A weights field joins obs_weights_view through its @wprop tag, and it needs no second tag.
The obs channel opens on a @wprop, a @fprop or an @oprop field. Every reader in a fit reads the observations of the same sample, so a composed child that holds weights deep in its tree must take the cut too, see shared_series_estimator. A struct whose only tags are @fprop therefore gets an obs_weights_view method that recurses into its children. @oprop marks a child that the obs channel alone reads, for a child that factory must not reach.
Related
PortfolioOptimisers.PROPAGATABLE_CONTRACTS — Constant
PROPAGATABLE_CONTRACTSEvery type declared with @propagatable, with the names of its @pprop fields.
The macro adds one entry immediately after the struct that it declares. When this module finishes its load, the list holds every type of this package, and check_propagatable_contracts reads it at that point. An external package adds its types when it loads, after that check, and it must call check_propagatable_contracts at the end of its own module.
Related
PortfolioOptimisers.prop_tag — Function
prop_tag(x) -> Union{Nothing, Symbol}
Return the tag of PROP_TAG_NAMES that x names, or nothing.
x is the head of a :macrocall node. It is a bare Symbol in a struct body written by hand, and a GlobalRef after another macro expanded around it, so the function resolves both. A name outside PROP_TAG_MACRO_NAMES gives nothing. The callers use this to tell a tag from any other macro. A name never resolves to a tag that it does not name.
Algorithm
- Read
namefromx. AGlobalRefgives itsnamefield, aSymbolgives itself, and any other value returnsnothingat once. - Walk
PROP_TAG_NAMESandPROP_TAG_MACRO_NAMEStogether. Return the tag whose macro name is identical toname. - No macro name matches: return
nothing.
The comparison is === on a Symbol, so only an identical name matches. A macro with a similar name never matches.
Arguments
x: The first argument of a:macrocallexpression.
Returns
tag::Symbol: The tag name, without the@.nothing: Ifxnames no tag.
Related
PortfolioOptimisers.is_prop_tag_call — Function
is_prop_tag_call(x) -> Bool
Return true if x is a macro call to any tag of PROP_TAG_NAMES.
peel_prop_tags and propagatable_parse_body use it to find the tagged fields of a struct body.
Algorithm
- Return
truewhen all three hold:xis anExpr, its head is:macrocall, andprop_tagof its first argument is notnothing. - Return
falseotherwise.
Arguments
x: Any expression appearing in a struct body.
Related
PortfolioOptimisers.prop_tag_expr — Function
prop_tag_expr(
channel::Symbol,
tag::Symbol,
fname::Symbol,
xf,
mod::Module,
thread
) -> Expr
Return the expression that a tag substitutes for one field, inside a generated method.
This function gives the field transform of each tag. It has one branch per tag of PROP_TAG_NAMES, and prop_channel_pairs calls it for every channel. A tag of the table with no branch here raises an error, so it cannot take the transform of another tag without a sign.
Algorithm
The function reads the channel first, and then the tag, because one tag has two transforms.
channelis:obs:tagis:wprop: returnobs_weights_index(xf, thread...). The field is the weights, so the method indexes it at the selected observations. An index keeps theAbstractWeightssubtype, and a view does not.obs_weights_indexstates the rule for a mask of the full sample.tagis:opropor:fprop: returnobs_weights_view(xf, thread...). The field is a composed child, so the verb recurses into it.
channelis any other channel:tagis:fprop: returnfactory(xf, thread..., args...; kwargs...). The traversal owns the identity on a leaf, so a field that holds data ornothingcomes back with no change.tagis:vprop: returnview_child(xf, thread..., args...), which callsport_opt_viewon every value but a precomputed optimisation result. This channel forwards no keywords.tagis:pprop: returnsel(xf, getproperty(pr, fname)). The prior result gives the property of the same name, so the field name is an argument.tagis:cprop: returnsel(xf, _ctx(args...)), which reads the context from the threaded arguments and not from the prior.tagis:wprop: return_wprop(xf, args...; kwargs...), which replaces the field with an incomingObsWeights.
- No branch matched: raise an error that names the tag and the channel, and asks for a branch here.
Steps 1.1 and 2.5 are the same tag with two transforms, so the channel decides what @wprop means. Every emitted name is qualified with mod, because the expansion is escaped into the module of the caller.
Arguments
tag::Symbol: A tag ofPROP_TAG_NAMES.fname::Symbol: The field name.@ppropneeds it to name the prior property.xf: The expression that reads the field from the incoming struct.mod::Module: The module that defines@propagatable. Every emitted name is qualified with it, because the expansion is escaped into the module of the caller.thread: The positional arguments that the channel threads beforeargs....
Returns
expr::Expr: The value of the field in the generated constructor call.
Related
PortfolioOptimisers.prop_channel_active — Function
prop_channel_active(
channel::Symbol,
tagged::AbstractDict
) -> Any
Return true if a channel of PROP_TAG_CHANNELS must emit a method.
A channel is active when at least one field carries a tag of the gate of the channel.
Algorithm
- Read the
gatetuple of the channel fromPROP_TAG_CHANNELS. - Return
truewhen at least one tag ofgatehas a non-empty entry intagged, andfalseotherwise.
This function does not read the precedence tuple. So a tag that a channel reads, but does not gate on, never makes that channel emit a method alone. The prior channel reads @wprop and @fprop, but it gates on @pprop and @cprop, so a type with @fprop alone gets no prior method.
Arguments
channel::Symbol: A channel name ofPROP_TAG_CHANNELS.tagged::AbstractDict: Map from a tag name to the names of the fields that carry it, frompropagatable_parse_body.
Related
PortfolioOptimisers.prop_channel_pairs — Function
prop_channel_pairs(
channel::Symbol,
tagged::AbstractDict,
all_fields::AbstractVector{Symbol},
obj::Symbol,
mod::Module,
thread
) -> Vector{Any}
Return the keyword pairs of the constructor call that one channel generates.
Every declared field gets one pair, in declaration order. The function reads the tags of a field in the precedence order of the channel. The first match gives the value through prop_tag_expr. A field with no tag of the channel keeps its value.
Algorithm
- Read the
precedencetuple of the channel fromPROP_TAG_CHANNELS. - For each field name
fnameofall_fields, in declaration order:- Build
xf, the expressionobj.fnamethat reads the field from the incoming struct. - Find
idx, the position of the first tag ofprecedencethatfnamecarries. - When
idxisnothing, the field carries no tag of this channel, and the value isxf. - Otherwise, the value is
prop_tag_exprof that tag, in this channel. - Push
Expr(:kw, fname, value)ontopairs.
- Build
- Return
pairs.
In step 2.2, the precedence decides the transform of one field. A field with @pprop and @fprop takes the @pprop transform on the prior channel and the @fprop transform on the factory channel, because the two channels put the tags in different orders.
Arguments
channel::Symbol: A channel name ofPROP_TAG_CHANNELS.tagged::AbstractDict: Map from a tag name to the names of the fields that carry it.all_fields::AbstractVector{Symbol}: Every declared field, in declaration order.obj::Symbol: The struct from which the generated method reads the fields.mod::Module: The module that defines@propagatable.thread: The positional arguments that the channel threads beforeargs....
Returns
pairs::Vector{Any}: OneExpr(:kw, field, value)per declared field.
Related
PortfolioOptimisers.check_prop_tag_macros — Function
check_prop_tag_macros()
check_prop_tag_macros(tags)
check_prop_tag_macros(tags, macro_names)
check_prop_tag_macros(tags, macro_names, channels)
check_prop_tag_macros(
tags,
macro_names,
channels,
mod::Module
)
Check that every tag of PROP_TAG_NAMES is complete.
A tag row is complete when it has a stub macro, a channel of PROP_TAG_CHANNELS that names it, and a field transform in prop_tag_expr for every channel that names it. A row without one of the three is a tag that parses but never propagates, and the table exists to stop that failure. A tag means what its channel says it means, so the function probes each channel. That probe finds a tag that a second channel names without a transform there. The function collects all the violations and reports them together.
The module calls this function once, at its end. It throws an ArgumentError that lists every violation, so the package does not precompile with a tag that never propagates.
The three tables are arguments, and each one has the table of the module as its default. The tables of the module are complete, so the call of the module never reports a violation. A caller can pass a table of its own to make each of the three clauses fail and read the message that it gives.
Algorithm
- Make
violations, an empty vector of strings. - For each tag of
PROP_TAG_NAMES, with its macro name fromPROP_TAG_MACRO_NAMES:- The macro name is not defined in
mod: push a message that the tag declares no stub macro. - The tag is in the
precedenceof no channel ofchannels: push a message that the tag is in no channel. - For each channel whose
precedencenames the tag, callprop_tag_exprwith the probe name:probe. When that call raises an error, push a message that names the tag and the channel.
- The macro name is not defined in
violationsis not empty: throw anArgumentErrorthat lists all of them.- Return
nothing.
Step 2.3 probes each channel that names the tag, not the tag alone. This probe finds a tag that a second channel names with no transform there, and a probe of the tag alone does not.
Arguments
tags: The tag names to check.macro_names: The stub macro name of each tag, in the order oftags.channels: The channel table whoseprecedencetuples the function reads.mod::Module: The module in which the function finds the stub macros. It also qualifies the names thatprop_tag_expremits.
Returns
nothing: Every row oftagsis complete.
Related
PortfolioOptimisers.is_doc_macro — Function
is_doc_macro(x) -> Union{Missing, Bool}
Return true if x is a reference to Julia's @doc macro (bare Symbol or GlobalRef).
propagatable_parse_body uses it to find the fields of a struct body that have a docstring.
Algorithm
- Return
truewhenxis aGlobalRefwhosenameisSymbol("@doc"). - Return
truewhenxis equal toSymbol("@doc"). - Return
falseotherwise.
The function needs both spellings for the same reason as prop_tag. A struct body written by hand has the bare Symbol, and a struct body that another macro expanded has the GlobalRef.
Related
PortfolioOptimisers._ctx — Function
_ctx(args...)Find the one optimiser value that a prior factory call threads, a solver of type Slv_VecSlv, in the variadic tail of the call, and return nothing when the tail holds none. The @cprop tag emits this call as the source argument of sel. The compiler unrolls the scan of the tuple, so the call is type-stable and does not allocate.
Related
PortfolioOptimisers.extract_field_name — Function
extract_field_name(expr) -> Symbol
Extract the field name Symbol from a bare field or field::Type expression.
The function raises an error that names expr when expr is neither a bare Symbol nor a field::Type annotation. Only those two forms are valid after a tag such as @fprop.
Algorithm
expris aSymbol: return it.expris anExprwhose head is:(::): return its first argument, which is the field name.expris any other expression: raise an error that names the expression.
Step 3 is the difference from try_field_name, which returns nothing in the same case. A tag states that the node is a field. So a node that is not a field is a defect in the struct body, and the function does not skip it.
Arguments
expr: ASymbol, anExprwith head:(::), or any other expression, which raises an error.
Returns
name::Symbol: The field name.
Related
PortfolioOptimisers.propagatable_find_struct — Function
propagatable_find_struct(
expr
) -> Union{Tuple{Expr, typeof(identity)}, Tuple{Expr, Union{PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, typeof(identity)}, PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, PortfolioOptimisers.var"#propagatable_find_struct##0#propagatable_find_struct##1"{Vector{Any}, rebuild}} where rebuild}}}
Find the innermost :struct node of a chain of macro calls, through a recursion.
The function returns (struct_node, rebuild_fn). rebuild_fn(new_struct) builds the original macro chain again, with new_struct in place of the original struct. So @propagatable can put a changed struct definition back inside any wrapping macro, such as @concrete.
Algorithm
expris not anExpr: raise an error that names its type.exprhas head:struct: returnexprandidentity, which rebuilds nothing.exprhas head:macrocall: takeinner, its last argument, and call this function again on it. That call givesstruct_nodeandrebuild. Readprefix, every argument ofexprexcept the last. Returnstruct_nodeand the functions -> Expr(:macrocall, prefix..., rebuild(s)).exprhas any other head: raise an error that names the head.
Step 3 builds the chain again from the inside out. A struct inside several macros comes back inside the same macros, in the same order, with the same arguments. The prefix holds the arguments of the macro and its LineNumberNode, so the rebuilt call loses nothing.
Arguments
expr: A:structexpression, or a:macrocallexpression around one.
Returns
struct_node::Expr: The innermost:structexpression.rebuild_fn::Function: A function that takes a replacement:structand returns the full macro chain, with the replacement in place of the original.
Related
PortfolioOptimisers.propagatable_bare_name — Function
propagatable_bare_name(n) -> Symbol
Get the plain struct name, as a Symbol, from a name expression that can have type parameters and a supertype.
The function accepts the forms Name, Name{T, ...} and Name{T, ...} <: SuperType. It peels the :curly and :<: nodes, through a recursion, until it gets a bare Symbol.
Algorithm
nis aSymbol: return it.nhas head:curly: call this function again on its first argument, which drops the type parameters.nhas head:<:: call this function again on its first argument, which drops the supertype.nis any other expression: raise an error that names the expression.
Steps 2 and 3 work together, so Name{T} <: Super loses the supertype first and then the parameters.
Arguments
n: ASymbol, or anExprwith head:curlyor:<:.
Returns
name::Symbol: The plain struct name.
Related
PortfolioOptimisers.try_field_name — Function
try_field_name(expr) -> Symbol
Return the field name, as a Symbol, of a plain field declaration, or nothing for a node that is not a field.
The function accepts a bare Symbol field and a field::Type annotation. It returns nothing for a LineNumberNode, an inner constructor and every other expression that does not declare one named field.
Algorithm
expris aSymbol: return it.exprhas head:(::)and its first argument is aSymbol: return that argument.expris any other expression: returnnothing.
Step 2 tests the first argument as well as the head, and extract_field_name does not. So a node such as a.b::Int, whose first argument is not a Symbol, gives nothing here.
Arguments
expr: Any expression appearing in a struct body.
Returns
name::Symbol: The field name, ifexpris a plain field declaration.nothing: Ifexpris not a plain field declaration.
Related
PortfolioOptimisers.peel_prop_tags — Function
peel_prop_tags(expr) -> Tuple{Set{Symbol}, Any}
Peel the tag macrocalls off a field expression, and record the tags.
A field can have a stack of tags in any order, such as @pprop @fprop field. The stack parses as nested :macrocall nodes. This function removes all of them and returns the bare field expression. It finds each tag with prop_tag, so a macro that is not a tag stops the peel.
Algorithm
- Make
tags, an emptySet{Symbol}. - While
is_prop_tag_callofexprholds, pushprop_tagof its first argument ontotags, and replaceexprwith its last argument, which is the expression the tag wraps. - Return
tagsand the peeledexpr.
tags is a set, so a tag written two times on one field is recorded once. The loop stops at the first node that is not a tag call, and a macro between two tags that is not a tag hides the tags below it.
Arguments
expr: A field expression, with or without tag macrocalls around it.
Returns
tags::Set{Symbol}: The tags ofPROP_TAG_NAMESthatexprcarries.stripped: The field expression with all tags removed.
Related
PortfolioOptimisers.propagatable_parse_body — Function
propagatable_parse_body(
body
) -> Tuple{Dict{Symbol, Vector{Symbol}}, Vector{Symbol}, Expr}
Walk a struct body. Collect the names of the tagged fields and of all fields, and remove the tags from the body.
The function accepts a bare tagged field such as @fprop field, a stack of tags in any order such as @pprop @fprop field, and a field with a docstring such as "doc" \n @fprop field. It keeps a node that is not a field, such as a line number or an inner constructor, with no change. The tags are the rows of PROP_TAG_NAMES, so a new tag needs no change here.
Algorithm
- Make
tagged, one empty vector per tag ofPROP_TAG_NAMES. Makeall_fields, an empty vector, andnew_args, an empty vector for the stripped body. - For each node
argof the struct body, in declaration order, take one of three branches:argis a@docmacrocall, which is how a field with a docstring parses. Peel the tags offinner, its last argument.- The field carries at least one tag: record the field name under each of its tags and in
all_fields. Then push a rebuilt@docnode whose last argument is the stripped field. - The field carries no tag: record its name in
all_fieldswhentry_field_namefinds one, and pushargwith no change.
- The field carries at least one tag: record the field name under each of its tags and in
argis a tag macrocall with no docstring: peel the tags, record the field name under each of them and inall_fields, and push the stripped field expression.argis any other node, such as aLineNumberNode, an untagged field or an inner constructor: record its name inall_fieldswhentry_field_namefinds one, and pushargwith no change.
- Return
tagged,all_fields, and the new body as one:blockexpression.
all_fields holds every declared field, with or without a tag. With it, the generated constructor call names every keyword. The returned body has no tag, so the wrapped macros and Julia never see one.
Arguments
body::Expr: The:blockexpression of the struct body.
Returns
tagged::Dict{Symbol, Vector{Symbol}}: One entry per tag ofPROP_TAG_NAMES, which holds the names of the fields that carry the tag, in declaration order.all_fields::Vector{Symbol}: The names of every declared field, with or without a tag.new_body::Expr: The struct body with no tags.
Related
PortfolioOptimisers.propagatable_register! — Function
propagatable_register!(
T::Type,
pprops::Tuple{Vararg{Symbol}}
)
Record the type T and the names of its @pprop fields in PROPAGATABLE_CONTRACTS.
@propagatable calls it at the declaration. The function only records. The outer keyword constructor is written below the struct, so it does not exist yet, and the function cannot check it. check_propagatable_contracts does the check when the module is complete.
Algorithm
- Push the pair
(T, pprops)ontoPROPAGATABLE_CONTRACTS. - Return
nothing.
T has @nospecialize, so one method serves every registered type, and the registration costs no compilation.
Related
PortfolioOptimisers.propagatable_keywords — Function
propagatable_keywords(T::Type) -> Vector{Symbol}
Return the keyword names that the outer constructors of T accept, as the union over its methods.
The function drops a kwargs... slurp and does not count it. A slurp accepts field = value and then discards it without an error. The contract check exists to find that failure, so a slurp must not satisfy the contract.
Algorithm
- Make
kws, an empty vector of symbols. - For each method
mof the constructorT, appendBase.kwarg_decl(m)tokws. The union is over every outer constructor, so a keyword that any one of them names counts. - Remove the repeats from
kws. - Remove every name whose string ends in
..., becauseBase.kwarg_declgives a slurp in that form. - Return
kws.
Step 4 is the reason for the function. A constructor with kwargs... gives the slurp as a keyword name. If the function counted it, every field would satisfy the contract.
Related
PortfolioOptimisers.propagatable_contract_violations — Function
propagatable_contract_violations(
T::Type,
pprops,
pool
) -> Vector{String}
Return the broken clauses of the @propagatable contract of one type, as messages.
The function checks two clauses. Both come from the code that the macro emits:
- Every field name is a keyword of the outer constructor. Each generated method builds the struct again with
StructName(; field = …)over all fields, with or without a tag. One field that the keyword constructor does not name causes aMethodErrorat the first call offactoryorport_opt_view. - Every
@ppropfield is a property of a prior result. The generatedfactory(x, pr::AbstractPriorResult, args...)readsgetproperty(pr, :field). So a name that is not inprior_result_property_poolthrows when a prior is threaded.
Each message has a suggestion from suggest_declared_key, so a field name with a typing error names the nearest valid name.
Algorithm
- Make
msgs, an empty vector of strings. - Read
kws, the keywords of the outer constructors ofT, withpropagatable_keywords. - For each field name of
Tthat is not inkws, push a message that names the type, the field and thesuggest_declared_keysuggestion fromkws. - For each name of
ppropsthat is not inpool, push a message that names the type, the field and the suggestion frompool. - Return
msgs.
The function collects every clause, and no clause stops the walk, so one call reports every violation of one type. An empty result means that the contract of the type holds.
Related
PortfolioOptimisers.check_propagatable_contracts — Function
check_propagatable_contracts()
check_propagatable_contracts(contracts)
check_propagatable_contracts(contracts, pool)
Check the @propagatable contract of every type in PROPAGATABLE_CONTRACTS.
The module calls it once, at its end. So the library enforces the contract of every generated method where the structs are declared, not at the first call of factory. The function collects the violations of every type and reports them together, because a run that stops at the first violation hides the others. See propagatable_contract_violations for the two clauses.
It throws an ArgumentError that lists every violation. So the package does not precompile with a type whose generated methods throw at their first use. A package that declares its own @propagatable types calls this function at the end of its own module to get the same check.
Algorithm
- Read
pool, the property names that a prior result can carry, withprior_result_property_pool. - Make
msgs, an empty vector of strings. - For each pair
(T, pprops)ofPROPAGATABLE_CONTRACTS, append the messages thatpropagatable_contract_violationsreports for that type. msgsis not empty: throw anArgumentErrornaming the count and listing every message.- Return
nothing.
Step 3 collects and never stops, so one run reports the violations of every registered type. propagatable_register! fills the registry at each declaration, so this function must run after the last declaration. For this reason, the module calls it at its end.
The registry and the pool are arguments, and each has the value of the module as its default. A caller can pass a registry of its own to read the message of a broken contract, and register no broken type.
Arguments
contracts: The pairs to check, each a type and the names of its@ppropfields.pool: The property names that a prior result can carry.
Returns
nothing: Every pair ofcontractssatisfies the contract.
Related