Feature selector: private API

panel_field_keys(f::NumericPanelField) -> Vector{String}
panel_field_keys(f::CategoricalPanelField) -> VecStr
panel_field_keys(f::TensorPanelField) -> VecStr

Return the namespace a Feature Selector entry pairs with a Panel Field's name.

A numeric Panel Field contributes one column and has no second namespace, so it answers with an empty vector: every key paired with it is absent, and strict decides what that means. A categorical Panel Field answers with its levels and a tensor Panel Field with its labels, and in both the position of a key is the position of the column it selects.

Algorithm

The method that Julia selects is the algorithm.

Arguments

• f: The Panel Field.

Returns

• keys::VecStr: The levels, the labels, or an empty vector.

Related

• AbstractPanelField
• select_fields
• feature_labels
• VecStr

panel_value_columns!(cols::AbstractVector{Tuple{Int, Int, Symbol}}, k::Integer, f::NumericPanelField) -> nothing
panel_value_columns!(cols::AbstractVector{Tuple{Int, Int, Symbol}}, k::Integer, f::CategoricalPanelField) -> nothing
panel_value_columns!(cols::AbstractVector{Tuple{Int, Int, Symbol}}, k::Integer, f::TensorPanelField) -> nothing

Push every value column of one Panel Field onto a resolved Feature Selector, in column order.

This is what a bare field name expands to, and what an absent selector expands to for every Panel Field of the panel. The mask is not among them: a bare name is the values alone.

Algorithm

The method that Julia selects is the algorithm. A numeric Panel Field pushes its one column, and the other two push one column per key of panel_field_keys.

Arguments

• cols: The resolved columns so far, pushed onto in place.
• k: The Panel Field's position in the panel.
• f: The Panel Field.

Returns

• nothing. cols carries the result.

Related

• select_fields
• panel_field_keys
• AbstractPanelField

panel_key_column!(cols::AbstractVector{Tuple{Int, Int, Symbol}}, k::Integer, f::AbstractPanelField, key::AbstractString, strict::Bool) -> nothing

Push the one value column a Panel Field's level or label names onto a resolved Feature Selector.

Algorithm

1. Find key among the Panel Field's keys, from panel_field_keys.
2. Push the column it names when it is there.
3. Otherwise hand panel_selector_msg to strict_diagnostic, which throws under strict and warns and drops otherwise.

Arguments

• cols: The resolved columns so far, pushed onto in place.
• k: The Panel Field's position in the panel.
• f: The Panel Field.
• key: One of its levels or labels.
• Whether a sel entry naming a field, a level or a label the Asset Panel does not hold throws instead of warning and being dropped.

Returns

• nothing. cols carries the result.

Related

• select_fields
• panel_field_keys
• panel_selector_msg
• strict_diagnostic

panel_column_label(pnl::AssetPanel, col::Tuple{Int, Int, Symbol})

Return the Feature Selector entry that selects exactly one resolved column.

A label is a selector entry, not a rendered string. So the labels of a Feature Matrix are themselves a Feature Selector, and stacking the panel against them rebuilds the same matrix column for column.

Algorithm

1. Read the Panel Field the column belongs to.
2. An observed-mask column takes the name paired with :observed.
3. A value column of a Panel Field that contributes one takes the bare name.
4. Any other value column takes the name paired with its level or its label.

Arguments

• pnl: The Asset Panel.
• col: One resolved column, from select_fields.

Returns

• label: The Feature Selector entry naming that column.

Related

• feature_labels
• select_fields
• panel_field_keys

panel_field_value_column!(zc::AbstractArray, f::NumericPanelField, l::Integer, rows) -> nothing
panel_field_value_column!(zc::AbstractArray, f::CategoricalPanelField, l::Integer, rows) -> nothing
panel_field_value_column!(zc::AbstractArray, f::TensorPanelField, l::Integer, rows) -> nothing

Write one Panel Field's value column into a Feature Matrix under construction.

The column is cut to rows along its leading axis before it is written, so a stack of one observation reads one row of the Panel Field, and a lifted static Panel Field, whose values are a RepeatedLeading, is read once rather than once per observation. The cut is a selectdim along the leading axis, so Colon() is the whole field. A static panel's leading axis is its asset axis, so a static panel is only ever cut by Colon(): stacked_axes refuses every other value before a column is written.

Algorithm

The method that Julia selects is the algorithm. A numeric Panel Field writes its values, a categorical Panel Field writes the 0/1 indicator of one level, and a tensor Panel Field writes one label slice of its values, each cut to rows.

Arguments

• zc: The column of the Feature Matrix, a view over the stacked observation rows and the panel's asset axis.
• f: The Panel Field.
• l: The position of the level or the label, and 0 for a numeric Panel Field.
• rows: The observation rows a time-varying Asset Panel stacks, Colon() for every row. A static panel has no observation axis and refuses any other value.

Returns

• nothing. zc carries the result.

Related

• feature_matrix
• stacked_axes
• select_fields
• AbstractPanelField
• RepeatedLeading

panel_field_observed_column!(zc::AbstractArray, f::NumericPanelField, rows) -> nothing
panel_field_observed_column!(zc::AbstractArray, f::CategoricalPanelField, rows) -> nothing
panel_field_observed_column!(zc::AbstractArray, f::TensorPanelField, rows) -> nothing

Write one Panel Field's observed mask into a Feature Matrix under construction, as a 0/1 column.

A Panel Field contributes one mask column, whatever its kind and however many value columns it contributes. The column says whether the cell that Panel Field describes was observed for that asset, so a tensor Panel Field, whose mask carries a label axis of its own, holds where every label of that asset was observed.

A Panel Field that carries no mask gives a column of ones: omsk === nothing means that the Panel Field cannot blank, so every cell was observed. A mask is cut to rows along its leading axis before it is written, as panel_field_value_column! cuts a value column; a column of ones is the same whatever rows holds.

Algorithm

The method that Julia selects is the algorithm.

1. Write ones when the Panel Field carries no mask.
2. A numeric or a categorical Panel Field writes its mask, cut to rows.
3. A tensor Panel Field cuts its mask to rows, reduces it over the label axis with all, and writes that.

Arguments

• zc: The column of the Feature Matrix, a view over the stacked observation rows and the panel's asset axis.
• f: The Panel Field.
• rows: The observation rows a time-varying Asset Panel stacks, Colon() for every row. A static panel has no observation axis and refuses any other value.

Returns

• nothing. zc carries the result.

Related

• feature_matrix
• panel_field_value_column!
• select_fields
• AbstractPanelField

stacked_axes(ax::Tuple, rows) -> Tuple

Cut the observation axis of an Asset Panel's axes to the rows a Feature Matrix stacks.

A time-varying panel is stated on (observations, assets), and its Feature Matrix stacks the observation rows rows names, so the matrix is stated on (length(rows), assets). Colon() is every row, and answers the axes unchanged, on a static panel too. A static panel is stated on (assets,) alone: it has no observation axis to cut, so any other rows is refused rather than cutting the asset axis by mistake.

Algorithm

The method that Julia selects is the algorithm.

1. Colon(): answer ax unchanged.
2. A vector of row positions: check that ax carries an observation axis and that every position lies on it, and answer (length(rows), ax[2]).

Arguments

• ax: The panel's axes, as panel_axes reads them.
• rows: The observation rows a time-varying Asset Panel stacks, Colon() for every row. A static panel has no observation axis and refuses any other value.

Validation

• length(ax) == 2 when rows is not Colon(). Raises an ArgumentError.
• Every entry of rows lies in 1:ax[1]. Raises an ArgumentError.

Returns

• ax::Tuple: (assets,) for a static panel, and (length(rows), assets) for a time-varying one.

Related

• feature_matrix
• panel_axes
• collapse_rows

select_fields(pnl::AssetPanel, sel, strict::Bool) -> Vector{Tuple{Int, Int, Symbol}}

Resolve a Feature Selector against an AssetPanel, and return the columns it names.

This is the one resolution. feature_matrix and feature_labels both read it, so the matrix and its labels cannot disagree about what was selected. It is unexported: a caller reads the labels, not the positions.

A resolved column is the triple (k, l, part). k is the Panel Field's position in the panel. part is :vals or :observed. l is the position of the level or the label within the Panel Field, and 0 where the Panel Field contributes one column of that part: a numeric Panel Field's value column, and every Panel Field's observed-mask column.

The order of sel is the column order, so a caller decides it. A nothing selector takes the panel's own order, and stacks each Panel Field's values without its mask.

Algorithm

1. Check that the panel holds a Panel Field. A panel with none carries no feature data at all, so it is refused here, where the cause is still known, rather than downstream where only the empty matrix is visible.
2. Validate sel with assert_feature_selector.
3. sel is nothing: push every Panel Field's value columns in panel order, and return.
4. Otherwise resolve each entry with select_fields_push!, in the order sel writes them.
5. Check that no two entries resolved to one column.
6. Check that the selection is not empty, which happens when every entry resolved against nothing and was dropped.

Arguments

• pnl: The Asset Panel.
• Feature Selector naming the Panel Fields the Feature Matrix stacks, or nothing to stack every field's values. An entry is a field name, a field paired with the levels or labels it keeps, a field paired with one level or label, or a field paired with :observed. The vector order is the column order.
• Whether a sel entry naming a field, a level or a label the Asset Panel does not hold throws instead of warning and being dropped.

Validation

• !isempty(pnl.pf). Raises an IsEmptyError naming the panel, the Feature Matrix and asset_panel.
• sel is well formed. See assert_feature_selector.
• No two entries resolve to one column. Raises an ArgumentError.
• The selection is not empty. Raises an IsEmptyError.

Returns

• cols::Vector{Tuple{Int, Int, Symbol}}: One triple per column, in column order.

Related

• AssetPanel
• feature_matrix
• feature_labels
• select_fields_push!
• assert_feature_selector

select_fields_push!(cols::AbstractVector{Tuple{Int, Int, Symbol}}, pnl::AssetPanel, entry, pool, strict::Bool) -> nothing

Resolve one Feature Selector entry against an AssetPanel's Panel Fields.

Algorithm

1. Read the Panel Field name off the entry: a paired entry names it first, and a bare entry is the name.
2. Find the Panel Field. Hand panel_selector_msg to strict_diagnostic and return when the panel holds none.
3. A bare name expands to the Panel Field's value columns, through panel_value_columns!.
4. A name paired with a Symbol claims the Panel Field's one observed-mask column.
5. A name paired with one key claims that key's column, through panel_key_column!.
6. A name paired with a vector of keys claims one column per key, in the order the vector writes them.

Arguments

• cols: The resolved columns so far, pushed onto in place.
• pnl: The Asset Panel.
• entry: One entry of the Feature Selector.
• pool: The panel's Panel Field names, the suggestion pool of the first namespace.
• Whether a sel entry naming a field, a level or a label the Asset Panel does not hold throws instead of warning and being dropped.

Returns

• nothing. cols carries the result.

Related

• select_fields
• panel_value_columns!
• panel_key_column!
• panel_selector_msg
• strict_diagnostic

panel_selector_msg(name, pool::VecStr) -> String
panel_selector_msg(f::NumericPanelField, key, keys::VecStr) -> String
panel_selector_msg(f::AbstractPanelField, key, keys::VecStr) -> String

Build the diagnostic a Feature Selector entry that resolves against nothing carries.

The messages name the two namespaces an entry resolves in. The first is the panel's Panel Field names, the second one Panel Field's levels or labels. Each hands its own suggestion pool to did_you_mean, so a misspelling is answered against the names it could have meant and not against every column of the panel. A numeric Panel Field has no second namespace, so a key paired with it is answered by naming the two entry forms the field does take rather than by an empty pool.

Algorithm

1. Name the entry that resolved against nothing, and the namespace it was resolved in.
2. Append the nearest match from the pool with did_you_mean, where there is a pool.

Arguments

• name: The Panel Field name that resolved against nothing.
• pool: The Panel Field names of the panel. See VecStr.
• f: The Panel Field whose key resolved against nothing.
• key: The level or label that resolved against nothing.
• keys: The Panel Field's levels or labels. See VecStr.

Returns

• msg::String: The diagnostic.

Related

• select_fields
• did_you_mean
• strict_diagnostic
• VecStr

assert_feature_selector(sel::Nothing) -> nothing
assert_feature_selector(sel::AbstractVector) -> nothing

Validate a Feature Selector: a non-empty vector of distinct entries, each of the four admitted forms.

An empty sel is refused rather than read as "every Panel Field": nothing already says that, and a selection that silently widens to the whole panel is the failure this selector exists to remove. A repeated entry is refused because it doubles that column's contribution to every distance.

Algorithm

The method that Julia selects decides whether there is anything to check.

1. sel is nothing: it stacks every Panel Field's values, so there is nothing to check.
2. sel is a vector: check that it is non-empty, that it repeats no entry, and that each entry takes an admitted form, with assert_selector_entry.

Arguments

• Feature Selector naming the Panel Fields the Feature Matrix stacks, or nothing to stack every field's values. An entry is a field name, a field paired with the levels or labels it keeps, a field paired with one level or label, or a field paired with :observed. The vector order is the column order.

Validation

• !isempty(sel). Raises an IsEmptyError.
• allunique(sel). Raises an ArgumentError.
• Each entry takes an admitted form. See assert_selector_entry.

Returns

• nothing.

Related

• select_fields
• assert_selector_entry
• FeatureDistance
• IsEmptyError

assert_selector_entry(entry) -> nothing

Check that one Feature Selector entry takes one of the four forms the grammar admits.

The four forms are a Panel Field name, a name paired with the levels or labels it keeps, a name paired with one level or label, and a name paired with :observed. No entry is a column position: after the Asset Panel became the one carrier every field, level and label carries a name, so an integer has nothing to index.

A name is refused when it is empty, and a paired vector is refused when it is empty, holds an empty key, or repeats a key. Each of those resolves to no column or to a doubled column, which is the failure the selector exists to remove.

Algorithm

1. A string: check that it is not empty.
2. A pair whose first element is a non-empty string: check its second element. A Symbol is :observed, a string is non-empty, and a vector of strings is non-empty, holds no empty key and repeats none.
3. Anything else is refused.

Arguments

• entry: One entry of a Feature Selector.

Validation

• entry takes one of the four forms. Raises an ArgumentError.

Returns

• nothing.

Related

• assert_feature_selector
• select_fields