Empirical Prior

PortfolioOptimisers.EmpiricalPrior — Type

struct EmpiricalPrior{__T_ce, __T_me, __T_horizon, __T_fill_limit, __T_max_scenarios, __T_cache} <: AbstractLowOrderPriorEstimator_A

Empirical prior estimator for asset returns.

EmpiricalPrior is a low order prior estimator that computes the mean and covariance of asset returns using empirical (sample-based) statistics. It supports custom expected returns and covariance estimators, as well as an optional investment horizon for log-normalisation and scaling.

Fields

ce: Covariance estimator.

me: Expected returns estimator.

horizon: Optional investment horizon for log-normalising returns. If nothing, returns are not adjusted.

fill_limit: Share of an investable column's own observations a scenario_fill may write in silence, tested against the worst column. If nothing, resolve_fill_limit derives it at the fit as 1 - min_coverage over the arms that state a coverage floor, which never fires; where no arm states one, no share passes in silence and every fill is named.

max_scenarios: Optional cap on the number of observations the Prior Result carries as scenarios. nothing carries every observation the fit read. A cap truncates X to the last max_scenarios rows and leaves mu and sigma fitted over every observation, so it bounds the memory a scenario-based measure reads and changes no moment. It is the same knob in batch and online, because it is a property of the result rather than of the fold. When the cap cuts, the result states the number of observations its moments were fitted over in ens, so a consumer that prices a sample size — an uncertainty set, a calibration rule — reads that count rather than the rows carried; a bootstrap, which can resample nothing but the rows carried, reads the cap.

cache: Optional partial-fit state. It is nothing until partial_fit! writes one, and the estimator's read-out verb reads it when the caller gives no data matrix. Each propagation channel does one thing with it: factory carries it unchanged, because a factory call resolves configuration rather than the sample; port_opt_view slices it to the selected assets by index copy, so the viewed estimator answers over those assets alone; and obs_weights_view drops it, because no slice of a state exists on the observation axis. A family whose state has no exact asset slice drops it on both axes and names the reason.

Constructors

EmpiricalPrior(;    ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),    me::AbstractExpectedReturnsEstimator = SimpleExpectedReturns(),    horizon::Option{<:Number} = nothing,    fill_limit::Option{<:Real} = nothing,    max_scenarios::Option{<:Integer} = nothing,    cache::Option{<:AbstractPartialFitState} = nothing) -> EmpiricalPrior

Keywords correspond to the struct's fields.

Validation

If horizon is not nothing, horizon > 0.
If fill_limit is not nothing, 0 < fill_limit <= 1. Zero is not a value, because nothing already means that no fill passes in silence. The fit refuses a value looser than 1 - min_coverage over the arms that state a coverage floor, because such a value could never fire.
If max_scenarios is not nothing, max_scenarios > 0.

The incremental fit

The empirical prior folds and carries. partial_fit! forwards each observation to me and to ce, both of which fold exactly, and appends the row to a PriorCarryState of its own; the one-argument prior reads mu and sigma off the two folded arms and X off the carried rows. The step is therefore quadratic in the number of assets and independent of the number of observations folded, and the buffer is memory rather than arithmetic: it exists because LowOrderPrior carries X for the scenario risk measures, not because anything needs refitting.

Two caps, two names

max_scenarios and Online's max_history are different knobs, and both are wanted.

EmpiricalPrior(; max_scenarios = w) fits mu and sigma over every observation and carries the last w rows as X. It is a property of the result, so it reads the same in batch and online, and it has no batch equal to assert: the moments and the scenarios come from different windows by construction.
Online(EmpiricalPrior(); max_history = w) windows the whole fit: the estimator refits from the buffer, so it equals prior(pe, X[end - w + 1:end, :]) exactly.

Both set, they nest: the moments come from the last max_history observations and X from the last max_scenarios of those. A max_scenarios at or above the number of observations carried is a no-op.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

ce: Recursively updated via factory.
me: Recursively updated via factory.
cache: Carried unchanged via factory.

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

ce: Recursively viewed via port_opt_view.
me: Recursively viewed via port_opt_view.
cache: Sliced to the selected assets via port_opt_view.

Observation view parameters

When obs_weights_view is called on this type:

cache: Dropped via obs_weights_view, because no slice of a state exists on the observation axis.

Examples

julia> EmpiricalPrior()EmpiricalPrior          ce ┼ PortfolioOptimisersCovariance             │   ce ┼ Covariance             │      │    me ┼ SimpleExpectedReturns             │      │       │   w ┴ nothing             │      │    ce ┼ GeneralCovariance             │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)             │      │       │    w ┴ nothing             │      │   alg ┼ FullMoment()             │      │     w ┴ nothing             │   mp ┼ MatrixProcessing             │      │     pdm ┼ Posdef             │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton             │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()             │      │      dn ┼ nothing             │      │      dt ┼ nothing             │      │     alg ┼ nothing             │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)          me ┼ SimpleExpectedReturns             │   w ┴ nothing     horizon ┼ nothing  fill_limit ┴ nothing

Related

References

[5] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 3.1.
[25] A. Meucci. Risk and Asset Allocation (Springer Berlin Heidelberg, 2005). Chapter 3.

source

PortfolioOptimisers.prior — Function

prior(pe::EmpiricalPrior{<:Any, <:Any, Nothing, <:Any}, X::MatNum,
      F::Option{<:MatNum} = nothing, pnl::Option{<:AssetPanel} = nothing;
      dims::Int = 1, strict::Bool = false, kwargs...)

Compute empirical prior moments for asset returns (no horizon adjustment).

prior estimates the mean and covariance of asset returns using the specified empirical prior estimator, without log-normalisation or scaling for investment horizon. The mean and covariance are computed using the estimators stored in pe, and returned in a LowOrderPrior result.

Mathematical definition

pe.me computes the mean and pe.ce computes the covariance, so both moments are whatever those estimators return. Under the default pair — SimpleExpectedReturns and PortfolioOptimisersCovariance with no observation weights — they reduce to the sample moments:

\[\begin{align} \hat{\boldsymbol{\mu}} &= \frac{1}{T} \sum_{t=1}^{T} \boldsymbol{x}_t\,, \\ \hat{\mathbf{\Sigma}} &= \frac{1}{T-1} \sum_{t=1}^{T} (\boldsymbol{x}_t - \hat{\boldsymbol{\mu}})(\boldsymbol{x}_t - \hat{\boldsymbol{\mu}})^\intercal\,. \end{align}\]

Where:

$\hat{\boldsymbol{\mu}}$: $N \times 1$ mean vector.
$\hat{\mathbf{\Sigma}}$: $N \times N$ covariance matrix.
$\boldsymbol{x}_t$: $N \times 1$ vector of asset returns at time $t$.
$T$: Number of observations.

Every choice inside pe.me and pe.ce reaches the result. A shrunk mean and a denoised covariance move both away from the display above rather than refining it.

This method takes the arithmetic moments of X directly. It applies no log transform, so it is not the $h = 1$ case of the horizon method: that one still passes through $\log(1 + x_t)$ and back, and the round trip is an identity only in the limit of small returns.

The scenario cap

pe.max_scenarios cuts the returns matrix the result carries down to its last max_scenarios rows, through scenario_window, and leaves mu and sigma fitted over every observation. The cut is taken before the fill, so the share the fill measures is the share of the window a consumer actually reads. A max_scenarios of nothing, the default, carries every row, and a cap at or above the number of observations is a view that copies nothing. When the cap cuts, the result states the number of observations the moments were fitted over in ens, through scenario_ens, so a consumer that prices a sample size reads that count and not the rows carried; otherwise ens stays nothing.

The scenario fill

Under a mask-aware pe.me and pe.ce — the exponentially weighted family, and the plain family wherever a CoveragePolicy is set — an asset that lists inside the window is answered from the observations it has, so it is investable and its column of X still carries a NaN at every earlier row. scenario_fill writes 0 at those entries once, so that every consumer of the result reads a finite investable column; a non-investable asset keeps its whole NaN column, and mu and sigma are untouched. The fill costs accuracy at four consumers, and the message names all four: a scenario-based measure understates that asset's risk over the filled rows, a hierarchical optimiser may branch the asset alone and overweight it, an entropy pooling view on it is calibrated on the filled column, and a meta-optimiser carries the fill into the outer problem. It is silent while the worst investable column's own filled share stays at or below the limit resolve_fill_limit derives, warns above it, and refuses any fill under strict. pe.fill_limit defaults to nothing, which derives 1 - min_coverage from the arms' coverage floor and never fires, and which names every fill where no arm states a floor. Under a plain estimator with no policy the fill never fires, because an asset the estimator could not cover leaves the Coverage Universe and is not investable.

Arguments

pe: Empirical prior estimator.
X: Asset returns matrix (observations × assets).
F: Factor returns matrix (ignored).
pnl: Optional AssetPanel, whose active mask the Coverage Universe of the fit is derived from. nothing makes the rule finiteness alone.
dims: Dimension along which to perform the computation.
strict: Whether a zero-filled scenario raises rather than warns. Any fill raises under strict; otherwise a column filled above the limit resolve_fill_limit derives warns, and every fill warns while that limit is nothing.
kwargs...: Additional keyword arguments passed to mean and covariance estimators.

Validation

dims in (1, 2).
At least one asset must be in the Coverage Universe.
pe.fill_limit is at most 1 - min_coverage over the arms that state a coverage floor, else a DomainError is thrown at the fit, because a looser limit is dead by construction.
The zero-filled share of the worst investable column of X is at or below the limit resolve_fill_limit derives, else a warning naming the assets is emitted, or an ArgumentError naming them is raised under strict, which any fill raises. A limit of nothing — no arm states a floor and pe.fill_limit is nothing — is at or below no share, so any fill is named.

Returns

pr::LowOrderPrior: Result object containing asset returns, mean vector, and covariance matrix.

Related

source

PortfolioOptimisers.prior — Function

prior(pe::EmpiricalPrior{<:Any, <:Any, <:Number, <:Any}, X::MatNum,
      F::Option{<:MatNum} = nothing, pnl::Option{<:AssetPanel} = nothing;
      dims::Int = 1, strict::Bool = false, kwargs...)

Compute empirical prior moments for asset returns with investment horizon adjustment.

prior estimates the mean and covariance of asset returns using the specified empirical prior estimator, applying log-normalisation and scaling for the investment horizon. The asset returns are log-transformed, moments are computed using the estimators stored in pe, and then rescaled according to the investment horizon. The final mean and covariance are transformed back to arithmetic returns and returned in a LowOrderPrior result.

Mathematical definition

pe.me and pe.ce are applied to the log-returns $\log(1 + x_t)$ rather than to X itself. The two log-moments are scaled by the investment horizon $h$, then converted back to arithmetic returns:

\[\begin{align} \tilde{\boldsymbol{\mu}} &= h \cdot \hat{\boldsymbol{\mu}}_{\log}\,, \\ \tilde{\mathbf{\Sigma}} &= h \cdot \hat{\mathbf{\Sigma}}_{\log}\,. \end{align}\]

\[\begin{align} \hat{\mu}_i &= \exp\!\left(\tilde{\mu}_i + \tfrac{1}{2}\tilde{\sigma}_{ii}\right) - 1\,, \\ \hat{\sigma}_{ij} &= (\hat{\mu}_i + 1)(\hat{\mu}_j + 1)\left(\exp(\tilde{\sigma}_{ij}) - 1\right)\,. \end{align}\]

Where:

$\tilde{\boldsymbol{\mu}}$, $\tilde{\mathbf{\Sigma}}$: Horizon-scaled log-return mean and covariance.
$h$: Investment horizon.
$\hat{\boldsymbol{\mu}}_{\log}$, $\hat{\mathbf{\Sigma}}_{\log}$: Mean and covariance of the log-returns $\log(1 + x_t)$, computed by pe.me and pe.ce.
$\hat{\mu}_i$: Arithmetic mean return for asset $i$.
$\hat{\sigma}_{ij}$: Arithmetic covariance between assets $i$ and $j$.

X in the returned LowOrderPrior is the arithmetic returns matrix the caller supplied. Only the moments are computed in log space.

Algorithm

The order of steps 5 to 7 is not free. Step 6 reads the mu that step 5 left, which is $\hat{\mu}_i + 1$ and not $\hat{\mu}_i$, because step 7 has not yet subtracted the one. The second closed form asks for exactly that factor, so the body meets it by ordering rather than by recomputing. Moving step 7 in front of step 6 replaces each $\hat{\mu}_i + 1$ by $\hat{\mu}_i$, and on a daily return series that collapses the covariance to a small fraction of its value.

Orient X to observations × assets with dims_oriented.
Take the log-returns X_log, as log1p.(X).
Compute the log-return mean mu on X_log, under pe.me, and the log-return covariance sigma on X_log, under pe.ce.
Scale both by pe.horizon, giving $\tilde{\boldsymbol{\mu}}$ and $\tilde{\mathbf{\Sigma}}$.
Overwrite mu with the exponential of the first closed form. This is the arithmetic mean plus one, because the subtraction is still to come.
Overwrite sigma with the second closed form, whose $\hat{\mu}_i + 1$ factors are the mu of step 5.
Subtract one from mu, giving $\hat{\boldsymbol{\mu}}$.
Return a LowOrderPrior carrying the arithmetic X of step 1 under scenario_fill, mu, sigma, and the ens of scenario_ens.

The scenario cap

The scenario fill

Step 8 takes the same fill the no-horizon method takes, on the arithmetic X the caller handed in and against the arithmetic moments the result carries. Under a mask-aware pe.me and pe.ce an asset that lists inside the window is investable and its column still carries a NaN at every earlier row; scenario_fill writes 0 there once, silently while the worst investable column's own filled share stays at or below the limit resolve_fill_limit derives, with a warning above it, and refuses any fill under strict. pe.fill_limit defaults to nothing, which derives 1 - min_coverage from the arms' coverage floor and names every fill where no arm states one. A scenario-based measure then understates that asset's risk over the filled rows, and mu and sigma are untouched.

Arguments

pe: Empirical prior estimator.
X: Asset returns matrix (observations × assets).
F: Factor returns matrix (ignored).
pnl: Optional AssetPanel, whose active mask the Coverage Universe of the fit is derived from. nothing makes the rule finiteness alone.
dims: Dimension along which to perform the computation.
strict: Whether a zero-filled scenario raises rather than warns. Any fill raises under strict; otherwise a column filled above the limit resolve_fill_limit derives warns, and every fill warns while that limit is nothing.
kwargs...: Additional keyword arguments passed to mean and covariance estimators.

Validation

dims in (1, 2).
At least one asset must be in the Coverage Universe.
pe.fill_limit is at most 1 - min_coverage over the arms that state a coverage floor, else a DomainError is thrown at the fit, because a looser limit is dead by construction.
The zero-filled share of the worst investable column of X is at or below the limit resolve_fill_limit derives, else a warning naming the assets is emitted, or an ArgumentError naming them is raised under strict, which any fill raises. A limit of nothing — no arm states a floor and pe.fill_limit is nothing — is at or below no share, so any fill is named.

Returns

pr::LowOrderPrior: Result object containing asset returns, mean vector, and covariance matrix.

Related

source

References

[5]: D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[25]: A. Meucci. Risk and Asset Allocation (Springer Berlin Heidelberg, 2005).