Mean test: private API

Functions

PortfolioOptimisers.long_run_quadratic_form — Function

long_run_quadratic_form(kernel::AbstractLongRunKernel, d::VecNum, B::Real, buf::VecNum)
long_run_quadratic_form(kernel::BartlettKernel, d::VecNum, B::Real, buf::VecNum)

Return the sum of the products of every pair of entries of a centred series, each weighted by the kernel at the distance of the pair.

The value is the long-run variance of the series times its count less one. The generic method sums the products at every lag that has a weight that is not zero. The Bartlett method reads the same value off sums of windows of the series, so its cost is linear in the length of the series. Write $B = m + f$, with $m$ the integer part. The squared sums of the windows of $m$ entries give the weight $m - |j|$ at every lag $|j| < m$. The products of each entry with the sum of the $2m + 1$ entries around it give the weight $f$ at every lag $|j| \leq m$. The two together give $B - |j|$ at every lag below $B$. An entry of d that is zero adds nothing, so a missing observation is zero in d, and the distance of a pair is its distance in the series.

Mathematical definition

\[\begin{align} Q &= \sum_{i, j} k\left(\frac{|i - j|}{B}\right) d_{i} d_{j}\,, \\ Q_{\mathrm{Bartlett}} &= \frac{1}{B} \left( \sum_{s = 2 - m}^{P} \left( \sum_{t = \max(s, 1)}^{\min(s + m - 1, P)} d_{t} \right)^{2} + f \sum_{i = 1}^{P} d_{i} \sum_{t = \max(i - m, 1)}^{\min(i + m, P)} d_{t} \right)\,. \end{align}\]

Where:

$d_{i}$: Entry $i$ of the centred series.
$k$: The kernel.
$B$: The bandwidth, $m$ its integer part and $f$ its fractional part.
$P$: The length of the series.

Arguments

kernel: The kernel.
d: The centred series, with a zero at each missing observation.
B: The bandwidth, in entries, B > 0.
buf: Work buffer of length length(d) + 1. The generic method does not read it.

Returns

Q::Real: The weighted sum.

Related

PortfolioOptimisers.lag_cross_sum — Function

lag_cross_sum(d::VecNum, j::Integer) -> Real

Return the sum of the products of the entries of a centred series that are j entries apart.

Arguments

d: The centred series, with a zero at each missing observation.
j: The lag.

Returns

a::Real: $\sum_{t} d_{t} d_{t + j}$.

Related

PortfolioOptimisers.assert_mean_test_rules — Function

assert_mean_test_rules(q::Real, min_obs::Option{<:Integer}, min_periods::Option{<:Real})

Refuse a significance level or a minimum-data rule that no series can meet.

Arguments

q: Significance level.
min_obs: Least number of finite entries, or nothing.
min_periods: Least number of windows in the span, or nothing.

Validation

0 < q < 1. Raises a DomainError.
min_obs >= 2 and min_periods > 0, when given. Raise a DomainError.

Returns

nothing.

Related

PortfolioOptimisers.bootstrap_exceedance_cap — Function

bootstrap_exceedance_cap(q::Real, M::Integer) -> Int

Return the largest count $c$ with $c / (M + 1) \leq q$, by the same division that gives the p-value of a wild bootstrap.

The p-value is $(1 + K) / (M + 1)$, so the test rejects at the level q exactly when $1 + K$ is at most this count. The confidence interval takes its order from the count, so the interval excludes zero exactly when the p-value is at most q. The function reads the count off the division and not off the product $q (M + 1)$, because the product can round below an integer, as 0.29 * 100 does. It starts one below the rounded product, which is never above the count, and counts up.

Arguments

q: Significance level.
M: Number of draws.

Returns

c::Int: The count, in 1:M when q * (M + 1) >= 1 and q < 1.

Related

WildBootstrapMeanTest

PortfolioOptimisers.resolve_mean_test_window — Function

resolve_mean_test_window(lags::Integer, window::Real) -> Real

Return the window that a summary gives its Mean Test, from the two keywords that can state it.

A summary takes lags, the number of entries on each side of a given entry that share data with it, and window, the number of entries that the data of one entry spans. window defaults to lags + 1, so a caller who states lags alone states the window of a stride that divides the horizon. A caller who states the window of a forward window, horizon / step, states the exact count of holding periods that the history rule of a Mean Test reads.

Arguments

lags: The order of the overlap.
window: The window.

Validation

lags >= 0 and window > 0. Raise a DomainError.
lags is 0, or lags == ceil(Int, window) - 1. Raises a ConflictingArgumentError.

Returns

w::Real: window.

Related

PortfolioOptimisers.mean_test_answers — Function

mean_test_answers(mt::AbstractMeanTestEstimator, m::Real, n::Integer, T::Integer,
                  window::Real) -> Bool

Return whether a series carries enough evidence for a Mean Test to answer.

The history rule reads $T / w$ as the number of holding periods that the history covers. For a forward window of horizon base periods scored every step of them, $w$ is horizon / step. Each entry stands for its stride of step base periods, so $T$ entries cover $T \cdot$ step base periods, which is $T / w$ holding periods. That is also the number of independent windows whose mean has the variance of the mean of the $T$ overlapping ones. For an integer $w$ that is exact up to an edge term of order one. So $T$ windows that do not overlap count as $T$ periods.

The oracle counts the base periods from the first date to the last, both included, $(T - 1) \cdot$ step $+ 1$. That counts the last entry as one base period and every other entry as step. So $T$ windows that do not overlap count as $T - 1 + 1 /$ horizon periods, and at a step above one the oracle needs one more entry than this rule. At step = 1 the two rules are equal. The library keeps its rule.

Arguments

mt: The test. The function reads its min_obs and its min_periods.
m: The mean of the finite entries.
n: The number of finite entries.
T: The span of the series.
window: Number of entries that the data of one entry spans.

Returns

ok::Bool: false when the mean is not finite, when n < 2, when n < mt.min_obs, or when T / window < mt.min_periods.

Related

mean_test

PortfolioOptimisers.finite_abs — Function

finite_abs(x::Real) -> Real

Return the absolute value of a statistic, with a statistic that is not finite as Inf.

Arguments

x: The statistic.

Returns

a::Real: abs(x), or Inf when x is not finite.

Related

wild_bootstrap_statistics

PortfolioOptimisers.mean_test_centred — Function

mean_test_centred(c::VecNum, span::AbstractMeanTestSpan) -> Tuple

Return the mean of the finite entries of a series, their count, the series centred on the mean from its first finite entry to its last, the mask of the finite entries of that span, and the span that span sets.

A missing entry is zero in the centred series, so it adds nothing to a sum of products. The centred series starts and stops at a finite entry. The span rule sets the span that the bandwidth and the history rule read.

A series whose spread is round-off of its mean is constant: every entry of d is zero, so the long-run variance is not positive and the test answers NaN. The sum of a constant series over its count can miss its value, as sum(fill(0.03, 100)) / 100 does, and two entries one unit in the last place apart, such as 0.3 and 0.1 + 0.2, have a spread of the same size. The computed mean of $n$ entries misses the exact mean by at most about $n \epsilon / 2$ times the largest absolute entry, the bound of a recursive sum, with $\epsilon$ the round_off_unit of the type of the mean. A largest absolute deviation of at most $n \epsilon$ times the largest absolute entry cannot be told apart from that error, and its t-statistic is of the order of $1 / \epsilon$. An exact number type has $\epsilon = 0$, so only an exactly constant series is constant.

Arguments

c: The series.
span: The span rule.

Returns

m::Real: The mean, NaN when no entry is finite.
n::Int: The number of finite entries.
d::Vector{<:Real}: The centred span, empty when no entry is finite, and all zero when the series is constant to round-off.
f::BitVector: Whether each entry of d is a finite observation.
T::Int: The span of mean_test_span_length, 0 when no entry is finite.

Related

PortfolioOptimisers.round_off_unit — Function

round_off_unit(::Type{T}) where {T <: Real} -> Real
round_off_unit(::Type{T}) where {T <: Rational} -> Rational

Return the relative round-off of one operation in a number type: eps(T), and zero for an exact Rational.

Arguments

T: The number type.

Returns

e::Real: The unit.

Related

mean_test_centred

PortfolioOptimisers.zero_round_off_spread! — Function

zero_round_off_spread!(d::VecNum, f::AbstractVector{Bool}, m::Number, n::Integer) -> VecNum

Set every entry of a centred series to zero when its spread is round-off of its mean, the rule that mean_test_centred states.

Arguments

d: The centred series, with a zero at each missing observation. It is changed in place.
f: Whether each entry of d is a finite observation.
m: The mean.
n: The number of finite entries.

Returns

d::VecNum: The series, all zero when its largest absolute entry is at most n times the round_off_unit of the type of m times the largest absolute entry of the series before the centring.

Related

mean_test_centred

PortfolioOptimisers.finite_sum_bounds — Function

finite_sum_bounds(c::VecNum, Tm::Type) -> Tuple

Return the number of finite entries of a series, their sum in the type Tm, and the positions of the first and the last of them.

Arguments

c: The series.
Tm: The type the sum is read in, which mean_test_centred derives from c.

Returns

n::Int: The number of finite entries.
s: Their sum.
i1::Int, i2::Int: The positions of the first and the last finite entry, 0 when no entry is finite.

Related

mean_test_centred

PortfolioOptimisers.studentised_mean — Function

studentised_mean(m::Real, Q::Real, n::Integer, nd::Real) -> Real

Return the mean over its long-run standard error.

The long-run variance is $Q / n_{d}$. WildBootstrapMeanTest divides by $n$, as its source does. Its divisor scales the t-statistic of the observed series and of every bootstrap draw by one factor, so it moves no p-value and no interval. OverlapMeanTest divides by the count of its AbstractOverlapStudentLaw, which removes the bias of the centring under its null hypothesis, and its p-value reads the divisor.

Arguments

m: The mean.
Q: The weighted sum of long_run_quadratic_form.
n: The number of finite entries.
nd: The divisor of the long-run variance.

Returns

t::Real: $m / \sqrt{Q / n_{d}} \sqrt{n}$, or NaN when Q is not positive.

Related

mean_test

PortfolioOptimisers.band_prefix_sums — Function

band_prefix_sums(f::AbstractVector{Bool}) -> Tuple

Return the prefix sums of the mask of the finite entries of a series, and of the mask times the position.

Arguments

f: Whether each entry is a finite observation.

Returns

F0::Vector{Int}: F0[l + 1] is the number of finite entries at positions up to l, F0[1] = 0.
F1::Vector{Int}: F1[l + 1] is the sum of their positions.

Related

PortfolioOptimisers.band_kernel_sum — Function

band_kernel_sum(F0::AbstractVector{<:Integer}, F1::AbstractVector{<:Integer}, w::Real,
                lo::Integer, hi::Integer, k::Integer, z::Number) -> Number

Return the sum of $w - |l - k|$ over the finite entries $l$ in lo:hi, which is an entry of $W \Sigma_{0}$ of NullMomentLaw.

The sum splits at k. On each side the term is linear in $l$, so the two prefix sums of band_prefix_sums give it in a fixed number of operations. The integer part is exact, and only the product with w rounds.

Arguments

F0, F1: The prefix sums of band_prefix_sums.
w: The window.
lo, hi: The range of $l$, clipped to the series. The clipped range must not be empty.
k: The position that the distance reads.
z: The zero of the type the sum is read in.

Returns

s::Number: The sum.

Related

PortfolioOptimisers.overlap_null_rows — Function

overlap_null_rows(f::AbstractVector{Bool}, F0::AbstractVector{<:Integer},
                  F1::AbstractVector{<:Integer}, w::Real, L::Integer, z::Number) -> Tuple

Return the row sums of the band of weights and of the null correlation of NullMomentLaw, their totals, and the inner product of the two.

The correlation is scaled by w, $w - j$ at the lag $j$. The divisor and the degrees of freedom do not read the scale.

Arguments

f: Whether each entry is a finite observation.
F0, F1: The prefix sums of band_prefix_sums.
w: The window.
L: The order of the band.
z: The zero of the type the sums are read in.

Returns

u::Vector{Int}: $W \mathbf{1}$, zero at a missing entry.
v::Vector: $w \Sigma_{0} \mathbf{1}$, zero at a missing entry.
a::Int: $\mathbf{1}^{\intercal} W \mathbf{1}$.
b: $w \mathbf{1}^{\intercal} \Sigma_{0} \mathbf{1}$.
uv: $u^{\intercal} v$.

Related

NullMomentLaw

PortfolioOptimisers.overlap_null_second_moment — Function

overlap_null_second_moment(f::AbstractVector{Bool}, F0::AbstractVector{<:Integer},
                           F1::AbstractVector{<:Integer}, w::Real, L::Integer,
                           u::AbstractVector{<:Integer}, v::AbstractVector,
                           abuv::Tuple, n::Integer, z::Number) -> Number

Return $\operatorname{tr}((A \Sigma_{0})^{2})$ of NullMomentLaw, with the correlation scaled by w.

The centring is a correction of rank two, $M \Sigma_{0} M = \Sigma_{0} + U C U^{\intercal}$ with $U = [\mathbf{1}, \Sigma_{0} \mathbf{1}]$. So the trace is $\operatorname{tr}(W \Sigma_{0} W \Sigma_{0})$ of overlap_null_square_trace, plus terms that read the vectors $\Sigma_{0} u$ and $W v$ and a product of $2 \times 2$ matrices.

Arguments

f: Whether each entry is a finite observation.
F0, F1: The prefix sums of band_prefix_sums.
w: The window.
L: The order of the band.
u, v: The row sums of overlap_null_rows.
abuv: Its totals (a, b, uv).
n: The number of finite entries.
z: The zero of the type the sums are read in.

Returns

tr2::Number: The trace.

Related

NullMomentLaw

PortfolioOptimisers.overlap_null_square_trace — Function

overlap_null_square_trace(f::AbstractVector{Bool}, F0::AbstractVector{<:Integer},
                          F1::AbstractVector{<:Integer}, w::Real, L::Integer,
                          v::AbstractVector, z::Number) -> Number

Return $\operatorname{tr}(W \Sigma_{0} W \Sigma_{0})$ of NullMomentLaw, with the correlation scaled by w.

The trace is $\sum_{i, k} K_{i k} K_{k i}$ with $K = W \Sigma_{0}$. An entry $K_{i k}$ sums the correlation with $k$ over the finite entries that are at most $L$ from both $i$ and $k$, and $K_{k i}$ sums the correlation with $i$ over the same entries. So each pair is two calls of band_kernel_sum, and only the pairs at most $2L$ apart are not zero. The diagonal entry $K_{i i}$ is v[i].

Arguments

f: Whether each entry is a finite observation.
F0, F1: The prefix sums of band_prefix_sums.
w: The window.
L: The order of the band.
v: The row sums of the correlation, from overlap_null_rows.
z: The zero of the type the sum is read in.

Returns

t::Number: The trace.

Related

overlap_null_second_moment

PortfolioOptimisers.band_pair_products — Function

band_pair_products(f::AbstractVector{Bool}, F0::AbstractVector{<:Integer},
                   F1::AbstractVector{<:Integer}, w::Real, L::Integer, i::Integer,
                   z::Number) -> Number

Return $\sum_{k > i} K_{i k} K_{k i}$ of overlap_null_square_trace, over the finite entries $k$ at most $2L$ after $i$.

Arguments

f: Whether each entry is a finite observation.
F0, F1: The prefix sums of band_prefix_sums.
w: The window.
L: The order of the band.
i: The row.
z: The zero of the type the sum is read in.

Returns

s::Number: The sum.

Related

overlap_null_square_trace