GitHub

Histogram

PortfolioOptimisers.AbstractBins — Type
abstract type AbstractBins <: AbstractAlgorithm

Abstract supertype for all histogram binning algorithms.

AbstractBins is the abstract type for all binning algorithm types used in histogram-based calculations within PortfolioOptimisers.jl, such as mutual information and variation of information analysis. Concrete subtypes implement specific binning strategies (e.g., Knuth, Freedman-Diaconis, Scott, Hacine-Gharbi-Ravier) and provide a consistent interface for bin selection.

A bin count is chosen per pair of variables, not per variable, because the measures that read it estimate a joint histogram.

Interfaces

In order to implement a new binning algorithm which will work seamlessly with the library, subtype AbstractBins with all necessary parameters as part of the struct, and implement the following method:

  • calc_num_bins(bins::AbstractBins, xj::VecNum, xi::VecNum, j::Integer, i::Integer, T::Integer) -> Integer: The number of histogram bins for the pair (xj, xi).

Arguments

  • bins: Binning algorithm or fixed number of bins.
  • xj: Data vector for variable j.
  • xi: Data vector for variable i.
  • j: Index of variable j.
  • i: Index of variable i.
  • T: Number of observations.

Returns

  • nbins::Integer: Number of histogram bins for the variable pair.

Examples

We can create a dummy binning algorithm as follows:

julia> struct MyBins <: PortfolioOptimisers.AbstractBins endjulia> function PortfolioOptimisers.calc_num_bins(bins::MyBins, xj::PortfolioOptimisers.VecNum,                                                  xi::PortfolioOptimisers.VecNum, j::Integer,                                                  i::Integer, T::Integer)           return 4       endjulia> PortfolioOptimisers.calc_num_bins(MyBins(), [1.0, 2.0, 3.0], [3.0, 2.0, 1.0], 1, 2, 3)4

Related

source
PortfolioOptimisers.BinWidthBins — Type
abstract type BinWidthBins <: AbstractBins

Abstract supertype for all histogram binning algorithms based on a bin width selection rule.

BinWidthBins is the abstract type for all binning algorithm types that select the number of bins by first computing an optimal bin width from the data, such as Knuth, Freedman-Diaconis, and Scott. Concrete subtypes implement specific binning strategies and provide a consistent interface for bin selection in histogram-based calculations within PortfolioOptimisers.jl.

A subtype states a bin width for a single variable. The shared calc_num_bins method turns that width into a bin count for the pair, so a subtype implements no calc_num_bins method of its own.

Interfaces

In order to implement a new bin width rule which will work seamlessly with the library, subtype BinWidthBins with all necessary parameters as part of the struct, and implement the following method:

  • bin_width(bins::BinWidthBins, x::VecNum) -> Number: The optimal histogram bin width for x.

Arguments

  • bins: Binning algorithm or fixed number of bins.
  • x: Data vector.

Returns

  • dx::Number: Optimal histogram bin width.

Examples

We can create a dummy bin width rule as follows:

julia> struct MyWidth <: PortfolioOptimisers.BinWidthBins endjulia> function PortfolioOptimisers.bin_width(bins::MyWidth, x::PortfolioOptimisers.VecNum)           return (maximum(x) - minimum(x)) / 4       endjulia> PortfolioOptimisers.calc_num_bins(MyWidth(), [1.0, 2.0, 3.0], [3.0, 2.0, 1.0], 1, 2, 3)4

Related

source
PortfolioOptimisers.Knuth — Type
struct Knuth{__T_args, __T_kwargs} <: BinWidthBins

Histogram binning algorithm using Knuth's rule.

Knuth implements Knuth's rule for selecting the optimal number of bins in a histogram [19]. This method maximises the posterior probability of a piecewise-constant density model given the data, so the binning balances bias against variance.

Fields

  • args: Additional positional arguments passed to the optimisation function.
  • kwargs: Additional keyword arguments passed to the optimisation function.

Constructors

Knuth(;  args::Tuple = (Optim.NelderMead(),),  kwargs::NamedTuple = (;)) -> Knuth

Examples

julia> Knuth()Knuth    args ┼ Tuple{Optim.NelderMead{Optim.AffineSimplexer, Optim.AdaptiveParameters}}: (Optim.NelderMead{Optim.AffineSimplexer, Optim.AdaptiveParameters}(Optim.AffineSimplexer(0.025, 0.5), Optim.AdaptiveParameters(1.0, 1.0, 0.75, 1.0)),)  kwargs ┴ @NamedTuple{}: NamedTuple()

Related

References

  • [19] K. H. Knuth. Optimal data-based binning for histograms and histogram-based probability density models. Digital Signal Processing 95, 102581 (2019).
source
PortfolioOptimisers.FreedmanDiaconis — Type
struct FreedmanDiaconis <: BinWidthBins

Histogram binning algorithm using the Freedman-Diaconis rule.

FreedmanDiaconis implements the Freedman-Diaconis rule for selecting the number of bins in a histogram [20]. This method determines bin width based on the interquartile range (IQR) and the number of data points, making it robust to outliers and suitable for skewed distributions.

Constructors

FreedmanDiaconis() -> FreedmanDiaconis

Examples

julia> FreedmanDiaconis()FreedmanDiaconis()

Related

References

  • [20] D. Freedman and P. Diaconis. On the histogram as a density estimator: L2 theory. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 57, 453–476 (1981).
source
PortfolioOptimisers.Scott — Type
struct Scott <: BinWidthBins

Histogram binning algorithm using Scott's rule.

Scott implements Scott's rule for selecting the number of bins in a histogram [21]. This method chooses bin width based on the standard deviation of the data and the number of observations, providing a good default for normally distributed data.

Constructors

Scott() -> Scott

Examples

julia> Scott()Scott()

Related

References

  • [21] D. W. Scott. On optimal and data-based histograms. Biometrika 66, 605–610 (1979).
source
PortfolioOptimisers.HacineGharbiRavier — Type
struct HacineGharbiRavier <: AbstractBins

Histogram binning algorithm using the Hacine-Gharbi–Ravier rule.

HacineGharbiRavier selects the bin count from the sample size and the Pearson correlation of the pair, minimising the mean square error of the joint entropy estimate. It is the default for the information-theoretic measures, mutual_info and variation_info.

Mathematical definition

Two closed forms, selected by the pair's Pearson correlation $\rho$. For $\rho^2 \neq 1$ the bi-histogram formula applies:

\[\begin{align} M &= \left[\frac{1}{\sqrt{2}} \sqrt{1 + \sqrt{1 + \frac{24 T}{1 - \rho^2}}}\right]\,. \end{align}\]

The form is singular at $\rho^2 = 1$, at both ends of the correlation range. There the pair is deterministic and carries no joint information beyond one marginal, so the univariate formula applies, which is the limit of the bi-histogram one:

\[\begin{align} z &= \sqrt[3]{8 + 324 T + 12 \sqrt{36 T + 729 T^2}}\,, \\ M &= \left[\frac{z}{6} + \frac{2}{3 z} + \frac{1}{3}\right]\,. \end{align}\]

Where:

  • $M$: Number of bins.
  • $T$: Number of observations.
  • $\rho$: Pearson correlation between the two series.
  • $[\cdot]$: Rounding to the nearest integer.

Constructors

HacineGharbiRavier() -> HacineGharbiRavier

Examples

julia> HacineGharbiRavier()HacineGharbiRavier()

Related

References

  • [22] A. Hacine-Gharbi, P. Ravier, R. Harba and T. Mohamadi. Low bias histogram-based estimation of mutual information for feature selection. Pattern Recognition Letters 33, 1302–1308 (2012).
  • [23] A. Hacine-Gharbi and P. Ravier. A binning formula of bi-histogram for joint entropy estimation using mean square error minimization. Pattern Recognition Letters 101, 21–28 (2018).
  • [9] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 3.
source
PortfolioOptimisers.bin_width — Function
bin_width(::Scott, x::VecNum)

Compute the optimal histogram bin width for x using Scott's rule [21].

Mathematical definition

\[\begin{align} \Delta_x &= \sigma_x \left(\frac{24 \sqrt{\pi}}{n}\right)^{1/3}\,. \end{align}\]

Where:

  • $\Delta_x$: Bin width.
  • $\sigma_x$: Uncorrected standard deviation of the data.
  • $n$: Number of observations.

Arguments

  • x: Data vector.

Returns

  • dx::Number: Optimal histogram bin width.

Related

References

  • [21] D. W. Scott. On optimal and data-based histograms. Biometrika 66, 605–610 (1979).
source
bin_width(::FreedmanDiaconis, x::VecNum)

Compute the optimal histogram bin width for x using the Freedman-Diaconis rule [20].

Mathematical definition

\[\begin{align} \Delta_x &= \frac{2 \, \mathrm{IQR}(x)}{n^{1/3}}\,. \end{align}\]

Where:

  • $\Delta_x$: Bin width.
  • $\mathrm{IQR}(x)$: Interquartile range of the data.
  • $n$: Number of observations.

Arguments

  • x: Data vector.

Returns

  • dx::Number: Optimal histogram bin width.

Related

References

  • [20] D. Freedman and P. Diaconis. On the histogram as a density estimator: L2 theory. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 57, 453–476 (1981).
source
bin_width(bins::Knuth, x::VecNum)

Compute the optimal histogram bin width for x using Knuth's rule [19].

Mathematical definition

The bin width is the range of x divided by the bin count $M$ that maximises the marginal posterior probability of a piecewise-constant density model with $M$ equal-width bins over that range:

\[\begin{align} F(M) &= n \log M + \log\Gamma\!\left(\frac{M}{2}\right) - M \log\Gamma\!\left(\frac{1}{2}\right) - \log\Gamma\!\left(n + \frac{M}{2}\right) + \sum_{k=1}^{M} \log\Gamma\!\left(n_k + \frac{1}{2}\right)\,. \end{align}\]

Where:

  • $M$: Number of bins.
  • $n$: Number of observations.
  • $n_k$: Number of observations in bin $k$.

The maximiser is an integer, so no closed form gives it. The method searches for it.

Algorithm

  1. Read the range of x into rx, the difference of its two extrema.
  2. Build the objective f, which takes a one-element vector Ms, floors its entry into the bin count M, and returns Inf when M is not positive.
  3. Inside f, bin the data into the counts nk over M equal-width bins of the range, and return the negated posterior of the mathematical definition. The optimiser minimises, so the sign is flipped.
  4. Take the starting point M0 from the bin count that the Freedman-Diaconis rule implies for x, plus one.
  5. Minimise f from M0 with Optim.optimize, passing bins.args and bins.kwargs. The default args is a Nelder-Mead simplex.
  6. Floor the minimiser into a bin count, and return the range divided by it.

Arguments

  • bins: Binning algorithm or fixed number of bins.
  • x: Data vector.

Returns

  • dx::Number: Optimal histogram bin width.

Related

References

  • [19] K. H. Knuth. Optimal data-based binning for histograms and histogram-based probability density models. Digital Signal Processing 95, 102581 (2019).
source
PortfolioOptimisers.calc_num_bins — Function
calc_num_bins(bins::BinWidthBins, xj::VecNum, xi::VecNum, j::Integer, i::Integer,
              args...)
calc_num_bins(bins::HacineGharbiRavier, xj::VecNum, xi::VecNum, j::Integer,
              i::Integer, T::Integer)
calc_num_bins(bins::Integer, args...)

Compute the number of histogram bins for a pair of variables using a specified binning algorithm.

This function determines the number of bins to use for histogram-based calculations (such as mutual information or variation of information) between two variables, based on the selected binning strategy. It dispatches on the binning algorithm type, and the three methods read different arguments: only the HacineGharbiRavier method reads T, and the Integer method reads none of them.

Algorithm

The BinWidthBins method turns a bin width into a bin count.

  1. Read the range of xj into xju - xjl, and divide it by bin_width of xj, giving k1.
  2. When j and i differ, repeat step 1 for xi, giving k2, and select the larger of k1 and k2. The joint histogram is square, so one count serves both axes, and the larger of the two keeps the finer resolution. When j and i are equal, the pair is a variable against itself, so select k1 and read xi no further.
  3. Round the selected value to the nearest integer, and return it.

The HacineGharbiRavier method reads the pair instead of a width.

  1. Take the Pearson correlation of the pair into corr.
  2. Select the closed form of HacineGharbiRavier that corr falls under.
  3. Round the selected value to the nearest integer, and return it.

The Integer method returns bins unchanged.

Arguments

  • bins: Binning algorithm or fixed number of bins.
  • xj: Data vector for variable j.
  • xi: Data vector for variable i.
  • j: Index of variable j.
  • i: Index of variable i.
  • T: Number of observations.
  • args...: Ignored arguments, so that the three methods share one call site.

Returns

  • nbins::Integer: Number of histogram bins for the variable pair.

Related

source

References

[9]
M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020).
[19]
K. H. Knuth. Optimal data-based binning for histograms and histogram-based probability density models. Digital Signal Processing 95, 102581 (2019).
[20]
D. Freedman and P. Diaconis. On the histogram as a density estimator: L2 theory. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 57, 453–476 (1981).
[21]
D. W. Scott. On optimal and data-based histograms. Biometrika 66, 605–610 (1979).
[22]
A. Hacine-Gharbi, P. Ravier, R. Harba and T. Mohamadi. Low bias histogram-based estimation of mutual information for feature selection. Pattern Recognition Letters 33, 1302–1308 (2012).
[23]
A. Hacine-Gharbi and P. Ravier. A binning formula of bi-histogram for joint entropy estimation using mean square error minimization. Pattern Recognition Letters 101, 21–28 (2018).