The source files can be found in examples/.
Cross validation
Cross-validation scores a model on data that the fit did not see. This example shows the cross-validation schemes of PortfolioOptimisers.jl, how to run them on a portfolio optimisation, and the measures and plots you can compute from their results.
You can use cross-validation on its own to score an estimator, or inside a hyperparameter search. NestedClustered and Stacking also take a scheme in their cv field, wrapped in an OptimisationCrossValidation. They then fit the outer estimator on the out-of-sample returns of the inner estimators.
using PortfolioOptimisers, PrettyTablestsfmt = (v, i, j) -> begin if j == 1 return Date(v) else return v endend;resfmt = (v, i, j) -> begin if j == 1 return v else return isa(v, Number) ? "$(round(v*100, digits=3)) %" : v endend;1. Setting up
We use five years of daily data, so that all the schemes have enough rows for their training windows and their test windows.
Cross-validation fits the estimator again on the training rows of every fold. Its fields must therefore be estimators, such as a prior estimator, and not results computed beforehand as in the earlier examples. cross_val_predict throws an error for a JuMP optimiser with a precomputed prior.
using CSV, TimeSeries, DataFrames, Clarabel, StatisticsX = TimeArray(CSV.File(joinpath(@__DIR__, "..", "SP500.csv.gz")); timestamp = :Date)[(end - 252 * 5):end]pretty_table(X[(end - 5):end]; formatters = [tsfmt])rd = prices_to_returns(X)slv = [Solver(; name = :clarabel1, solver = Clarabel.Optimizer, settings = Dict("verbose" => false), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel2, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.95), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel3, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.9), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel4, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.85), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel5, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.8), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel6, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.75), check_sol = (; allow_local = true, allow_almost = true)), Solver(; name = :clarabel7, solver = Clarabel.Optimizer, settings = Dict("verbose" => false, "max_step_fraction" => 0.70), check_sol = (; allow_local = true, allow_almost = true))];┌────────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ timestamp │ AAPL │ AMD │ BAC │ BBY │ CVX │ GE │ HD │ JNJ │ JPM │ KO │ LLY │ MRK │ MSFT │ PEP │ PFE │ PG │ RRC │ UNH │ WMT │ XOM │
│ Date │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ 2022-12-20 │ 131.916 │ 65.05 │ 31.729 │ 77.371 │ 169.497 │ 62.604 │ 310.342 │ 173.109 │ 127.844 │ 61.841 │ 357.55 │ 108.229 │ 240.67 │ 178.765 │ 49.754 │ 147.661 │ 25.65 │ 516.245 │ 142.919 │ 104.964 │
│ 2022-12-21 │ 135.057 │ 67.68 │ 32.212 │ 78.729 │ 171.49 │ 64.67 │ 314.798 │ 175.09 │ 129.282 │ 62.836 │ 365.872 │ 109.611 │ 243.287 │ 180.017 │ 50.084 │ 149.015 │ 26.574 │ 523.519 │ 144.04 │ 106.312 │
│ 2022-12-22 │ 131.846 │ 63.86 │ 31.927 │ 78.563 │ 168.918 │ 63.727 │ 311.604 │ 174.45 │ 127.814 │ 62.383 │ 363.187 │ 109.739 │ 237.077 │ 178.627 │ 50.065 │ 149.359 │ 25.232 │ 523.072 │ 142.354 │ 104.168 │
│ 2022-12-23 │ 131.477 │ 64.52 │ 32.005 │ 79.432 │ 174.14 │ 63.742 │ 314.177 │ 174.893 │ 128.421 │ 62.855 │ 365.762 │ 110.35 │ 237.614 │ 179.781 │ 50.249 │ 149.781 │ 26.226 │ 527.26 │ 142.641 │ 106.922 │
│ 2022-12-27 │ 129.652 │ 63.27 │ 32.065 │ 79.93 │ 176.329 │ 64.561 │ 314.985 │ 174.844 │ 128.871 │ 63.24 │ 362.76 │ 110.607 │ 235.852 │ 180.58 │ 49.57 │ 151.086 │ 26.375 │ 527.935 │ 142.681 │ 108.408 │
│ 2022-12-28 │ 125.674 │ 62.57 │ 32.301 │ 78.279 │ 173.728 │ 63.883 │ 311.22 │ 174.085 │ 129.575 │ 62.609 │ 363.098 │ 109.581 │ 233.434 │ 179.278 │ 49.25 │ 149.133 │ 24.497 │ 524.422 │ 140.181 │ 106.627 │
└────────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘We use a MeanRisk estimator. Cross-validation works for every optimisation estimator except the finite allocations, DiscreteAllocation and GreedyAllocation, and it also works for an estimator that computes a Pareto frontier.
mr = MeanRisk(; opt = JuMPOptimiser(; slv = slv))MeanRisk
opt ┼ JuMPOptimiser
│ pe ┼ EmpiricalPrior
│ │ ce ┼ PortfolioOptimisersCovariance
│ │ │ ce ┼ Covariance
│ │ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ │ w ┴ nothing
│ │ │ │ ce ┼ GeneralCovariance
│ │ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ │ w ┴ nothing
│ │ │ │ alg ┼ FullMoment()
│ │ │ │ w ┴ nothing
│ │ │ mp ┼ MatrixProcessing
│ │ │ │ pdm ┼ Posdef
│ │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ │ dn ┼ nothing
│ │ │ │ dt ┼ nothing
│ │ │ │ alg ┼ nothing
│ │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ │ me ┼ SimpleExpectedReturns
│ │ │ w ┴ nothing
│ │ horizon ┼ nothing
│ │ fill_limit ┼ nothing
│ │ x_src ┴ nothing
│ slv ┼ 7-element Vector{Solver}
│ │ Solver ⋯
│ │ Solver ⋯
│ │ Solver ⋯
│ │ Solver ⋯
│ │ Solver ⋯
│ │ Solver ⋯
│ │ Solver ⋯
│ wb ┼ WeightBounds
│ │ lb ┼ Float64: 0.0
│ │ ub ┴ Float64: 1.0
│ bgt ┼ Float64: 1.0
│ sbgt ┼ nothing
│ gbgt ┼ nothing
│ lt ┼ nothing
│ st ┼ nothing
│ lcse ┼ nothing
│ cte ┼ nothing
│ gcarde ┼ nothing
│ sgcarde ┼ nothing
│ smtx ┼ nothing
│ sgmtx ┼ nothing
│ slt ┼ nothing
│ sst ┼ nothing
│ sglt ┼ nothing
│ sgst ┼ nothing
│ tn ┼ nothing
│ fees ┼ nothing
│ sets ┼ nothing
│ tr ┼ nothing
│ ple ┼ nothing
│ ret ┼ ArithmeticReturn
│ │ settings ┼ JuMPReturnsSettings
│ │ │ scale ┼ Float64: 1.0
│ │ │ lb ┼ nothing
│ │ │ rte ┼ Bool: true
│ │ │ fee ┼ Bool: true
│ │ │ mic ┴ Bool: true
│ │ ucs ┼ nothing
│ │ mu ┼ nothing
│ │ mtx_sqrt ┴ EigenFallbackSquareRoot()
│ sca ┼ SumScalariser()
│ ccnt ┼ nothing
│ cobj ┼ nothing
│ sc ┼ Int64: 1
│ so ┼ Int64: 1
│ ss ┼ nothing
│ card ┼ nothing
│ scard ┼ nothing
│ l2c ┼ nothing
│ lpc ┼ nothing
│ linfc ┼ nothing
│ l1 ┼ nothing
│ l2 ┼ nothing
│ lp ┼ nothing
│ linf ┼ nothing
│ xbgt ┼ Bool: false
│ strict ┴ Bool: false
r ┼ Variance
│ settings ┼ RiskMeasureSettings
│ │ scale ┼ Float64: 1.0
│ │ ub ┼ nothing
│ │ rke ┴ Bool: true
│ sigma ┼ nothing
│ chol ┼ nothing
│ rc ┼ nothing
│ alg ┼ SquaredSOCRiskExpr()
│ mtx_sqrt ┼ EigenFallbackSquareRoot()
│ ratio_alg ┼ SquareRootRatio()
│ corner ┴ nothing
obj ┼ MinimumRisk()
wi ┼ nothing
fb ┴ nothing
2. Cross validation
2.1 KFold
The simplest scheme is K-fold. It cuts the data into K consecutive folds, trains on K - 1 of them and tests on the fold that is left. It does this K times, and each fold is the test set once.
You can make the KFold indices without an optimisation. With 5 folds, a fold is one year of 252 rows.
kfold = KFold(; n = 5)KFold
n ┼ Int64: 5
purged_size ┼ Int64: 0
embargo_size ┼ Int64: 0
wd ┼ nothing
fa ┼ nothing
store_weight_path ┼ Bool: false
strict ┴ Bool: false
To show the splits, we make them with split. cross_val_predict makes them itself, so you do not need this step.
kfold_res = split(kfold, rd)show(kfold_res.train_idx)show(kfold_res.test_idx)[[253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 1009, 1010, 1011, 1012, 1013, 1014, 1015, 1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026, 1027, 1028, 1029, 1030, 1031, 1032, 1033, 1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044, 1045, 1046, 1047, 1048, 1049, 1050, 1051, 1052, 1053, 1054, 1055, 1056, 1057, 1058, 1059, 1060, 1061, 1062, 1063, 1064, 1065, 1066, 1067, 1068, 1069, 1070, 1071, 1072, 1073, 1074, 1075, 1076, 1077, 1078, 1079, 1080, 1081, 1082, 1083, 1084, 1085, 1086, 1087, 1088, 1089, 1090, 1091, 1092, 1093, 1094, 1095, 1096, 1097, 1098, 1099, 1100, 1101, 1102, 1103, 1104, 1105, 1106, 1107, 1108, 1109, 1110, 1111, 1112, 1113, 1114, 1115, 1116, 1117, 1118, 1119, 1120, 1121, 1122, 1123, 1124, 1125, 1126, 1127, 1128, 1129, 1130, 1131, 1132, 1133, 1134, 1135, 1136, 1137, 1138, 1139, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1150, 1151, 1152, 1153, 1154, 1155, 1156, 1157, 1158, 1159, 1160, 1161, 1162, 1163, 1164, 1165, 1166, 1167, 1168, 1169, 1170, 1171, 1172, 1173, 1174, 1175, 1176, 1177, 1178, 1179, 1180, 1181, 1182, 1183, 1184, 1185, 1186, 1187, 1188, 1189, 1190, 1191, 1192, 1193, 1194, 1195, 1196, 1197, 1198, 1199, 1200, 1201, 1202, 1203, 1204, 1205, 1206, 1207, 1208, 1209, 1210, 1211, 1212, 1213, 1214, 1215, 1216, 1217, 1218, 1219, 1220, 1221, 1222, 1223, 1224, 1225, 1226, 1227, 1228, 1229, 1230, 1231, 1232, 1233, 1234, 1235, 1236, 1237, 1238, 1239, 1240, 1241, 1242, 1243, 1244, 1245, 1246, 1247, 1248, 1249, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 1259, 1260], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008]]UnitRange{Int64}[1:252, 253:504, 505:756, 757:1008, 1009:1260]We run the cross-validation.
kfold_pred = cross_val_predict(mr, rd, kfold)MultiPeriodPredictionResult
pred ┼ 5-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 1260-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 1260-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
We load the plot packages and draw four plots of the K-fold prediction.
using StatsPlots, GraphRecipesThe cumulative returns of the portfolio over all the KFold test windows.
plot_portfolio_cumulative_returns(kfold_pred)
The distribution of the weight of every asset over the folds.
plot_weight_stability(kfold_pred)
The turnover of the portfolio between consecutive folds.
plot_turnover(kfold_pred)
The score of every fold, here the second moment, which is the variance.
plot_cv_scores(LowOrderMoment(; alg = SecondMoment()), kfold_pred)
The result is a MultiPeriodPredictionResult. Its field pred is a vector of PredictionResult values, one per fold. Every PredictionResult keeps the optimisation result of its training fold in res, and a PredictionReturnsResult of the optimised portfolio on its test fold in rd.
You can index pred to get one fold. The property mrd joins the returns of all the folds into one PredictionReturnsResult, and the property res gives the vector of the optimisation results of the folds. The test folds cover all the rows once. KFold can drop training rows next to a test fold: purged_size rows on each side of it, the purge, and embargo_size more rows after it, the embargo. Both are zero here, and neither drops a test row. We compare the timestamps of mrd with the timestamps of the returns.
println("isequal(kfold_pred.mrd.ts, rd.ts) = $(isequal(kfold_pred.mrd.ts, rd.ts))")isequal(kfold_pred.mrd.ts, rd.ts) = trueYou can also compute risk measures on the predicted returns. A prediction holds only the return series of the portfolio. A measure that needs the weights or data per asset cannot score it, and expected_risk throws an error for such a measure. These measures cannot score a prediction: StandardDeviation, Variance, UncertaintySetVariance, NegativeSkewness, TurnoverRiskMeasure, TrackingRiskMeasure with WeightsTracking, EqualRisk, ExpectedReturn and ExpectedReturnRiskRatio. Neither can RiskTrackingRiskMeasure, VarianceSkewKurtosis, a ValueatRisk with DistributionValueatRisk, a moment measure whose target mu is a vector per asset, or a ratio that uses one of these. Most of them have a replacement.
- For the variance, use
LowOrderMomentwithalg = SecondMoment(). For the standard deviation, use it withalg = SecondMoment(; alg2 = SOCRiskExpr()). - For
NegativeSkewness, useHighOrderMomentorSkewness. - For
ExpectedReturnandExpectedReturnRiskRatio, useMeanReturnandMeanReturnRiskRatio.
We compute the variance.
println("KFold(5) prediction variance = $(expected_risk(LowOrderMoment(; alg = SecondMoment()), kfold_pred))")KFold(5) prediction variance = 0.000132669139899212682.2 Combinatorial
CombinatorialCrossValidation cuts the rows into n_folds consecutive folds. Every choice of n_test_folds of them is the test set of one split, and the other folds are its training set. A purge and an embargo remove the training rows next to the test folds. The test folds of the splits then join into paths, and a path covers all the rows once. The scheme needs one fit per split, many more than K-fold, and it gives many paths instead of one.
optimal_number_folds chooses n_folds and n_test_folds from the number of rows, a target size of the training set and a target number of paths. The keywords train_size_w and n_test_paths_w set the weight of each target.
T = size(rd.X, 1)target_train_size = 200target_n_test_paths = 70n_folds, n_test_folds = optimal_number_folds(T, target_train_size, target_n_test_paths)cfold = CombinatorialCrossValidation(; n_folds = n_folds, n_test_folds = n_test_folds)CombinatorialCrossValidation
n_folds ┼ Int64: 13
n_test_folds ┼ Int64: 11
purged_size ┼ Int64: 0
embargo_size ┼ Int64: 0
wd ┼ nothing
fa ┼ nothing
store_weight_path ┼ Bool: false
strict ┴ Bool: false
We make the splits.
cfold_res = split(cfold, rd)CombinatorialCrossValidationResult
train_idx ┼ 78-element Vector{Vector{Int64}}
test_idx ┼ 78-element Vector{Vector{Vector{Int64}}}
path_ids ┴ 11×78 Matrix{Int64}
The scheme uses 13 folds with 11 test folds, so it has binomial(13, 11) = 78 splits. A split trains on only 2 folds, about 194 rows, and tests on the other 11. The 78 splits give 78 × 11 = 858 test folds, which join into 66 paths.
cfold_res.path_ids has one row per test fold of a split and one column per split. Every entry is the number of the path that the test fold belongs to.
cfold_res.path_ids11×78 Matrix{Int64}:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 56 57 58 59 60 61 62 63 64 65 66 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 46 47 48 49 50 51 52 53 54 55 55 56 57 58 59 60 61 62 63 64 65 65 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 37 38 39 40 41 42 43 44 45 45 46 47 48 49 50 51 52 53 54 54 55 56 57 58 59 60 61 62 63 64 64 65 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 29 30 31 32 33 34 35 36 36 37 38 39 40 41 42 43 44 44 45 46 47 48 49 50 51 52 53 53 54 55 56 57 58 59 60 61 62 63 63 64 65 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 22 23 24 25 26 27 28 28 29 30 31 32 33 34 35 35 36 37 38 39 40 41 42 43 43 44 45 46 47 48 49 50 51 52 52 53 54 55 56 57 58 59 60 61 62 62 63 64 65 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 16 17 18 19 20 21 21 22 23 24 25 26 27 27 28 29 30 31 32 33 34 34 35 36 37 38 39 40 41 42 42 43 44 45 46 47 48 49 50 51 51 52 53 54 55 56 57 58 59 60 61 61 62 63 64 65 66
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 11 12 13 14 15 15 16 17 18 19 20 20 21 22 23 24 25 26 26 27 28 29 30 31 32 33 33 34 35 36 37 38 39 40 41 41 42 43 44 45 46 47 48 49 50 50 51 52 53 54 55 56 57 58 59 60 60 61 62 63 64 65 66
1 2 3 4 5 6 7 8 9 10 7 8 9 10 10 11 12 13 14 14 15 16 17 18 19 19 20 21 22 23 24 25 25 26 27 28 29 30 31 32 32 33 34 35 36 37 38 39 40 40 41 42 43 44 45 46 47 48 49 49 50 51 52 53 54 55 56 57 58 59 59 60 61 62 63 64 65 66
1 2 3 4 5 6 4 5 6 6 7 8 9 9 10 11 12 13 13 14 15 16 17 18 18 19 20 21 22 23 24 24 25 26 27 28 29 30 31 31 32 33 34 35 36 37 38 39 39 40 41 42 43 44 45 46 47 48 48 49 50 51 52 53 54 55 56 57 58 58 59 60 61 62 63 64 65 66
1 2 3 2 3 3 4 5 5 6 7 8 8 9 10 11 12 12 13 14 15 16 17 17 18 19 20 21 22 23 23 24 25 26 27 28 29 30 30 31 32 33 34 35 36 37 38 38 39 40 41 42 43 44 45 46 47 47 48 49 50 51 52 53 54 55 56 57 57 58 59 60 61 62 63 64 65 66
1 1 1 2 2 3 4 4 5 6 7 7 8 9 10 11 11 12 13 14 15 16 16 17 18 19 20 21 22 22 23 24 25 26 27 28 29 29 30 31 32 33 34 35 36 37 37 38 39 40 41 42 43 44 45 46 46 47 48 49 50 51 52 53 54 55 56 56 57 58 59 60 61 62 63 64 65 66We run the cross-validation. The result has one prediction per path.
cfold_pred = cross_val_predict(mr, rd, cfold)PopulationPredictionResult
pred ┼ 66-element Vector{MultiPeriodPredictionResult}
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
│ MultiPeriodPredictionResult ⋯
Every one of the 66 paths is an out-of-sample prediction. To score the model, you pick one path or summarise them. The median path is a common choice, because outliers do not move it. You can pick a path with your own function or with a subtype of PredictionScorer. NearestQuantilePrediction computes a measure on all the paths, finds a quantile of the values, by default the median, and returns the first path whose value is nearest to it. It leaves out paths whose optimisation failed or whose measure is not finite.
We use the mean return over the variance as the measure. This ratio divides by the variance, not by the standard deviation, so it is not a Sharpe ratio.
ratio_scorer = NearestQuantilePrediction(; r = MeanReturnRiskRatio(; rk = LowOrderMoment(; alg = SecondMoment())))NearestQuantilePrediction
r ┼ MeanReturnRiskRatio
│ settings ┼ HierarchicalRiskMeasureSettings
│ │ scale ┴ Float64: 1.0
│ rt ┼ MeanReturn
│ │ settings ┼ HierarchicalRiskMeasureSettings
│ │ │ scale ┴ Float64: 1.0
│ │ w ┼ nothing
│ │ flag ┴ Bool: false
│ rk ┼ LowOrderMoment
│ │ settings ┼ RiskMeasureSettings
│ │ │ scale ┼ Float64: 1.0
│ │ │ ub ┼ nothing
│ │ │ rke ┴ Bool: true
│ │ w ┼ nothing
│ │ mu ┼ nothing
│ │ alg ┼ SecondMoment
│ │ │ ve ┼ SimpleVariance
│ │ │ │ me ┼ nothing
│ │ │ │ w ┼ nothing
│ │ │ │ corrected ┴ Bool: true
│ │ │ alg1 ┼ FullMoment()
│ │ │ alg2 ┴ SquaredSOCRiskExpr()
│ sca ┼ SumScalariser()
│ rf ┴ Float64: 0.0
q ┼ Float64: 0.5
r_kwargs ┼ @NamedTuple{}: NamedTuple()
q_kwargs ┴ @NamedTuple{}: NamedTuple()
A scorer is a callable object. It takes the population of paths and returns the path it picks. The field id of that path is its position in the population.
median_pred = ratio_scorer(cfold_pred)MultiPeriodPredictionResult
pred ┼ 13-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 1260-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 1260-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ Int64: 18
opt ┴ nothing
true means that the path is the entry id of cfold_pred.pred.
median_pred === cfold_pred.pred[median_pred.id]trueAs for K-fold, a path covers all the rows. true means that the timestamps of the path are those of the returns.
isequal(median_pred.mrd.ts, rd.ts)trueWe compute the ratio for all the paths and find the path nearest to the median by hand, to compare it with the pick of the scorer. This comparison leaves out no path. It matches the scorer only when no path failed.
mean_var_ratios = expected_risk(MeanReturnRiskRatio(; rk = LowOrderMoment(; alg = SecondMoment())), cfold_pred)argmin(abs.(mean_var_ratios .- median(mean_var_ratios))) == median_pred.idtrueThe next plot shows the distribution of the weight of each asset over the paths. A wide distribution means that the weights depend on which folds the model trains on.
plot_weight_stability(cfold_pred)
plot_cv_scores draws the ratio of every path. A wide spread means that the ratio depends on which folds form the path.
plot_cv_scores(MeanReturnRiskRatio(; rk = LowOrderMoment(; alg = SecondMoment())), cfold_pred)
Any measure that scores a return series works in the scorer. We now pick the median path by variance.
variance_scorer = NearestQuantilePrediction(; r = LowOrderMoment(; alg = SecondMoment()))median_pred_min_variance = variance_scorer(cfold_pred)MultiPeriodPredictionResult
pred ┼ 13-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 1260-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 1260-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ Int64: 32
opt ┴ nothing
The id of this path also indexes it in cfold_pred.pred.
median_pred_min_variance === cfold_pred.pred[median_pred_min_variance.id]truetrue means that this path also covers all the rows.
isequal(median_pred_min_variance.mrd.ts, rd.ts)true2.3 Walk-forward
There are two walk-forward schemes, IndexWalkForward and DateWalkForward. The first sizes its windows in rows. The second sizes them in dates, and you can use Julia's Dates module to put the window boundaries on chosen dates.
A walk-forward trains on the past and tests on the rows that follow, which is how you use a model in practice. The folds run in time order, so a fold can use the weights of the fold before it. A non-fixed Turnover, a WeightsTracking or a TurnoverRiskMeasure takes those previous weights as its reference.
2.3.1 IndexWalkForward
IndexWalkForward is the simpler of the two, so we start with it. We train on one year, 252 rows, and test on the next quarter, 63 rows. There is no purge, and a test window starts on the row after its training window. The first test window starts at row 253, and the test windows together cover the returns from there to the end.
idx_walk_forward = IndexWalkForward(252, round(Int, 252 / 4))idx_walk_forward_res = split(idx_walk_forward, rd)show(idx_walk_forward_res.train_idx)show(idx_walk_forward_res.test_idx)UnitRange{Int64}[1:252, 64:315, 127:378, 190:441, 253:504, 316:567, 379:630, 442:693, 505:756, 568:819, 631:882, 694:945, 757:1008, 820:1071, 883:1134, 946:1197]UnitRange{Int64}[253:315, 316:378, 379:441, 442:504, 505:567, 568:630, 631:693, 694:756, 757:819, 820:882, 883:945, 946:1008, 1009:1071, 1072:1134, 1135:1197, 1198:1260]We run the walk-forward.
idx_walkforward_pred = cross_val_predict(mr, rd, idx_walk_forward)MultiPeriodPredictionResult
pred ┼ 16-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 1008-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 1008-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
true means that the joined test windows start at row 253 and end at the last row.
isequal(idx_walkforward_pred.mrd.ts, rd.ts[253:end])trueThe cumulative return over all the test windows, joined in time order.
plot_portfolio_cumulative_returns(idx_walkforward_pred)
The weights of every fold as a stacked bar.
plot_composition(idx_walkforward_pred)
The turnover at every rebalance, without a turnover constraint.
plot_turnover(idx_walkforward_pred)
Every column of the table holds the weights of one fold.
pretty_table(hcat(DataFrame(:tickers => rd.nx), DataFrame(reduce(hcat, getproperty.(idx_walkforward_pred.res, :w)), Symbol.(1:16))); formatters = [resfmt])┌─────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┐
│ tickers │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │ 9 │ 10 │ 11 │ 12 │ 13 │ 14 │ 15 │ 16 │
│ String │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├─────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┤
│ AAPL │ 3.104 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.007 % │ 0.0 % │ 0.0 % │
│ AMD │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 2.333 % │ 3.919 % │ 2.403 % │ 1.694 % │ 0.0 % │ 0.0 % │
│ BAC │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ BBY │ 3.634 % │ 3.773 % │ 4.216 % │ 1.757 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.003 % │ 0.0 % │ 0.0 % │ 0.001 % │ 0.0 % │ 0.0 % │
│ CVX │ 2.537 % │ 5.289 % │ 2.886 % │ 6.628 % │ 11.588 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 14.346 % │ 12.645 % │ 11.832 % │
│ GE │ 5.126 % │ 2.433 % │ 0.684 % │ 1.042 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.506 % │ 1.704 % │ 3.497 % │ 1.647 % │ 0.001 % │ 0.0 % │ 2.011 % │
│ HD │ 0.0 % │ 2.601 % │ 3.602 % │ 2.514 % │ 6.347 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.323 % │ 5.48 % │ 6.083 % │ 4.549 % │ 2.474 % │ 3.178 % │ 1.051 % │
│ JNJ │ 0.0 % │ 0.017 % │ 0.797 % │ 7.841 % │ 14.529 % │ 7.359 % │ 14.724 % │ 17.588 % │ 16.51 % │ 22.651 % │ 10.841 % │ 6.473 % │ 12.59 % │ 12.574 % │ 30.304 % │ 33.477 % │
│ JPM │ 6.409 % │ 10.765 % │ 10.995 % │ 7.026 % │ 3.402 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 4.915 % │ 7.891 % │ 5.921 % │ 9.788 % │ 1.38 % │ 3.776 % │ 1.918 % │
│ KO │ 49.958 % │ 26.25 % │ 22.85 % │ 18.299 % │ 12.377 % │ 24.085 % │ 15.424 % │ 13.414 % │ 15.026 % │ 0.0 % │ 4.651 % │ 9.889 % │ 10.025 % │ 7.446 % │ 6.847 % │ 6.673 % │
│ LLY │ 2.11 % │ 9.026 % │ 10.324 % │ 5.688 % │ 4.636 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 4.635 % │ 2.62 % │ 0.326 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ MRK │ 6.173 % │ 1.687 % │ 2.78 % │ 6.572 % │ 5.126 % │ 23.968 % │ 17.989 % │ 19.781 % │ 24.507 % │ 1.756 % │ 14.051 % │ 16.279 % │ 7.602 % │ 7.463 % │ 9.406 % │ 9.785 % │
│ MSFT │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.679 % │ 9.157 % │ 8.664 % │ 4.531 % │ 2.318 % │
│ PEP │ 8.202 % │ 9.381 % │ 11.103 % │ 12.631 % │ 5.716 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.021 % │ 4.802 % │ 3.953 % │ 4.767 % │ 10.725 % │ 14.384 % │ 13.194 % │
│ PFE │ 0.007 % │ 2.932 % │ 0.0 % │ 0.0 % │ 1.14 % │ 2.727 % │ 4.949 % │ 8.457 % │ 4.082 % │ 9.51 % │ 5.014 % │ 3.304 % │ 9.675 % │ 4.727 % │ 3.086 % │ 3.253 % │
│ PG │ 5.488 % │ 8.859 % │ 9.692 % │ 8.159 % │ 8.183 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 28.456 % │ 26.017 % │ 22.985 % │ 14.989 % │ 10.624 % │ 0.0 % │ 3.525 % │
│ RRC │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.479 % │ 1.506 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.481 % │ 0.45 % │ 1.772 % │ 0.344 % │ 0.0 % │ 0.0 % │
│ UNH │ 0.0 % │ 2.356 % │ 3.744 % │ 2.8 % │ 4.091 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.328 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ WMT │ 2.094 % │ 12.826 % │ 11.984 % │ 18.292 % │ 22.379 % │ 37.987 % │ 45.238 % │ 39.163 % │ 39.875 % │ 26.226 % │ 14.114 % │ 15.243 % │ 9.71 % │ 16.504 % │ 11.103 % │ 9.596 % │
│ XOM │ 5.155 % │ 1.805 % │ 4.342 % │ 0.751 % │ 0.005 % │ 2.368 % │ 1.676 % │ 1.597 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.027 % │ 0.739 % │ 1.368 % │
└─────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┘Some assets change weight by more than 20 % between two consecutive folds. Four tools limit this when their reference weights are not fixed: a turnover constraint, a turnover fee, a turnover risk measure and a tracking of the previous weights. We use a turnover constraint that lets no asset's weight change by more than 2 % from one fold to the next. Equal weights are the reference of the first fold. A Turnover has fixed = false by default, and a fold then takes the weights of the fold before as its reference.
N = size(rd.X, 2)tn = Turnover(; w = range(; start = 1 / N, stop = 1 / N, length = N), val = 0.02)Turnover
w ┼ 20-element StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}
val ┼ Float64: 0.02
fixed ┴ Bool: false
We add the constraint to the optimiser and run the walk-forward again.
mr = MeanRisk(; opt = JuMPOptimiser(; slv = slv, tn = tn))idx_tn_walkforward_pred = cross_val_predict(mr, rd, idx_walk_forward)MultiPeriodPredictionResult
pred ┼ 16-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 1008-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 1008-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
The table gives the weights again, so you can compare every column with the one before it.
pretty_table(hcat(DataFrame(:tickers => rd.nx), DataFrame(reduce(hcat, getproperty.(idx_tn_walkforward_pred.res, :w)), Symbol.(1:16))); formatters = [resfmt])┌─────────┬─────────┬─────────┬─────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┐
│ tickers │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │ 9 │ 10 │ 11 │ 12 │ 13 │ 14 │ 15 │ 16 │
│ String │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├─────────┼─────────┼─────────┼─────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┤
│ AAPL │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.99 % │ 1.562 % │ 0.0 % │ 0.0 % │
│ AMD │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.688 % │ 3.652 % │ 3.009 % │ 1.855 % │ 0.0 % │ 0.0 % │
│ BAC │ 3.0 % │ 1.354 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.453 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ BBY │ 3.0 % │ 2.903 % │ 3.93 % │ 1.93 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.901 % │ 0.242 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ CVX │ 3.0 % │ 4.681 % │ 3.399 % │ 5.398 % │ 7.398 % │ 5.398 % │ 3.398 % │ 1.398 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 2.0 % │ 4.0 % │ 6.0 % │
│ GE │ 4.288 % │ 2.345 % │ 0.808 % │ 0.718 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.731 % │ 2.542 % │ 2.005 % │ 1.328 % │ 1.373 % │ 1.891 % │
│ HD │ 3.0 % │ 5.0 % │ 5.948 % │ 3.948 % │ 5.947 % │ 3.947 % │ 1.947 % │ 0.0 % │ 0.0 % │ 0.004 % │ 2.004 % │ 4.003 % │ 3.674 % │ 3.681 % │ 3.798 % │ 1.799 % │
│ JNJ │ 7.0 % │ 5.987 % │ 4.334 % │ 6.334 % │ 8.333 % │ 10.333 % │ 12.333 % │ 14.333 % │ 16.333 % │ 18.329 % │ 16.329 % │ 14.329 % │ 12.331 % │ 14.329 % │ 16.329 % │ 18.329 % │
│ JPM │ 5.712 % │ 7.712 % │ 9.712 % │ 8.063 % │ 6.152 % │ 4.152 % │ 2.152 % │ 0.152 % │ 0.0 % │ 0.0 % │ 2.0 % │ 3.992 % │ 5.991 % │ 3.993 % │ 5.313 % │ 3.313 % │
│ KO │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 12.391 % │ 14.391 % │ 16.391 % │ 18.391 % │ 19.668 % │ 17.668 % │ 15.668 % │ 13.668 % │ 11.67 % │ 12.063 % │ 10.097 % │ 11.104 % │
│ LLY │ 7.0 % │ 8.987 % │ 9.091 % │ 7.091 % │ 5.423 % │ 3.786 % │ 4.658 % │ 2.674 % │ 0.674 % │ 2.673 % │ 1.973 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ MRK │ 7.0 % │ 8.75 % │ 6.751 % │ 7.327 % │ 6.199 % │ 8.199 % │ 10.199 % │ 12.199 % │ 14.199 % │ 12.199 % │ 13.137 % │ 13.866 % │ 11.866 % │ 9.867 % │ 11.867 % │ 12.331 % │
│ MSFT │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.907 % │ 2.906 % │ 4.906 % │ 4.626 % │ 2.626 % │
│ PEP │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 11.015 % │ 9.015 % │ 7.015 % │ 5.015 % │ 3.015 % │ 1.015 % │ 0.0 % │ 0.0 % │ 1.57 % │ 3.569 % │ 5.568 % │ 7.568 % │
│ PFE │ 7.0 % │ 5.0 % │ 3.0 % │ 1.001 % │ 2.12 % │ 4.12 % │ 6.12 % │ 8.12 % │ 10.12 % │ 8.12 % │ 6.12 % │ 4.12 % │ 6.12 % │ 5.591 % │ 5.773 % │ 5.981 % │
│ PG │ 7.0 % │ 9.0 % │ 11.0 % │ 12.193 % │ 10.297 % │ 12.297 % │ 14.297 % │ 14.866 % │ 12.993 % │ 14.993 % │ 15.645 % │ 17.331 % │ 15.333 % │ 13.334 % │ 11.334 % │ 9.335 % │
│ RRC │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.347 % │ 0.872 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.353 % │ 0.347 % │ 1.469 % │ 2.196 % │ 0.197 % │ 0.0 % │
│ UNH │ 3.0 % │ 2.281 % │ 3.172 % │ 3.142 % │ 5.14 % │ 3.14 % │ 1.14 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.066 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ WMT │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 14.999 % │ 16.999 % │ 18.999 % │ 20.999 % │ 22.999 % │ 24.999 % │ 22.999 % │ 21.0 % │ 19.0 % │ 17.724 % │ 15.725 % │ 13.725 % │
│ XOM │ 7.0 % │ 5.0 % │ 5.856 % │ 3.856 % │ 4.238 % │ 3.351 % │ 1.351 % │ 1.854 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 2.0 % │ 4.0 % │ 5.999 % │
└─────────┴─────────┴─────────┴─────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┘The constraint limits the change of each asset, not the total. The turnover of a rebalance is the sum of the absolute changes over the 20 assets, so it can be larger than 2 %.
The cumulative returns with the turnover constraint.
plot_portfolio_cumulative_returns(idx_tn_walkforward_pred)
The turnover at every rebalance.
plot_turnover(idx_tn_walkforward_pred)
The distribution of the weight of every asset over the folds.
plot_weight_stability(idx_tn_walkforward_pred)
2.3.2 DateWalkForward
DateWalkForward works like IndexWalkForward, but you give the windows in dates. Use it to align the windows with calendar periods, such as fiscal years or quarters.
We use lastdayofmonth from Dates to define an adjuster. It moves the dates of a range to the last day of their month, and drops a month end that comes after the last date of the range.
function ldm(x) val = lastdayofmonth.(x) while !isempty(val) if val[end] > x[end] val = val[1:(end - 1)] else break end end return valend;The first argument, the training size, is an integer count of steps of the date range, or a Period or CompoundPeriod from Dates. The second argument, the test size, is an integer count of steps. period sets the step of the date range, and adjuster changes the range. Here the step is one month and the adjuster moves the dates to month ends, so we train on 12 months and test on 3.
The scheme maps each date of the range to a row. A date that is a timestamp maps to its own row. A date between two timestamps maps to the earlier one when previous = true, and to the later one when previous = false. A month end can fall on a weekend, so we set previous = true. A test window then starts on the last trading day of a month.
date_walk_forward = DateWalkForward(12, 3; period = Month(1), adjuster = ldm, previous = true)DateWalkForward
train_size ┼ Int64: 12
test_size ┼ Int64: 3
period ┼ Month: Month(1)
period_offset ┼ nothing
purged_size ┼ Int64: 0
adjuster ┼ typeof(Main.ldm): Main.ldm
previous ┼ Bool: true
expand_train ┼ Bool: false
reduce_test ┼ Bool: false
wd ┼ nothing
pws ┼ nothing
fa ┼ nothing
store_weight_path ┼ Bool: false
strict ┴ Bool: false
We look at the splits of the new scheme.
date_walk_forward_res = split(date_walk_forward, rd)show(date_walk_forward_res.train_idx)show(date_walk_forward_res.test_idx)UnitRange{Int64}[3:253, 64:314, 128:377, 191:441, 254:505, 315:567, 378:630, 442:694, 506:758, 568:819, 631:882, 695:946, 759:1010, 820:1072, 883:1134]UnitRange{Int64}[254:314, 315:377, 378:441, 442:505, 506:567, 568:630, 631:694, 695:758, 759:819, 820:882, 883:946, 947:1010, 1011:1072, 1073:1134, 1135:1198]We run the new scheme with the turnover constraint.
date_tn_walkforward_pred = cross_val_predict(mr, rd, date_walk_forward)MultiPeriodPredictionResult
pred ┼ 15-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 945-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ne ┼ nothing
│ E ┼ nothing
│ ts ┼ 945-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
The table has one column per fold.
pretty_table(hcat(DataFrame(:tickers => rd.nx), DataFrame(reduce(hcat, getproperty.(date_tn_walkforward_pred.res, :w)), Symbol.(1:15))); formatters = [resfmt])┌─────────┬─────────┬─────────┬─────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┬──────────┐
│ tickers │ 1 │ 2 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 │ 9 │ 10 │ 11 │ 12 │ 13 │ 14 │ 15 │
│ String │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├─────────┼─────────┼─────────┼─────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┼──────────┤
│ AAPL │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.913 % │ 1.378 % │ 0.0 % │
│ AMD │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.699 % │ 3.606 % │ 2.962 % │ 1.82 % │ 0.0 % │
│ BAC │ 3.0 % │ 1.388 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.489 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ BBY │ 3.0 % │ 2.859 % │ 4.005 % │ 2.005 % │ 0.005 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.881 % │ 0.089 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ CVX │ 3.0 % │ 4.56 % │ 3.364 % │ 5.364 % │ 7.364 % │ 5.364 % │ 3.364 % │ 1.364 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 2.0 % │ 4.0 % │
│ GE │ 4.296 % │ 2.36 % │ 0.788 % │ 0.688 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.747 % │ 2.692 % │ 2.056 % │ 1.329 % │ 1.372 % │
│ HD │ 3.0 % │ 4.999 % │ 5.964 % │ 3.965 % │ 5.964 % │ 3.965 % │ 1.965 % │ 0.0 % │ 0.0 % │ 0.007 % │ 2.007 % │ 4.007 % │ 3.693 % │ 3.743 % │ 3.836 % │
│ JNJ │ 7.0 % │ 5.998 % │ 4.248 % │ 6.247 % │ 8.247 % │ 10.247 % │ 12.247 % │ 14.247 % │ 16.247 % │ 18.24 % │ 16.24 % │ 14.24 % │ 12.24 % │ 14.239 % │ 16.239 % │
│ JPM │ 5.704 % │ 7.704 % │ 9.703 % │ 7.982 % │ 6.127 % │ 4.127 % │ 2.127 % │ 0.127 % │ 0.0 % │ 0.0 % │ 2.0 % │ 4.0 % │ 5.999 % │ 4.0 % │ 5.364 % │
│ KO │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 12.603 % │ 14.603 % │ 16.603 % │ 18.603 % │ 19.636 % │ 17.636 % │ 15.636 % │ 13.636 % │ 11.637 % │ 12.064 % │ 10.212 % │
│ LLY │ 7.0 % │ 8.931 % │ 9.187 % │ 7.187 % │ 5.495 % │ 3.87 % │ 4.765 % │ 2.766 % │ 0.766 % │ 2.765 % │ 1.988 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │
│ MRK │ 7.0 % │ 8.699 % │ 6.7 % │ 7.22 % │ 6.239 % │ 8.239 % │ 10.239 % │ 12.239 % │ 14.239 % │ 12.24 % │ 13.254 % │ 13.581 % │ 11.581 % │ 9.586 % │ 11.586 % │
│ MSFT │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.271 % │ 3.271 % │ 5.27 % │ 4.627 % │
│ PEP │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 11.002 % │ 9.002 % │ 7.002 % │ 5.002 % │ 3.002 % │ 1.002 % │ 0.0 % │ 0.0 % │ 1.564 % │ 3.563 % │ 5.563 % │
│ PFE │ 7.0 % │ 5.001 % │ 3.001 % │ 1.001 % │ 1.961 % │ 3.961 % │ 5.961 % │ 7.961 % │ 9.961 % │ 7.961 % │ 5.961 % │ 3.961 % │ 5.961 % │ 5.646 % │ 5.841 % │
│ PG │ 7.0 % │ 9.0 % │ 11.0 % │ 12.222 % │ 10.225 % │ 12.225 % │ 14.225 % │ 14.905 % │ 13.149 % │ 15.149 % │ 15.746 % │ 17.518 % │ 15.518 % │ 13.519 % │ 11.519 % │
│ RRC │ 3.0 % │ 1.0 % │ 0.0 % │ 0.0 % │ 0.4 % │ 0.895 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.351 % │ 0.4 % │ 1.493 % │ 2.222 % │ 0.223 % │
│ UNH │ 3.0 % │ 2.501 % │ 3.016 % │ 3.094 % │ 5.094 % │ 3.094 % │ 1.094 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 1.114 % │ 0.0 % │ 0.0 % │
│ WMT │ 7.0 % │ 9.0 % │ 11.0 % │ 13.0 % │ 15.0 % │ 17.0 % │ 19.0 % │ 21.0 % │ 23.0 % │ 24.999 % │ 22.999 % │ 20.999 % │ 18.999 % │ 17.619 % │ 15.619 % │
│ XOM │ 7.0 % │ 5.001 % │ 6.023 % │ 4.024 % │ 4.273 % │ 3.408 % │ 1.408 % │ 1.787 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 0.0 % │ 2.0 % │ 4.0 % │
└─────────┴─────────┴─────────┴─────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┴──────────┘The cumulative returns over the month-end windows.
plot_portfolio_cumulative_returns(date_tn_walkforward_pred)
The turnover at every month-end rebalance.
plot_turnover(date_tn_walkforward_pred)
The date walk-forward sizes its windows in months and puts their bounds on month ends. The index walk-forward sizes them in rows. Because twelve months are about 252 trading days, the training windows of the two schemes cover almost the same dates. The same 2 % limit applies to both. To see how far the weights move, compare this table with the second table of section 2.3.1. The date walk-forward has 15 folds and the index walk-forward has 16. Compare the weights of an asset over the folds, not column by column.
MultipleRandomised is one more scheme. It runs a walk-forward on random subsets of the assets, and section 5.2 of the pipelines example uses it.
This page was generated using Literate.jl.