Optimized Indexation

What it is

A Financial Economics Institute research project advised by Prof. Benjamin Gillen. The question: can portfolio optimization scale to large asset universes — where a full covariance optimization is ill-conditioned — while still beating naive benchmarks like equal-weight (1/N)? The pipeline is engineered so every result is byte-for-byte reproducible: a single JSON config fully specifies a run, and a SHA-256 "run key" derived from it guarantees the same inputs always produce the same output.

How it works

A six-stage, checkpointed pipeline — data → universe → moments → portfolios → backtest → metrics. Each stage reads the previous stage’s artifact and emits its own schema-versioned JSON, so runs resume safely and stay auditable.

1. Universe & moments — rank stocks by market cap into top-N universes and estimate return means and covariances over a rolling 120-month window.
2. Portfolio construction — solve long-only Global Minimum Variance (minimize wᵀΣw) and Mean-Variance (maximize μᵀw − (λ/2)·wᵀΣw) as convex programs via CVXPY with the OSQP solver, subject to fully-invested, no-short constraints.
3. Subset optimization — to scale to large universes, deterministically sample B random subsets of size ñ, solve each small sub-problem, lift the weights back to the full universe, then average and project onto the long-only simplex.
4. Backtest & metrics — a no-look-ahead engine with delisting-adjusted returns rebalances annually and reports annualized mean/volatility, Sharpe, certainty-equivalent, and max drawdown against 1/N and value-weighted baselines.

Technical stack

Results & validation

On the Ken French 48-industry dataset (1973–2023, annual rebalancing), the long-only Global Minimum Variance portfolio beat the 1/N equal-weight benchmark on both risk-adjusted return and drawdown: at a 30-name universe, Sharpe 1.03 vs 0.76 and maximum drawdown −33% vs −53%. Reproducibility is test-enforced — identical configs produce byte-identical outputs — and a full 51-year CRSP-scale run completed on the Rutgers Hopper cluster in under an hour on a single CPU. TODO(juhi): add the CRSP paper-baseline results (N = 100 / 250 / 500) once they are published to the repo.

Limitations

This is an actively-developed research scaffold (v0.1). The subset-optimization method is implemented but its large-universe results are still being finalized; Direct Index Optimization and turnover metrics are specified but not yet implemented; the covariance estimator is the sample estimator; and the headline CRSP paper-baseline numbers currently live on the HPC cluster rather than in the public repo.

Frequently asked questions

What is Optimized Indexation?

A reproducible research pipeline for building and backtesting large-universe stock portfolios, developed at the Financial Economics Institute under Prof. Benjamin Gillen. It solves long-only minimum-variance and mean-variance problems with CVXPY/OSQP and includes a subset-optimization method for scaling to large universes.

What is the subset-optimization approach?

Instead of one large, ill-conditioned covariance optimization over the whole universe, it deterministically samples many small subsets of assets, solves each sub-problem, lifts the weights back to the full universe, and averages them onto the long-only simplex — a route to scaling optimization to large N.

What were the results?

On the Ken French 48-industry data (1973–2023, annual rebalancing), long-only Global Minimum Variance beat the 1/N benchmark: Sharpe 1.03 vs 0.76 and max drawdown −33% vs −53% at a 30-name universe. Every result is byte-for-byte reproducible via config-derived run keys.