Interactive research paper
Realistic evaluation of DRL portfolio claims
An interactive paper on how conclusions from a TD3 model change when evaluated with costs, explicit cash, matched benchmarks and statistical validation.
Evaluation problem
An attractive backtest can create false confidence
Apparent result
A DRL portfolio candidate can rank well after training and produce an attractive backtest.
Conceptual illustration
Sources of overstatement
The credibility of the result depends on the evaluation design around the algorithm.
- Weak or mismatched benchmarks
- Simplified transaction costs
- Ambiguous cash treatment
- Candidate-search bias
- Limited statistical validation
Therefore, the thesis evaluates the claims around TD3.
Evidence hierarchy
Three increasingly strong claims
Research question
Can a falsification-oriented evaluation framework distinguish ranking competitiveness, statistical credibility and practical feasibility in TD3-based cross-asset portfolio allocation?
Performs well against matched benchmarks?
Survives uncertainty and data-snooping controls?
Remains plausible under mandates and execution stress?
Interpretive rule
Ranking first does not prove statistical superiority, and statistical superiority would still not guarantee deployability.
Test bed
Asset universe
The universe is deliberately compact. It does not try to represent the whole global market; it creates an interpretable test bed.
SPY
01Equity / growth risk
Adds growth beta and broad US equity risk.
TLT
02Duration / interest-rate risk
Adds long-duration Treasury and rate sensitivity.
GLD
03Real safe-haven / hard-asset exposure
Tests defensive diversification and hard-asset exposure.
BTC-USD
04Digital alternative / speculative convexity
Adds speculative convexity and extreme volatility.
CASH / BIL
05Defensive liquidity / optionality
Serves as the defensive sleeve and mandate control.
TD3 mechanism
How TD3 produces portfolio allocations
TD3 is used as a continuous-control learner; the contribution is not a new algorithm variant.
Actor network
Twin critics Q1 / Q2
Evaluation framework
The candidate faces a demanding protocol
Temporal separation and comparison under matching assumptions are part of the test.
- 01Data and feature families
- 02TD3 candidate training
- 03Walk-forward validation and out-of-sample testing
- 04Matched deterministic benchmarks
- 05Bootstrap uncertainty and White Reality Check
- 06Regime, mandate, Pareto, execution and budget diagnostics
Temporal separation is part of the test.
Ranking evidence
Competitiveness survives under matched assumptions
The table compares benchmark-matched TD3 candidates with Trend SPY/CASH, the clean comparator used in statistical validation.
Zero-CASH
| Metric | TD3 | Trend SPY/CASH |
|---|---|---|
| Annualized return: | 8.69% | 9.79% |
| Sharpe ratio: | 0.923 | 0.880 |
| Maximum drawdown: | -10.4% | -17.8% |
BIL-CASH
| Metric | TD3 | Trend SPY/CASH |
|---|---|---|
| Annualized return: | 10.65% | 10.24% |
| Sharpe ratio: | 1.141 | 0.917 |
| Maximum drawdown: | -10.3% | -17.3% |
Cash treatment changes candidate selection and economic interpretation.
Candidates are selected through diagnostic rankings; economic interpretation relies on standard metrics and statistical validation.
Statistical validation
The intervals do not establish superiority
Bootstrap Sharpe-difference intervals cross zero in both cash protocols.
Bootstrap Sharpe-difference intervals
TD3 remains ranking-competitive, but statistical superiority over the clean benchmark is not established.
The intervals are pairwise evidence; White Reality Check is search-adjusted candidate-set evidence.
Practical feasibility
Execution stress and mandates change the reading
The sensitivity check is post-training: it does not retrain TD3, alter selection or create new winners.
Sharpe degradation under spread stress
Sharpe change · ≈ -0.0132 in the paper
Stress: additional half-spreads of 3/5/5/50/0 bps for SPY/TLT/GLD/BTC-USD/CASH, volatility-adjusted with β = 0.5; post-training sensitivity, with no retraining.
Claim survival
Only some claims survive
The final reading separates competitiveness, statistical inference and practical feasibility.
| Claim | Reading | Evidence boundary |
|---|---|---|
| Ranking competitiveness | Supported | Selected candidates remain competitive against the matched comparator; return leadership depends on cash. |
| Statistical superiority | Not supported | Bootstrap intervals include zero and White Reality Check does not reject. |
| Cash assumption matters | Supported | Zero-CASH and BIL-CASH select different candidates. |
| Execution assumptions matter | Supported | TD3 histories degrade more under spread stress. |
| Conservative mandate feasibility | Not supported | No strategy passes every conservative hard filter. |
| Pareto competitiveness | Conditional | Candidates sit on or near relevant frontiers, depending on profile and trade-off. |
| Deterministic benchmarks remain credible | Supported | Trend SPY/CASH remains a strong clean comparator. |
Methodological contribution
From an attractive backtest to a defensible claim
TD3 is the case study; the falsification-oriented evaluation framework is the contribution.
- Explicit cash treatment
- Transaction costs
- Matched deterministic benchmarks
- Bootstrap uncertainty
- Data-snooping control
- Practical feasibility filters
The contribution is methodological rather than algorithmic: it does not propose a new TD3 variant, deployable alpha or universal superiority.
Final answer
Ranking competitiveness
Selected TD3 candidates remain competitive against matched deterministic benchmarks on risk-adjusted performance.
Statistical superiority
Bootstrap intervals cross zero and White Reality Check does not reject the search-adjusted null.
Practical feasibility
Cash, execution stress, mandates, Pareto trade-offs and market regimes materially affect interpretation.
TD3 is a competitive research candidate for dynamic allocation—not a statistically dominant trading strategy.
Main result: Realistic evaluation materially changes what can responsibly be claimed from favourable DRL portfolio experiments.
Interactive appendix
Methods, limits and traceability
These notes extend the protocol without competing with the main evidence sequence.
Net returns include asset-specific costsOpenClose
SPY, TLT and GLD use 2 bps; BTC-USD uses 10 bps; CASH uses 0 bps under Zero-CASH; BIL uses 2 bps under BIL-CASH. Costs affect realized returns and the learning signal.
Cash is not a footnoteOpenClose
Zero-CASH is a synthetic zero-return, zero-cost sleeve. BIL-CASH uses an investable short-term Treasury proxy. They are different investable environments and are evaluated separately.
TD3 candidate design and feature familiesOpenClose
The search evaluates V2-V8 families, including clean macro, GARCH-style volatility, no-volatility ablations and EWMA/GARCH combinations. PCA was audited but is not the default representation.
Candidate search is controlled by protocolOpenClose
The search combines feature families, caps, seeds, folds and cash. TD3-only selection is separated from benchmark-comparison selection.
Diagnostic selection scoreOpenClose
Mandate-aware and robust scores are diagnostic summaries. They are not superiority tests, calibrated probabilities or proof of deployability.
Benchmark familiesOpenClose
The set includes buy-and-hold, equal weight, 60/40, defensive rules, momentum, rolling Markowitz, minimum variance, inverse volatility and Trend SPY/CASH under matching assumptions.
Walk-forward evaluationOpenClose
Candidates are trained only inside each training window and read out of sample. Four temporal folds reduce dependence on one split but do not create independent market histories.
Bootstrap and White Reality CheckOpenClose
Bootstrap quantifies pairwise uncertainty in the Sharpe difference. WRC provides search-adjusted candidate-set evidence. Neither replaces economic interpretation.
TD3 internal update loopOpenClose
The actor proposes continuous actions; two critics estimate value; target networks, smoothing and delayed updates seek to reduce instability and overestimation.
Reward designOpenClose
Reward begins with net return and includes an active drawdown penalty. Turnover is penalized economically through costs; concentration is evaluated outside the reward.
Feasible-action projectionOpenClose
The raw action is projected onto the simplex: non-negative weights that sum to one. The same feasible action is executed and stored for learning.
Regime analysisOpenClose
TD3 leads selected Zero-CASH slices, while benchmarks and momentum rules win many regimes, especially under BIL-CASH. There is no all-weather dominance.
Mandate filtersOpenClose
Conservative, moderate and aggressive profiles apply drawdown, volatility, effective-diversification and turnover limits. No strategy passes the conservative hard-filter layer.
Training-budget convergenceOpenClose
The check compares 30, 60, 100 and 150 episodes. It finds no obvious undertraining at 60 episodes but does not prove global budget optimality.
Scope limitsOpenClose
The universe is compact and US-centric. It excludes credit, real estate, broad commodities, international equities, taxes, market impact, liabilities and withdrawal needs.
Execution limitsOpenClose
Execution modeling is approximate: it does not model order-book depth, market impact, intraday liquidity, broker routing, custody or taxes.
ReproducibilityOpenClose
The paper documents data, environment, training, benchmark, statistical-validation and robustness scripts. Traceability supports inspection and replication; it does not replace live-forward evidence.
Code, scripts and reproducible evidence for the TD3 protocol.