Methodology
Every threshold used to judge a backtest is on this page. If you disagree with a grade, you should be able to see exactly which number caused it and argue with the number rather than with us.
Harsh about evidence, not about results. Assay does not mark a strategy down for being unprofitable — the profit factor already says that. It marks down claims the data cannot support: too few trades, a win rate that clears break-even by a whisker, an edge that dies on costs.
Never fail something for behaving normally. Abandoning a working system during an ordinary losing run is the most common way traders destroy an edge. Losing runs are expected, their length is calculable, and a strategy is only marked down when its worst run is meaningfully worse than chance predicts.
Below 100 trades the check fails outright. Results at that size are dominated by luck, whatever the profit factor looks like.
Above that, Assay computes a t-statistic — how many standard errors the average trade sits above zero. It answers the question a raw profit factor cannot: could this result have happened by chance?
A win rate means nothing without the reward-to-risk beside it. Assay works out the rate this payoff needs just to stand still:
break-even win rate = average loss ÷ (average win + average loss)
A strategy with 2:1 winners needs to win 33.3% of the time to break even. One with 1:2 winners needs 66.7%. The check compares your actual win rate against that figure and grades the gap.
The expected longest run of losses in n trades is not intuitive, and almost nobody works it out before panicking. For a strategy that wins with probability p:
expected longest losing run = log(n × p) ÷ log(1 ÷ (1 − p))
A strategy winning 41% of the time over 647 trades should produce a worst run of about 11 losses through chance alone. A run of 13 is therefore unremarkable, and Assay says so rather than raising an alarm. The check grades the ratio of actual to expected.
Under 2.0 expected losses the estimate is too rough to grade, and the check reports the run without judging it.
Charging every trade the same fixed cost reduces total profit by that cost times the number of trades, so the charge that drags profit factor down to exactly 1.00 is simply the average profit per trade. That figure is the kill-point.
To express it in price terms without knowing your instrument, Assay derives the value of one point from your own data — a least-squares fit of P&L against price movement times position size — and compares the kill-point to a typical move.
If the arithmetic of your export does not reconcile exactly, your Pine script is already deducting commission or slippage. Assay detects that, says so, and treats the results as already net of costs.
Each check scores 2 for a pass, 1 for a warning, 0 for a failure. The total decides the letter — then three overrides apply, and they matter more than the arithmetic:
An A means the evidence behind a backtest holds together. It does not mean the strategy will make money. Assay judges a backtest, and a backtest is a claim about the past made by software you configured yourself. It cannot see look-ahead bias in your Pine, it cannot know whether you fitted the parameters to this exact data, and it has no opinion on whether the market will keep behaving the way it did.
A bad grade is strong evidence against a strategy. A good one is only the absence of the most common evidence against it.