Skip to main content
In short: a backtest replays your card over past days with fees and slippage. A paper run lets the approved card act on live prices with simulated fills. Neither moves money. A good backtest is evidence in proportion to how few variants were tried, and it never promises a return.

​
The backtest

When you agree to a card in chat, the agent saves it as a draft and starts the backtest straight away. It runs as a background job and takes minutes. The agent gives you a link to the report when it is ready. The defaults are fixed so that two reports compare. The measures warm up on the bars before the start date. You can ask for other settings, such as an earlier start or a walk-forward run, after you have seen the first report.

​
Reading the report

Read it in this order: the verdict, what the card can do, how much evidence there is, then the returns. The report carries ten items.
  1. The Sharpe ratio, in two definitions.
  2. Buy and hold over the same bars. This is the bar to clear.
  3. The fill and cost convention.
  4. Out-of-sample trades.
  5. How many variants were tried.
  6. The deflated Sharpe ratio.
  7. Where every constant came from.
  8. The data’s own flags.
  9. How each data product was made.
  10. What could not be tested.
Beat holding, after costs, and read the drawdown beside the return. A return without its deepest drawdown hides how much you would have lived through.

​
Out of sample

Zero out-of-sample trades means the report has no evidence out of sample. It does not mean the card is safe. Choose where out of sample starts before you look at the results. Every look at it while you adjust the card spends it. With a walk-forward run, judge the windows, not the whole span.

​
Variants tried

The system counts every backtest per owner and per card lineage, not per title. Renaming a card does not reset the count. Running the identical configuration twice does not raise it. Changing a threshold does. The count appears beside every result, and the approval page shows it too. A result from three principled variants is worth more than the best of two hundred. Change a card on purpose, never to shop for a better-looking number.

​
Flags to pass on

  • Bars before a venue’s own data begins are price-only, so costs are understated.
  • A feed marked revised is read as revised today, so the backtest knows more than a live card will.
  • A feed’s history starts when recording began, so a backtest on a newer feed covers a shorter span.
  • The list of what could not be tested limits what the verdict covers.

​
What has no backtest

Some card kinds have no backtest because the lab has no model for them: resting orders, cards that trade several assets, fee collection and repayment, supply, withdraw and send steps, cards that run once, and Binance cards. A card that only sells cannot be backtested either, because every backtest starts in USDC. The approval page says which case applies, and what you sign instead. See Going live.

​
Paper runs

A paper run uses the approved version of the card, live bars and live feeds, and moves no money.
  • Each decision is sized from the card’s own simulated balance and filled at the open of the next bar, with the venue’s fee, by the same fill model as the backtest.
  • Nothing is signed or submitted. A paper card has no way to hand a trade on.
  • Every paper firing is recorded and marked paper, apart from live firings. A paper run cannot be mistaken for, or change, what a live card did.
  • A paper run needs your signed approval, and the kill switch stops it like any other card.
  • Paper is optional and has no fixed length. Nothing counts how long a card ran on paper, and nothing requires paper before live.
Start a paper run from the approval page after you sign, or from the console’s start form on the kill switch page. Choose the approved version and “On paper”. The worker starts it within a few seconds. Read its firings next to the backtest. To go live from paper, stop the paper card and start the version live. That asks for your approval again.

​
Under the hood

This part is for readers who want the mechanism. A backtest is a background job. The fill and fee model is shared between the backtest and paper runs. The deflated Sharpe ratio discounts the best result for the number of variants tried. The report also names the data flags of every feed the card read.