Lyra Labs Docs Markdown

How a check grades

For agents
  • Lets you: Read a check's grade correctly and avoid burning trials.
  • Tools: test_run_check, alpha_simulate
  • Limit: Every distinct version you check counts as a trial of its family.
  • Limit: The sealed period is 365 days and cannot be changed.

The dot row

  • Each dot is one past case: a start date and a horizon.
  • A filled dot: the claim held over that case.
  • Lyra says it as "It held in 10 of 15 past cases."

The four grades

Grade Means
Awesome Idea Passed the bar on past cases, passed the sealed period, passed CPCV, on a screened universe
Good Idea Past cases agree, but not every test above passed
Not sure Past cases do not back it
Impossible to verify Too little history or no way to test it
  • Hand-picked lists and event dates grade at most Good Idea.
  • Only Awesome Idea can run a rule pod. Rule pods.
  • Past cases need at least 730 days of history.
  • An event check needs at least 10 events.

The sealed 365 days

  • The most recent 365 days are kept out of the test.
  • They open once per idea family and calendar period (the period ends at a quarter start).
  • After it opens, the next version of the same family sees it as spent.
  • A renamed family or a changed constant counts as the same family.

The bar rises with every try

  • Each version you check is a trial.
  • The bar uses the larger of the family's trial count and yours.
  • The score is deflated for the spread of the family's results.
  • From 4 trials on, the chance the best version is overfit (PBO) must be below 0.5.
  • Alphas: more than 8 versions of one shape inside 60 minutes stops simulation for the rest of the UTC day.

CPCV in plain words

  • History is cut into 10 blocks.
  • Every pair of 2 blocks takes a turn as the test, the rest as training.
  • Days next to a test block are dropped (purge = look-back plus horizon; embargo = the larger of 5 days and 1%).
  • Awesome Idea needs most paths positive and the worst 5% of path Sharpes above -0.5.

What you see

  • Grade and one line: "Good Idea. It held in 10 of 15 past cases."
  • Holding cost per month.
  • Versions tried.
  • A curve when the book holds at least 5 names on average, at most 250 points.