The CoE Numerai Newsletter- Edition #9
April 1st 2022
Numercon Is Here
Taking place at 1pm PST on April 1, 2022 at the Scottish Rite Masonic Center in San Francisco, Numercon will be in full force today.
Speakers and Panelists:
Joey Krug Co-CIO at Pantera Capital, Co-Founder of Augur, an investor in and advisor to Numerai
Jonathan Larkin Kaggle Master, Managing Director and member of the investment team at Columbia Investment Management Co., LLC (CIMC), former CIO at Quantopian, former Global Head of Equities at Millennium
Howard L. Morgan co-founded Renaissance Technologies and First Round Capital, and is an investor in and advisor to Numerai (joining by video conference)
Richard Craib, Founder and CEO of Numerai
Anson Chu, Numerai CTO, formerly at Uber
Carlo Lepelaars, Data Science at CrowdCent and Kaggle Master presenting on NumerBlox
Jo-fai Chow, Data Scientist and 360 Selfie Guy, Numerati Dashboard
Kaggle Grandmasters in attendance
Bojan Tunguz, Quadruple Kaggle Grandmaster - currently ranked 27/179940
Rohan Rao / Vopani, Quadruple Kaggle Grandmaster, H2O.ai
Dmitry Larko, Chief Data Scientist at H2O.ai
Sanyam Bhutani of Chai Time Data Science, Weights & Biases and newly Kaggle Grandmaster
Walter Reade is a Kaggle GM and has worked at Kaggle as a Competitions Data Scientist since 2017
Three times Kaggle Grandmaster SRK
SMLY / zoi_trader - Kaggle Grandmaster and currently top of the Numerai Signals leaderboard
Peiyuan Liao, Kaggle Grandmaster, machine learning lead at Praxis Pioneering
Agenda: The Master Plan
Monopolize intelligence
Numerai vision, Richard Craib, Founder and CEO
Technical update: True Contribution, Michael Oliver, Minister of Research
Monopolize data
Data in a True Contribution World, Michael Phillips, Minister of Data
Synthetic Data, Michael Oliver, Minister of Research
Signals Vision, Richard Craib
Monopolize money
Deep-dive with Richard Craib
Panel with Joey Krug of Pantera Capital, Jonathan Larkin of Columbia Endowment, Richard Craib of Numerai
Fireside chat with Richard Craib and Howard Morgan
Decentralize the monopoly
Decentralization vision and update, Anson Chu, CTO of Numerai
Closing Remarks, Richard Craib
Read Numerai's full master plan (2017): https://medium.com/numerai/numerais-master-plan-1a00f133dba9
Community speakers include Carlo Lepelaars and Jason Rosenfeld: CrowdCent Numerblox, and Jo-Fai Chow of Numerati Dashboard and #memes fame
Signals: Information Coefficient (IC)
On March 15th 2022, Numerai released a new metric on Signals.
IC is defined as the spearman correlation of your unneutralized submission with raw returns.
You can't stake on IC, but it may give you new insights into the behavior of your models. Additionally, Numerai plans to add IC to diagnostics soon.
Some models to look at with interesting IC scores
- apprentice_key, a Numerai internal model that is built on features that we neutralize to. Its corr is close to zero, but its IC is quite high
- FactorOverflow, a user model that has a highly correlated IC and Corr over recent rounds
True Contribution Is Here
On March 22nd 2022, Numerai officially released TC(True Contribution) to the world.
True Contribution is computed by treating Numerai as an end-to-end artificial intelligence system. By computing the gradient of optimized portfolio returns with respect to the NMR staked on a signal using differentiable convex optimization layers, Numerai can now surface and incentivize signals making the largest intelligence contributions to their hedge fund.
Numerai is essentially creating a feedback loop designed to continuously incentivize the creation and submission of signals which make profitable alterations to their hedge fund and disincentivize all other signals. Every round of Numerai will become like another pass of backpropagation on the overall cybernetic system of Numerai. Feedback and error correction propagate through a layer of distributed AI models, a blockchain staking layer, a Meta Model, and a convex optimization. In other words, with every round of Numerai, Numerai takes a step closer towards becoming a Type IV hedge fund.
A Type IV hedge fund has a signal which no one can make profitable alterations to. A Type IV hedge fund wouldn’t just be the best hedge fund in the world, it would be a hedge fund where no other known signals anywhere in the universe could be combined to its signal to improve it. A Type IV hedge fund does not necessarily trade a perfect signal with perfect stock market prediction accuracy, it just means the signal it does trade is maximally good for all that is currently known. It has integrated all known signals perfectly.
A Type IV hedge fund would be like an alien super intelligence for the stock market. It would be a bit like the best possible version of DeepMind’s AlphaZero playing Go where no alterations to its game by humans (or older versions of AlphaZero or AlphaGo) could improve it.
To maximize backward compatibility while maximizing the impact of TC, starting April 9th users will be able to stake on (0x or 1x CORR) + (0x or 1x or 2x TC). Staking on MMC will be automatically discontinued on that date. So if you are currently staking on 1x CORR and 2x MMC, your stake will be 1x CORR only starting April 9th unless you also elect to stake on 1x TC or 2x TC. Numerai will not automatically convert any MMC stakes to TC stakes. TC staking will start as opt-in only. There will be no changes to the payout factor for the time being.
Forum post by michael.oliver 📝 https://forum.numer.ai/t/true-contribution-details/5128
Medium piece by richardcraib 🔮 https://medium.com/numerai/alien-stock-market-intelligence-numerais-true-contribution-6bc7652bd6ac
Follow up post demonstrating methods for directly optimizing metrics like FNC and TB200
FNCv3
On March 27th 2022, Numerai Released FNCv3
In the beginning Numerai created Corr and MMC
And the tournament was without accurate signal evaluation
And the spirit of the Metamodel moved upon the face of the waters
And Numerai said, Let there be TC and FNCv3: and there was TC and FNCv3
FNCv3 is a model's correlation with targets after neutralization to the 420 features in the medium subset of the V3 features
This medium subset is listed in `features.json`, which is included in the weekly zip
V4 Tournament Data
On March 28th 2022, A new page has been added to Numerai: https://numer.ai/data
It contains descriptions of the datasets and their files, along with download links. They have also included examples on how to use NumerAPI to download datasets via code.
The download buttons on the leaderboard will now bring you to this data page. "Download Legacy Data" brings you to the V2 datasets page, and "Download Data" brings you to the latest available dataset.
With is addition, a forum post was dropped on the same day: V4 Tournament Data Announcement
On April 5, there will be a new dataset available. It contains 141 new features, as well as targets available for all eras - including what was previously labeled “test”.
What’s New?
There are 141 new features included, for a total of 1191 features, plus targets available for all eras.
Test eras are now considered part of validation. This means there is a train.parquet, validation.parquet, and live.parquet file available.
Each live era will graduate to validation data, and targets are populated in the graduated eras as soon as possible.
The existing features have been slightly improved, and thus renamed. There will be a map between v3 and v4 features so that you can use your previous feature research on new data without starting over, if you wish.
The optional targets have also been slightly improved and so are named with a new pattern, eg. “target_jerome_v4_20”.
Submissions are now only required to have live predictions. Any other predictions submitted will be safely ignored. We will accept submissions from any version, as long as you have predictions for all of the live indices.
As always, neither the legacy (v2) or the current versions (v3) are being changed in any way, so any automated model you have will continue to work as always.
The example script repo will be updated with v4 versions of each example model. These models will be functionally identical, but will be updated to read from the v4 dataset.
Numerblox (Solid Numerai Pipelines)
On March 17th 2022, perfect_fit, along with jrai announced they are open sourcing numerblox, a Numerai library which they have been building and using internally at CrowdCent.
They built it mainly to simplify and scale their weekly inference pipelines, but are also using components of it for training models.
Vision:
The vision behind numerblox is that while the Numerai community builds models that may greatly differ from each other, CrowdCent’s inference pipelines often have a similar structure:
Download
Preprocess
Make predictions
Postprocess
Evaluate
Submit
Building good data pipelines is mostly a software engineering effort, which currently every Numerai participant has to basically build from scratch.
numerbloxis designed to simplify every step in this process. It allowed us to focus more on training Numerai models and less on software engineering.
Highlights:
NumerFrame: A data structure extending Pandas DataFrame to easily slice, split and chunk Numerai data.
Explanation notebook on NumerFrame
ModelPipeline: Combine preprocessors, models and postprocessors to build robust inference pipelines. Also works with most scikit-learn objects. This is because
numerbloxobjects follow a similar interface with familiar methods like.transformand.predict. They even have pipelines where a single Model consists of a large scikit-learn FeatureUnion.
Explanation notebook on pipeline construction
Evaluators: Compute all relevant Numerai metrics with two lines of code. For Numerai Classic and Signals.
See full forum post: Open-sourcing numerblox (Solid Numerai pipelines)
If you are interested in contributing to this project, have a look at CONTRIBUTING.MD
Numerai Self-Supervised Learning & Data Augmentation Projects
At the end of February 2022, Richard Craib tweeted:
I’m getting hints that Numerai could soon start trading stocks in the same way that AlphaZero plays Go i.e. at an alien super intelligence level.
Some of these hints come from a handful of new things I’ve seen like diffusion models, self-supervised learning and data augmentation.
I’m curious if any young AI people or data scientists are interested to help out by working directly with me remotely on these new methods.
Project 1: Download Numerai’s free data and produce a new representation of the data in the form of new features through any self-supervised method. Prove that models trained on these new features improve performance.
Project 2: Use Numerai’s data to produce new training examples of the data in the form of new rows through any new method. Prove that the models trained on these new rows Improve performance.
Present your work at NumerCon (Numerai’s conference on April 1 in SF). Win massive retroactive bounties for great breakthroughs shared publicly? Get interviewed by me and our chief scientist? Make history. DM me.
A forum post was created: Numerai Self-Supervised Learning & Data Augmentation Projects
Many in the community have flocked to this forum post to talk openly about their ideas for this project.
Join in!
A True Contribution Backtest
On Mar 27th 2022, user bvmcheckking created a forum post in anticipation of the TC rollout. Using Numer.ai’s GraphiQL API, bvmcheckking downloaded all round results of round 285 up to 304. The user then attempted to figure out if other users used the same model while submitting by assuming that somebody switched models if the correlation with meta model was higher than 5% from one round to the next. Using this data, bvmcheckking was able to do some simple checks to see whether TC correlates with other metrics.
Check out the in depth forum post here: A True Contribution backtest
Memes of the Month
by JRB
by ia_ai_Joe
by hellno
Payouts as USD (Unofficial improvement implemented)
On March 13th 2022, user wigglemuse re-created a mouse rollover script that automatically converts NMR to USD that was originally available but removed by Numerai
See how to do it here: Payouts as USD (Unofficial improvement implemented)
Gnosis Transactions March 2022
Mar 2nd -66.3 NMR Travel Stipend
Mar 6th -13.11 NMR Newsletter
Mar 7th -38 NMR Numerbay
Mar 10th -39.3 NMR Travel Stipend
Mar 13th -56 NMR Numerbay
Mar 14th -14.6 Youtube series
Mar 17th -1.81 Browser script for NMR/USD conversion
Mar 20th -50 NMR Numerbay
Mar 29th -45 NMR Numerbay
Numerbay Updates
2022-02-27 — 2022-03-06: The following includes changes from the theme-v2 branch (local, not yet pushed).
Finished new theme migration for order/sales and listings pages
In-progress new theme migration for artifacts management
Work plan is in post #2
Next week to continue the theme change, and start preparing for code release
2022-03-06 — 2022-03-13: The following includes changes from the theme-v2 branch (local, not yet pushed).
Finished new theme migration for core marketplace features
Work plan is in post #2
Next week to work on code optimization, testing, cleaning and release
2022-03-13 — 2022-03-20: The following includes changes from the theme-v2 branch and master branch.
Live deployment of new theme with core marketplace features
Started theme migration for non-core features
Work plan is in post #2
Next week to continue to work on theme migration for non-core features and docs
2022-03-20 — 2022-03-27: The following includes changes from the theme-v2 branch and master branch.
Achieved feature parity for the new theme migration (except for product review feature)
Updated docs for the new theme
Other bug fixes
Work plan is in post #2
Next week to continue to improve the UI, and start to work on webhook support for submission and integration with numerblox
Other News
Check out some of the new forum posts from March 2022:
Payments dashboard including currencies by quantized
Open Source Datasets for Numerai Signals by thomasxthomas
Tokenomics, where staked NMR comes from by nyuton
Disclaimer: This is not an official Numerai newsletter. It is sponsored by the Numerai CoE, a decentralized autonomous organization. Every effort is made to provide accurate and complete information but there is no claims, promises or guarantees about the accuracy of the contents.