The CoE Numerai Newsletter- Edition #12
July 1st, 2022
Happy Birthday CoE Numerai Newsletter!
Today issue #12 is released. I want to thank everyone so much for the opportunity to work on this ongoing project for the community. I hope that everyone who has subscribed has found great value to these monthly newsletters. Looking forward to the future!
A Wild Month for NMR
What started out as a continuation of pain in NMR performance along with the entire crypto market has ended up turning itself around in the last several days of June. This month we saw fresh new NMR lows with some exchanges seeing prices below $7.50. In the last two days of the month we saw very heavy volume pushing the price of NMR up to over $30 followed by fairly stable price action around $27. This represents a 112% price increase for the month and a 200% increase in price over the last 2 days alone. We have seen a 200% increase in 24hr volume and NMR is now currently ranked in the top 150 tokens! So far there is no apparent news to support the move. #decorrelateNMR
Official Numerai Announcements
On June 14th, 2022 Numerai released True Contribution for Numerai Signals.
True Contribution computes the value of adding a new signal into Numerai’s Meta Model. Numerai has now built True Contribution into Signals. It takes the signal you’ve submitted to Numerai Signals and calculates its True Contribution with respect to all all stake-weighted predictions in Numerai’s Meta Model (including Tournament and Signals; Numerai Tournament TC will not change).
They prepare each signal as follows:
Percent-rank the signal with no tie-breaking
fill missing predictions with 0.5
Rank again with no tie-breaking
normalize into a gaussian distribution (we do this for Numerai predictions too)
include all signals submitted to Numerai Signals in the same Meta Model as Numerai predictions at their current stake value.
The 20-round Signals TC Leaderboard (rounds 291 - 310) has moderate correlation with the Corr and MMC leaderboards, but lower correlation with the IC leaderboard:
corr_20d_rep 0.400512
mmc_20d_rep 0.366898
ic_rep 0.079883
Along with its relationship to existing metrics, Signals TC could be easy to optimize for if Numerai starts showing something called RIC - the Residual Information Coefficient - which is a signals correlation with residual returns. Stay tuned for updates on RIC, medal announcements, and staking.
On June 15th, 2022 Numerai Announced an updated payout calculation (all tournaments) that applies the payout factor before clipping. This change takes effect immediately.
New payout calculation:
payout = at_risk_stake * MAX(-0.25, MIN(0.25, payout_factor * (x_score * x_multiplier + y_score * y_multiplier)))
On June 17th, 2022 Numerai announced that 3xTC staking will be available on the Numerai Tournament when Round 321 opens
On June 28th, 2022 Numerai announced the date of their next fireside chat.
July 7th 2022 at 09:00 PST // 16:00 UTC
Slido: http://sli.do/season7_q3
Zoom registration: https://us02web.zoom.us/webinar/register/WN_au__Y-aTQNmEH48nFOBJgg
Numerai is trying the webinar format this time to ensure no unexpected or unwanted interruptions :)
Memes of the Month
by quantverse
by ia_ai_joe
Proposal RNumerai Rewrite
theomniacs placed a proposal within the CoE forums stating:
Fun Fact: Did you know that the Rnumerai package on CRAN has been downloaded over 23,000 times? Since its debut in 2017, it has received some solid praise, adoption and appreciation.
Problem: Despite its darling status among R tournament participants, RNumerai is in need of some serious love. It hasn’t had a substantial rewrite in nearly 5 years…and “the streets” are begging for a new version…
Proposal: The Omni Analytics team will do a complete rewrite of the RNumerai package to better support automated model submission and return the package to its former glory. This will include code refactoring, improved documentation and the incorporation of the new model assessment metrics into the package.
Deliverable: A fully updated and functioning package hosted on both Github and CRAN.
Compensation: $5,000 in $NMR (50 man-hrs @ $100/hr) with 50% dispersed at the start and 50% upon successful re-submission to CRAN.
After some discussion about fixed rates of $50/hr with possible retro bounties, the CoE successfully funded fully upfront for the project. We look forward to the rewrite!
[Review] Estimated NumerBay Impact
On June 17th, 2022 Restrading gave everyone a glimpse into the impact of Numerbay.
Here is what he stated:
94475 NMR are staked using predictions bought on NumerBay as of round 320, accounting for 9.6% of total stake in the tournaments (5.9% in Numerai main tournament and 24.5% in Signals)
Approach for Estimation
For orders using auto-submission, the destination model’s stake for the round is used. For orders not using auto-submission, the total stake of buyers’ models with matching file name is used. Multi-round orders are flattened before estimation.
This is an under-estimation of NumerBay’s total impact
The estimation does not include:
Users who used auto-submission but also submitted files to other slots
Users who had insufficient API permissions for NumerBay to perform file matching
Users who used files for other purposes such as ensembling with their own
Sales Leaderboard
A sales leaderboard is now available here
Other Usage Stats
Number of users: 242
Number of listings: 363
Total quantity sold: 847
Total sales value: 1560.55 NMR
Average unique visitors per day: 225
Pain Points and Potential Improvements
Return chasing vs. diversification: most purchases are concentrated on the top models at any point in time which suggests return-chasing behaviors. It would be better to both the buyers and the tournament as a whole to encourage more diversification. Tools need to be created to guide users towards models that not necessarily maximize past returns but complement well with their portfolios.
Portfolio management: Many users buy more than 1 models per round and weight stakes across them. Tools can be created to track their performance and to aid decisions on portfolio weights.
Auto-submission limitation: The auto-submission function on NumerBay currently does not allow changes after purchase. This makes multi-round purchases difficult to manage. Soon support for per-round adjustment of submission destinations will be added for multi-round orders.
Still web2: NumerBay relies heavily on Numerai’s API and is therefore still largely a web2 app despite using MetaMask etc for other purposes such as encryption.
Lack of refund mechanism: There is no automatic refund mechanism for missing submissions. For obvious reasons NumerBay does not hold the payments. In future as NumerBay migrates to web3 it could be possible to introduce some on-chain solution for automatic conflict resolution
Reward for modelers vs stakers: It is not hard to see from the sales values and estimated stake that the two are out of proportion. This may become better when direct staking and profit sharing are introduced in future.
See the full forum post with graphs here
Numerbay Updates
2022-05-22 — 2022-06-05:
Added more stats
Pending orders can be canceled
Uploads will be blocked when tournament round is not open
Coupon min spend will include all spend on the same seller’s products for the current round, post discount price will be capped such that min spend will always be respected
Other bug fixes and improvements (See list of closed issues)
Time table is in post #2
Next two weeks to continue to work on codebase maintenance
2022-06-05 — 2022-06-19:
Published estimated stake through NumerBay (thread here 1)
Added sales leaderboard here 3
Added warning and handling for encryption key change
Other bug fixes and improvements (See list of closed issues)
jrAI Is Stepping Down from The CoE
On June 21st, 2022 jrAI placed his intentions in the CoE forums to step down from the CoE.
Forum Post: Nominations for open Council of Elders seat (June 21 through June 28, 2022)
Here is the current status of nominations
bor1 [accepted]
wigglemuse [declined]
yxbot (yifan) [accepted]
jefferythewind [accepted]
dev0n [declined]
degerhan [declined]
theomniacs (omnianalytics) [declined]
restrading [declined]
ia_ai (ai joe) [declined]
malding (studym8) [declined]
greyone [declined]
qeintelligence [declined]
perfect_fit (Carlo) [declined]
of_s [declined]
Voting is now live for the three accepted nominations
bor1
yxbot
jefferythewind
Forum Post: [Vote] Elder Voting (Election #3)
All votes must come from active Numerai participants and votes are blind and log-weighted by stake size. Results are blind until poll closes. The poll will close at 8pm ET on July 5, 2022
We are so sad to see you leave the CoE but know you still have major plans for the future of the CoE and Numerai!
Help Numerai improve Numerai Compute!
On June 7th, 2022 2chanes placed a post in the main forums with a list of questions and proposals to comment on to help Numerai improve compute. He states:
Research has shown that trading earlier (at market open instead of market close) and more often (daily instead of weekly) can lead to better execution and better performance for the Numerai Hedge Fund.
To support this future, we are exploring the idea of daily rounds with much shorter submission windows. This change will effectively make model automation mandatory.
Currently, only 4.4% of staked submission models are automated using the Numerai Compute system. Our goal is to bring this number up to 100% by making the developer experience of model automation as smooth and pain free as possible.
Some questions for you:
Are your submissions automated now?
Do you submit from a home machine, compute/cloud or website?
Do you want to train your model locally or in the cloud?
How often do you retrain your models?
What cloud platform are you most comfortable with?
Do you use version control for your model code?
What are the biggest pain points with the current Compute setup?
How do you typically deploy a model to production?
Some proposals we have:
Automated command-line tool to deploy Sagemaker Studio notebooks and webhook. The CLI tool would deploy: a training notebook, a submission notebook, and a webhook that Numerai calls. This setup would remove the need for Docker and Terraform and would hopefully give users a more familiar UX by running things in a notebook.
Using Linode Stackscript to deploy a cloud computing machine with all the dependencies installed. It’s much easier to get up and running in Linode vs. AWS, so this path would be helpful to users who aren’t comfortable working with AWS.
Automated command-line tool to deploy a scheduled Google Colab notebook (h/t to bor for his Google cloud post here)
Which of these is most appealing to you? What changes would you make?
Please make you way over to this forum post to submit your responses: Help us improve Numerai Compute!
CoE Sponsored Numerai Meet Up London- Updates
We now have 40+ registered to attend the meetup on July 16th at the Microsoft Reactor in London!
We also have a growing list of speakers with some great topics. See below:
Introduction to Numerai by Jo-fai Chow aka ia_ai (Numerai Participant & Meetup Organiser)
A Complete Beginner's Guide to Numerai: From Zero to Hero by Alex Benton aka Aventurine (Numerai Council of Elders)
Welcome to the Numerai Universe by Jos Eilers (Numerai Participant)
Signals - the Non-ML Approach by Jon Taylor aka Arbitrage (Numerai Council of Elders)
Building a Profitable Signals Model: the Principles Behind Alpha_X/Beta_X by Boogies999 (Numerai Participant)
From Tournament to Trading: a Brief Tour of How We Turn Signals into a Hedge Fund by James Elford (Numerai Team Member)
There is another surprise coming soon for attendees…
Sign up for free attendance and tell all your data science friends
https://www.eventbrite.co.uk/e/numerai-community-meetuphackathon-tickets-353999751787
CoE June 2022 Transaction Report
June 3 -34.36 NMR Newsletter
June 7 -217.39 NMR Omnianalytics for RNumerai Rewrite
June 7 -124 Numerbay
June 19 -191 Numerbay
June 30 -145 aventurine travel stipend London meetup
Forum Posts To Check Out
updates in Classics tournament reporting including TC by qeintelligence
Disclaimer: This is not an official Numerai newsletter. It is sponsored by the Numerai CoE, a decentralized autonomous organization. Every effort is made to provide accurate and complete information but there is no claims, promises or guarantees about the accuracy of the contents.