by LiamHz
Deleting Diagnostics Is Live
Deleting diagnostics runs is now live, courtesy of 2chanes
Numerai's 2021 Growth
Meet Up’s Around The World?…Soon
NJ recently gave some stats about the recent Numerai Meet up in Japan:
>100 attendees of the recent numerai Japan meetup have a total of 716 known numerai classic models of our total 3,757 contributing to the meta model and 414 models of the 768 staked signals models. amazing. they also currently account for about 4% in total NMR staked
In response, Richard tweeted
we’re going to do way more Numerai meet ups this year all around the world
A new channel in Rocket Chat was created #events
Canada, London, Germany and Austin, TX has been targeted for possible locations recently.
Tournament Stakes Hit All Time Highs
Numerai classic has crossed over 700k NMR staked. Signals is working on crossing 100k staked.
On Jan 27th 2022 an announcement was made that the Signals staking cap is being increased to 150,000 NMR effective at round 301.
Also, Starting with round 299, Signals users will be able to select `1xCORR
` payout multiplier. The default Signals `CORR
` multiplier remains at `2xCORR
`.
Numerbay Docs
You can now interact with NumerBay easily with the new Python client "numerbay" package : https://github.com/councilofelders/numerbay-api
Tutorial on how to use the Python/Cli client to automate seller submission workflow: https://docs.numerbay.ai/docs/tutorial-extras/api-automation
Tutorial on how to build an ensemble with preds bought on NumerBay: https://docs.numerbay.ai/docs/tutorial-extras/ensemble
NumerBay provides REST APIs at the backend endpoint https://numerbay.ai/backend-api/v1
Ensemble with NumerBay | NumerBay
In this tutorial we are going to make a simple ensemble Numerai predictions file from predictions bought on NumerBay.
CoE Starts Regular Twitter Spaces Events
On Jan 18th 2022, the CoE made an announcement:
Each Friday at 9:30am PST/5:30pm UTC, the CoE will be hosting weekly twitter spaces for 30mins-1hr. We are doing this to become a little more active on social media for Numerai/Numerai COE visibility as well as keeping the community consistently up to date on everything CoE related. Topics will be, for the most part, open discussion on CoE proposal updates, Numerai updates/news, brainstorming new proposals, classic/signals tournaments etc. We are also planning to begin hosting "fireside" CoE zoom chats every quarter as well to coincide with Numerai fireside chats. Looking forward to seeing everyone there!
These twitter spaces will be shared on the CoE twitter account: https://twitter.com/NumeraiCoE as well as on the Rocket Chat CoE Channel
Please follow the CoE twitter with notifications on!
Tournament 20D2L Scoring
If you remember back from Nov/Dec 2021, Numerai announced that the Tournament would begin scoring predictions against a 20D2L target and that the change was “right around the corner,” well, the change has recently come into effect.
New rounds resolve on Fridays now
“numermatic”- Numerai model training in the cloud
Community member forstmeier is almost done creating a platform for training Numerai models in the cloud. It provides a free to use API hosted in your own AWS account.
Help has been requested if possible to finish some fine details
See below:
GitHub: https://github.com/forstmeier/numermatic
Numerbay Updates
Jan 2nd: The following includes changes from the master branch 1.
Published documentation site at https://docs.numerbay.ai
(WIP)
Minor UI improvements
Work plan is in post #2
Next week to continue working on improving user experience and documentation
Jan 9th: The following includes changes from the master branch and numerbay Python client.
Release of numerbay Python client and docs
Added support for exporting orders/sales to CSVs
Added support for image upload in product descriptions
Other minor fixes and improvements
Work plan is in post #2
Next week to continue the work on improving UI and documentations
Jan 16th: The following includes changes from the master branch and numerbay Python client.
Added TC and more metrics to product page, added percentiles to chart
Updated tutorials
Other minor fixes and improvements
Work plan is in post #2
Next week to start preparing for decentralization
Jan 23rd: The following includes changes from the web3 branch 3.
Started implementing client-side file encryption/decryption (WIP)
Fixed catalog page loading error
Other minor fixes and improvements
Work plan is in post #2
Next week to continue the implementation for client-side encryption for web and Python client
Jan 30th: The following includes changes from the web3 branch and numerbay Python client and are not yet live.
Implemented client-side file encryption/decryption
Added support for the same for Python client
Implemented key pair generation UI
Improved upload user experience
Started implementing direct Numerai submission for per-order artifacts
Other minor fixes and improvements
Work plan is in post #2
Next week to continue the implementation for client-side encryption, test, document changes and prepare for rollout
Numerai YouTube Tutorial Series
Next YouTube video by malding is out now. Watch and Subscribe
CoE Transactions Jan 2022
-37 NMR Numerbay
-32 NMR Numerbay
-12.61 NMR Newsletter
-61 NMR Numerbay
-35 NMR Numerbay
-86 NMR Numerbay
-59 NMR Numerbay
CoE Fireside Chat with Richard and Anson Q1 2022
Slido Question Timestamps
[0:00] Intro
[0:14] Anything you can tell us about how the targets other than NOMI are generated? (Jerome, Janet, Ben, etc.) We have not given out too much about that but if you do analyze them you could tell what they could be. We are working on releasing the test targets
[1:16] ]When will True Contribution be rolled out? Will that be something we can replicate or train explicitly towards locally? It is rolling out. Some are still be computed. Its a very difficult computation and there is so much compute involved. It is still a work in progress. It creates multiple “bags” sort of like MMC. Documentation will come soon. It is different than CORR or even MMC or even FNC.
[8:16] Can you talk about fun/challenging stuff that happened at Numerai that is invisible to us (i.e. not user-facing). Like portfolio optimizer, data vendors, etc. We are kind of doing all the above. Always trying to add. The most challenging stuff right now is TC.
[9:31] How to convince my DS friends to join the tournament? When NMR/₿TC now is -90% from 2020 highs and -99% since 2018. And just x2 since all time lows. If you involved in crypto, you do need a long term view. We will never promise NMR can get to a certain level.
[11:26] Any chance we'll see additional targets in Signals this year? We are thinking about using TC in signals as well right now. That would basically unify things together. Goal would be that all models on all tournaments will make things the same level of “hardness” and payouts could use TC for staking
[13:04] What are your plans for 2022? We are still thinking about it but we want to raise more money for the fund. We are under capacity. We could be managing a lot more, like around 500M. There is also an idea of what other products could we come up with.
[15:28] Are you happy with the signal participation vs. classic? Any plans to incentivize signals further? Last year we 10x signals which we are very proud of. Growth seems like its in a decent place. The next thing for signals could be TC. We are not looking to incentivize signals specifically but to unify both with TC.
[16:40] What is the target AUM of the HF that is expected to be achieved by end of 2022. Why is AUM not visible on numerai.fund? AUM should be visible. We will try to add it. We want 500M and are trying to get them through SMA’s possibly.
[18:52] Does the MM generated from models submitting with new data outperforms the MM from old data? What is the boost provided by new data (in live) all else equal? We don’t make a metamodel just based on old data and compare to the new. The IDs are the same but overall the metamodel itself has improved after the new data release. We have also made optimizer improvements.
[20:39] Is there any interesting news or milestones for Numerai as a start-up that can be shared, such as a new funding round, or interesting feedback from your investor? Had an investor call and is excited about Numerai. One thing they had talked about is the long term future of Numerai, like maybe going public etc. Overall sentiment was very bullish. If we get up to 300M in AUM we should be profitable. There is no plans to go “public” or even funding rounds.
[25:27] Any chance we also get to see the metamodel performance results per round on the website? I am just super curious Never thought about that but it would be interesting. We will have the team look into it.
[27:38] Currently, neutralization in Signals is a bit obscure. Do you have any plans to change this? Yes well by changing to TC it will also be obscure but we can do a lot in diagnostics to make things a bit more clear. We will work on it.
[28:46] When will the HF's performance be correlated to NMR? Some people want to see NMR trade like the fund but there is regulations if things were like this, sort of like a swap. This is not the case. We dont want it to be a security. Its for staking. It is its own thing.
[31:16] What will Numerai do when NMR runs out? Well it will never run out. We are in charge of how fast it runs out so we will make it not run out. This is all we will say
[31:48] Plz share thoughts on why NMR should be decorrelated when the HF's perf has no bearing on the token. It's merely a demand-supply eq similar to other cryptos (edited) Well its not an investment with our fund and at the same time it is its own thing. It should not follow along with something like dogecoin or shiba. We want to see it trade on its own mechanics.
[34:34] How about adding shorter horizon targets like 5 or 10 days to let us optimize trajectory of targets. Good for both diversification and risk reductions. I dont think we will go with shorter times. We dont want to see fast moving signals like that.
[36:20] Of 700k NMR sold to Pantera, 120k is staked by a single entity (Crowdcent). Defeats crowdsourcing when 17% is staked by one. Also it eats into everyone's payouts. I am actually sort of inspired because the bigger the stake the more confidence. In the sort term it might seem distorted but in the future it might change.
[39:07] NMR has taken a back seat in priority. What are the plans to increase the utility of the token? You are right it has taken a backseat. Our core focus has been keeping growth going for the company and making the hedge fund work. Anson wants to change this by working on payout and staking changes. We will talk more about decentralizing Numerai at Numeraicon in Apr.
[41:05] When do you plan to stop paying for the example prediction? Soon. You might make a little off TC but a model like this will not work as good when TC comes out.
[41:45] Any details about the data expansion now scheduled for March? More targets? Test targets and a couple of new features but we dont want people to have to abandon their models. We want to see all data utilized in some way.
[43:28] Do you see a strong correlation between test and live performance for individual models? We do a bit but there is no guarantee it will always work all the time
[44:34] Will the new data release be compatible with the current pipeline? Super compatible
[45:25] Does the march release also include the split in live and test predictions? (as in: we only need to upload live results) YES
[45:38] Any plans to expand the available model slots to more than 50? I'm currently using all of mine. We want to make it infinite. There is some things like gas and other issues but we will add more and hopefully can make it infinite in the future
[46:28] How much gas spent? How much $WTF were claimable from transactions' fees? We spent a lot in gas. 200 ETH last year
[47:19] Will Numerai internally create or find an investment partner/internally managed pension fund to create a S&P500 portable alpha product? We published our performance recently. We wanted to show just our longs as well. Some people wanted to see like a long only ETF. Its crossed our minds.
[50:08] Do you have a plan to release neutralization factors of signals? No because it would be giving away our datasets. When TC comes it will be more clear.
[51:53] What is happening with the staking mechanism (the at risk bit). Is that in preparation for having more overlapping tournaments (like 1 every weekday?) We are making some small changes. There was some confusion over stake value etc. We want to make it so there is only one stake value because right now there is multiple other aspects that factor into stake values/payouts etc. There will be some more UI changes. We are going to rework staking contracts. We are modeling instant changes on chain etc. We want to in the future maybe even have a real time system and not weekly payouts. Look out for changes soon and there will be a forum post coming soon.
[56:08] Sometimes the expected payout has huge swings, like from -10% to +10% in two days. Do those swings happen when parts of the global stock market are closed? Can can be related to global stock events. Some markets can swing up/down while other markets are closed etc.
[57:36] Any plans on extending your compute solution to other cloud providers? (noticing a lot of proposals in the forum about compute now). I think so but compute works pretty well but its pretty complicated. I would like to see compute a community owned project.
[59:20] What's the data validation process like with signals? Recently the Yahoo tickers in the ticker map weren't being updated. This erodes confidence in the tourney. We agree. We need to do a lot more for signals. We want to hire more people as well. We want to have a plan for a clean system.
[1:01:36] How focused are you on getting clients for the hedge fund? Is it easier to market this kind of fund vs more traditional long/short ones? There are some investors that understand quant/market neutral funds and others that are used to what they already invest in. We have to shoot for very sophisticated investors. Its a long game for us. We are working on it.
[1:03:25] Are the features we’re neutralized against in Signals changed? If so, how often? They have not changed since we launched Signals.
[1:04:25] When docs updates on TC? We are going to do this soon.
[1:04:42] Thoughts on macro and bull/bear cycle? Just curious about your opinion. Things cannot go up and to the right forever. Fed will need to raise rates soon etc. These things are normal in cycles. If you mean crypto they will go up forever :)
[1:06:15] Any chance we'll see Numerai data in Signals? No
[1:06:27] I think everyone has a model with awful validation but amazing live performance. Will time bring the performance closer to the validation? How validation created? In the end, its just an estimate. When TC comes things will be a bit different.
[1:09:58] Can you give an update on the trends? like stake weighted test correlation, proportion staked on smds models, metamodel sharpe. Right now its going up a lot but the future will be interesting.
[1:12:28] Have you considered having subsets of data? We have but have not looked into how we would add this to the current data. Could happen in the future.
[1:16:55] When is the targeted rollout for de-staking changes? We are working on it right now. We are looking into the design as well.
[1:17:25] Please give details on how to generate small feature set my Michael Oliver. The code for doing it is in the forum post. There is a lot to be said about feature sets and subsets etc. We also want to post some other examples of this.
[1:19:00] What will missing data look like? It could look many different ways but we are going to try our best to give all the data. We don’t want to paper over data. We will communicate it to you all.
[1:20:11] TC would be calculated against all stake predictions or just TB200? Whatever the optimizer chooses. We are still working things out. Currently its set up where we assume a 1% stake and then run in bags multiple times and have the optimizer choose when you are in or out to calculate TC. There could be some slight changes.
[1:24:04] When will signals payout factor start to decrease? We don’t want to ever see it happen and this is a reason why TC is coming soon. We want to move away from CORR because even if everyone gets better and better CORR does not mean Numerai makes more money.
[1:28:02] How important is Richard to the fund? What would happen if Richard went missing? We would be really sad if he went missing but we would not lose our alpha. Richard says he is not going to go missing
[1:28:50] My highest performing model in terms of TC is a tree ensemble, should I be worried? Why, because its common? Don’t worry about that. I don’t know why you should be concerned.
[1:32:36] No more questions
True Contribution
After the fireside, TC has been the “talk of the town.” Many messages have been traded in the general chat channel of the Rocket Chat. Also a forum post was created to talk more about it here: https://forum.numer.ai/t/thoughts-on-true-contribution/4832/12
See Richards response below.
Thank you for the all the TC feedback and great examples. I think it's best to think of current TC as in an early stage. We wanted to show you where we were with it on the fireside but there are still improvements and bug fixes we want to make to it which is why we haven't released documentation on it yet. Over the coming weeks we'll get closer to finalizing it, release docs, and may re-build / re-backfill it. So when you look at it now keep in mind you're looking at a very early version of TC.
-Richard
Disclaimer: This is not an official Numerai newsletter. It is sponsored by the Numerai CoE, a decentralized autonomous organization. Every effort is made to provide accurate and complete information but there is no claims, promises or guarantees about the accuracy of the contents.