Some more info on the Testnet 05/21/2019

[Music] okay hi everybody I hope everyone can hear me all right hi I wanted to make a quick video talk about upcoming tests Nets of Shelly test net and things we're going to be doing and how these things are at work so basically we've had a lot of internal discussions about what is the best way to roll out a test net and the most meaningful way to roll out a test that so there's are currently two code bases that are converging to test nets and those code bases are visions of a possible Shelly and they're kind of unified by a collection of formal specifications so well both are inspired from the same specifications there are some meaningful differences between the rust and the Haskell clients and this is due to the way that the rust and the Haskell clients were constructed the Haskell clients were constructed very slow methodical a kind of sciency approach where we'd write a formal specification right reference Haskell code and then there'd be a series of refinements and iterations that would eventually lead to a production system and basically what we've done is we started moving towards a demo based development model for the Haskell client where we've shown demo after demo proof of functionality where the client is demonstrating it can do something like connecting the network or validate the blockchain and etc etc the rest team is running much more a traditional software project that's agile so basically they did sprints and every week for two weeks they complete some stories and they just keep moving their way forward and working through a backlog and they're if you were a professional developer and a major company that follows agile you'd see those team dynamics and they'd be very comfortable for you so this was done because we wanted to have one team a little bit ahead of the other team so they can basically act as a team that would be able to following that slow moving lumbering academic approach so it's gotten to a point where the Russ client is almost ready for a test net release and so we had a lot of discussions of what would be the most me and best way of doing a test them and what we feel is probably the best way of doing that is a self node set up so basically what this says is that for version 1 of the test that basically you'll have some sort of ml file generator which allows you to parametrize your own version of a test that it compiles it basically bootstraps your own Genesis block and then you're able on your machine to run a multi node environment where you can actually see how stake pools operate you can move funds between different nodes you can run your own version of warhorse Genesis and you can actually demonstrate the cryptocurrency working on your computer and the nice part about this self now setup is it's almost a blockchain in a box so you can change the parameters and you can see one node or a hundred nodes and you can see different distributions and you can kind of play around with staking and delegation on a command line side for the more you can implement against this self node and as we add in more functionality namely the network and the incentives component of it you're actually code that you've developed against the self node won't have to change much or at all for example we're right now implementing the haskell card on a wallet back-end against the rest node and as we had the network stack into that it's not going to change it much so for June what we're probably going to do is a yellow generator that allows you to parameterize your personal version of shelly then there'll be a script associated with the RUS code you put the ml file end it'll compile it bootstrap your own Genesis block and then you can run your system the way that you want now there's something called the state pool best practices telegram group for Cardinal and has about twelve twelve hundred ninety members in it and we're sending out surveys to these members and the purpose of that effort is to basically get a sense of people's technical acumen their ability to report bugs and their willingness to work with us to actually perform several testing scenarios throw throughout the month of June we'll be rolling those people over and allowing people host self notes and then the hope is in July that we can then upgrade that network to now be a single unified public test that so version one is you run something on your own machine and then it gives this a sense that the configurations are working correctly it's working on different types of hardware at different operating systems and a lot of things we think are true that we've tested internally are actually now being tested by you guys and look at a much broader set for that beta test then level 2 is saying ok here's a production system that is now in a mode word we're now connecting to a single unified test net so instead of you creating your own blockchain and playing around with that and moving funds between notes hosted locally you're now moving to a system where you're talking to nodes across the general internet you're gossiping you're downloading blocks from other peers and now we have a single unified system which will look a lot more like what the main that looks and then a next up grave will be adding in close to a final incentive system so basically the system of incentives that we've written down on paper to actually demonstrate that working in practice and stake rewards going to the state pools but that's for test Eneida instead of realated so there's kind of a series of three major milestones that need to be done there the self node milestone the network milestone and then finally the incentive milestone and once those three thresholds have been crossed then we have high confidence that shelly is is ready for primetime from a design perspective and that users can use it correctly and then there's a series of things that will happen in parallel as we walk our way through those three major milestones the self node the network site and the incentive site such as the security audit that Cadell ski is going to be doing with us a lot of fine tuning and tweaking I demonstrated a certain pieces can work with each other for example the Haskell wallet back-end can work with the Russ client and then we can connect the ask wallet back in with its new api's to the Corrado wallet GUI and Daedalus and demonstrate all these components working together so that's where the rest side of the world and then on the Haskell side of the world the demos are cumulative lis building their way up to basically a test net for Haskell and we think that somewhere in July those demos will conclude to that so while the Haskell side is a little bit by the rest side Russa is always running a faster due to the processes its following there will also be that out and there's going to be a delta between the Haskell client and the rest client so there's certain things that the Haskell clients done which may be more principled and better begin certainly took a lot more time like for example the design of the network protocol for the Haskell site is quite sophisticated it's very elegant but it did take a year to build versus what we're doing is more standard with the the Russ client and some decisions are going to have to be made about how we want to close that Delta that's the advantage of having parallel teams working basically with their own schedules their own budgets their own timelines is that you can kind of have multiple views and you can converge to an optimal design so that's that's what's coming we're going to work very closely with the mergo and basically make sure that they'll help us with testing and also that as more of these staking components come out that they can ensure that your ROI has this interoperability and we'll also make sure that say says it could blotchy and explore with them for visualizing a lot of the things that we do that are unique in our ecosystem so I just wanted to to give you guys a kind of a brief overview of what's coming it's important managing expectations so there will be a blog post that Vincent writes with Tim and the other guys specifically about what are the USPS in the rest client that they've been working on for the past nine months there's a lot of really magical things about for example how they handle the crypto they've abstracted it into its own crypto core which is very amenable to being compiled untrusted hardware like SGX or other platforms or just use standalone and there's a lot of architectural designs that are really elegant and are quite impressive on the Haskell side not to be outdone there's a lot of brilliance there as well especially on upgrade ability and maintenance we've now rewritten Cardno three times and we've done it slightly differently each time and we've learned a huge amount along the way but that's how software works and we're in a pretty good position to understand how new features are going to fit in and how we can do a four projection on that roadmap we also have two designs for multi-sig there's one that involves using a blue multi-sig contract and one that has native protocol level support from multi-sig and we've had a lot of discussions of how to best implement that on the rust side they've already implemented multi-sig in is a native protocol level support and we have a pretty good idea what the polluters contract would look and how these pieces would fit together and I've made it a prerequisite for multi-sig for Shelly's launch it's long overdue and it's something we do have to pull in let your nano ex support is also coming so so that's a pretty exciting device and it's a nice addition to the ecosystem I would love to be able to use trusted hardware devices like ledger devices with multi-sig so we're also going to be working with vacuum labs to ensure we have support for that it's very important for enterprise management of funds we've also made some considerable enhancements to Germany or the rest client we have C bindings and a few other things we've made for certain exchanges and those are of course available now to the general public so to recap in June there's going to be the self node it'll have a llamo generator you just load it into a bootstrap script you can build your own blockchain with it it's pretty cool you'll it it's gonna be fun and on top of that a series of videos will come out explaining how to use the command line interface manually build transactions do all kinds of things and uses Shelley specific features then also there's going to be some videos on how to report bugs and issues using git so you can you can push those to the github repo and we can respond in kind we're trying to move more as an open-source project instead of a hierarchical top-down company managed product and then coming July Network enhancements will come in that might slip a little bit but may not but it July is more likely than not network enhancements come in and we moved from a self hosted self node to a single unified test that and that unified test net will then go through series of upgrades adding more sophistication to the networks ACK additional features of functionality to the self note and finally the kind of candidate reward scheme and an incentive scheme and it'd be nice to design that collaboratively we have ideas would like to do but it would be nice to get some community feedback from it as well so if you're really interested in participating joined the Cardno task force i'll show you the title of the channel so you guys can join it's on telegram anybody can join its a open group and yes here's the title for it I know a it's a card a low stake pool work group his name anna has got twelve thousand twelve hundred ninety people are in it surveys are coming out will send the survey it will spam a link for the survey and pin it as well so fill that out just let us know who you are and then basically bola pull a subset into a ver focus group and we'll work very closely with people who are doing self notes so that includes having our DevOps people provide some tutorials on how to configure and build from source how to use the script that will allow you to take your e ml and turn it into your own blockchain and then we'd love you to run it for a series of different configurations and scenarios and see if you can break it because that's very valuable feedback for us and then then after we do that the next step would be to ship an update you'll download a new piece of software install that and then suddenly you're known well no longer be self parameterised the node will be connected to an actual unified test net and then it will it'll be faucets and these types of things you can get some data from that then you can use that either self host a stake pool or to delegate to a state pool and you'll be able to take the testing that you did with the self node and then do that testing with the test net and then have a very primitive reward scheme for the first generation and the next major updates that system among side network enhance Mintz will be basically the the close to final reward scheme that we propose so candidate one and then there'll be a series of small iterations and tuning in addition to whatever hardening is required because at that point the security audit will be close to concluding and a lot of the parts of the system will have to have some minor modifications undoubtedly to address whatever concerns the auditors have come up with as well as bugs that come through but the teams are moving very quickly the haskell bullet back-end team updates weekly the rust team every few weeks they're shipping something and the self node update is almost done it's probably this week or next week and then it just lender a little bit of QA and we're waiting for the house call it back-end to be connected with the rust node we anticipate June 7th for that so those are our two fastest moving projects but the Haskell teams are also moving a lot faster thanks to the demo driven development and then we'll kind of reassess where we're at at the end of June early July when we can ship a Haskell test net and that'll look a little different and it has a slightly different design than the rest and then we'll have to make a decision about the delta between the two and how aggressively we were going to close that delta and we'll update the specs accordingly but overall we're pretty happy with the way things are we're pretty happy with the progress and also the quality of the code especially the modularity of the code in fact when you discovered that we really couldn't out match the Haskell wallet back-end and so we said let's modular eyes it make it its own surface and then allow us to connect it to the rest code in the process of doing that we actually built a universal UTI XL wallet so you can rebrand surprise it to work with a Bitcoin node or a dash node or something like that and it has all that magic on how we handle input selection and basically the the design being very stateless it's it's really cool and it's just a great piece of infrastructure and it's evolved very rapidly over the last four months and there's just been some things we shopped for we said well through it let's go and do it and I'm really happy about that it's also a very concise the code line count is much lower than than anything in competing in that class which means maintenance and testing and these types of things look pretty good so that's that's the story they are self no network incentives and then after those things are finalized it's just a question of when is the best time for upgrading to Shelley then on the main net side of things we're going to ship one update to the main net to make some minor modifications to the OPF t deployment that we've done so that'll come sometime in June more likely than not and then we'll announce a date for the hard fork to opf t and then at that point there is kind of an upgrade path that we have to decide whether we just go straight to Shelley or we do an intermediate upgrade where we completely ditch the zurich l code and go on to the new Haskell code so there is an integration team right now working at pulling all the different Haskell pieces together and at some point those Haskell pieces will be able to replicate all the functionality of the existing note that we have that's in production so the advantage is that we'd be able to preach le migrate to basically a complete rewrite of byron end-to-end and that is a potential so we are having some internal discussions whether it's worth that slight diversion to upgrade that maintenance because a lot of our time is on the other side is being invested doing a test that exploration and it may take a little while to finalize some parameters so in the meantime it may be better to get us on a more stable code base than the historical Sharelle code which is actively hostile so so look for that as well at least one update minor updated so kind of a hotfix and some small changes and then after that we may do another update to throw away the old core and replace it with the new haskell core it's a good way of testing the production readiness of the system and also it's just significantly better code and a lot of old bugs that we chose not to patch because we're migrating from the old code the new code would then be resolved so we'll make an announcement sometimes June and July about to the final upgrade path their extensive documentation is underway we're going to completely redo Cardno docks and also we're writing a pretty good set of documentation for the rust client though several technical writers on it and we'll we'll make an announcement of when that documentation is is going to be ready for some public consumption but our first set of readers is going to be the card on a developer's channel as well as stateful task force and documentation is kind of a team sport where you write something somebody reads it and they tell you that what they and what they can't understand and what was easy it's not easy and then you revise it so there's there's kind of a back and forth there so I'm looking forward to getting into that that iteration and my I believe that documentation will improve considerably throughout the summer finally Plutus is looking pretty good it's evolving a pretty rapid rate we're working pretty hard at high school to jcj s so high school of JavaScript and we're working really hard at Haskell to webassembly and we've got a lot of progress there and throughout the summer that progress will continue to evolve please as the language is pretty pretty finalized at this point I've got the point we're starting to formalize a lot of these things in papers that for peer review and Plutus is now being taken to hackathons the most recent hackathon we did was actually in Mongolia we had 66 people show up and it was a marlowe hackathon and there's going to be a series of hackathons we do for both flues and Marla throughout the summer we also released two courses one core one on Marlow one on Plutus they both have more than 2,000 students on udemy and we'll start soliciting student feedback and probably do another edition of these courses based upon what we learned there's also you plead his book coming out in July and likely will do in a riley book or something like that starting and either queue for this year or q1 of next year to add to the existing pedagogy that is present so more videos are coming more hackathons are coming and we're going to a great feedback on usability also the 23 girls retrained in Ethiopia we've actually hired several of them specifically just to be beta testers and QA people for Plutus so basically their job is to build stuff with it and then tell us what they didn't and it allows us to learn a lot about things that we ought to change and we're going to start working a lot on developer experience at the moment the advantage of flutist is we can reuse a lot of things that if historic would come from the functional programming world especially in the haskell world and Haskell's been around since the 1980s so there's an enormous amount of accumulated knowledge tooling best practices and techniques which are basically portable to our system so we don't have to reinvent the wheel we just have to kind of make the wheel fit in a slightly different box and that's quite nice so so these things are these things are moving along quite well and I'm very happy with minwell chunk of our T's a great leadership there that team is extremely good and they're making good decisions about how to handle product management so we could integrate Plutus if we wanted to at any time but it's always been a post Shelly activity we wanted to focus on decentralisation therefore smart contracts so once all the code is rewritten and put into production there's gonna be a lot of conversations about the best time window to put Plutus in marlowe in that said this does not stop you guys from using the languages I'd highly recommend you play around with blue sprayground and I'd highly recommend that you play around with meadow and give us some feedback of what you like built some stuff there's tutorials there's examples there's courses now there's really no excuse not to start writing gaps and telling us the good the bad and the ugly of that particular experience so that it helps us and lets us to get to the next level we're gonna continue aggressively pushing hackathons and every Haskell course we teach from here on out will have at least one month of additional training specifically for Plutus so we're creating a lot of great flutist developers and it's really not hard for us to get several hundred people floating around so do build things do let us know if you like these things or not and what's the good the bad and the ugly okay so that's all I have on the update nothing significant but I trying to be a bit more regular than my communications and these test nuts are gonna go pretty quickly they're gonna be a lot of fun a huge amount of work on our end lots of messaging lots of communication lots of telegram communication lots of emails and that's just the way these things go oh it's gotta be whirlwind and I before you guys know it things will be ready for me net so thanks for your patience thanks for your time and I look forward to working with everybody and getting this done let's go launch something