More on Concurrency

hi this is charles hoskinson broadcasting live from warm sunny colorado always warm always sunny sometimes colorado today is september 10th 2021 fun and exciting day and we just published something on the iohk website on our blog it's titled concurrency and all that cardano smart contracts and the extended utxo model now you've noticed that over the past seven days that there's been a gradual trickling of blog posts by both community projects like mnswap and sunday swap and meld as well as some of the blog posts that have come from us as and we've even started updating some documentation on specific design patterns to help developers so anyway i'll show you guys the blog so let me go ahead and share my screen here oh and i'll kill the slack so we don't get bothered by click click click click all right so this is the blog post it just literally came out and it says concurrency and all that cardinal smart contracts the extended utxo model date is wrong and it says cardano's utxo based blockchain which utilizes a different programming paradigm for decentralized applications from other account based blockchains like ethereum the specific flavor that cardano uses is the extended unspent transaction output model introduced by the alonzo upgrade eutxo offers greater security allowing for smart contract execution cost predictability without unpleasant surprises and as a result offers a different approach to parallelization eutxo inherits the per branch design of the utxo model which is actually the model that bitcoin pioneered where one branch is by definition a sequence of transactions that requires a sequence of validations to split the logic across different branches and to enforce more parallelism it is essential to build daps and other solutions using multiple utx cells that's the first issue that we've seen is that people aren't doing that this provides benefits in terms of scaling just like developing bitcoin services prerequisites splitting more one wallet into sub wallets dapps built on cardano are not limited to one transaction per block as many people have been running around and reporting and yelling and barking at in fact the block budget that is the maximum number of transactions it can hold allows the execution of hundreds of simple transactions and several complex scripts however the eutxo model allows spending a transaction output only once given that users can face contention issues trying to access the same utxo it's important to use many different utx cells that's the design pattern component of it note that this is important unless such a design would benefit from a strict ordering of clients sets of utxos can be used to implement design patterns that include semaphores in addition different users can interact with one smart contract without any concurrency failure this is because a smart contract can handle a number of different ut xls that make it make its current state and off chain metadata that allows interpreting those utx cells so then the blog post goes on to kind of discussing how to do things in parallel how to understand concurrency a bit about how deploying dapps is a little different and we actually end with some examples it says we propose a solution within our recent digit stable coin paper for the jet implementation of cardano and order book modeling pattern is favored whereby an order makes maker is responsible for forwarding minting or burning tokens to stable coin smart contract with additional incentive blah blah blah so we're going to publish a whole article on this particular order book modeling pattern and that article is going to include code that we've written specifically for the jet so this is not just hey it could be done we're actually going to show you how to do it and how we did it in addition to that we wrote a few more things like how to design a scalable plus application we keep updating that and we actually wrote a little bit about the order book patterns and this so these three links are here inside the paper and this talks about some guidelines on scalable applications this is a living document over time as we get a little bit more time past the conference we're going to continue adding to it but it gives a whole bunch of things on utxo congestion minting policy scripts scalability guidelines and give some examples of different things and then we talk about the order book pattern which is a very important thing and in some way people are having to do this for their stuff okay and then concurrent and deterministic batching on the utxo ledger this is an article that was written by the meld guys and actually it's a very good article and it goes through a lot of the things that they've had to do to resolve this issue on their side okay anyway the long of the short is that this is an example of a problem that has to be solved for anybody who's implementing something so the developer has to think about it how do i want to handle concurrency you can do it on chain you can do it off chain now it's a new model extended utxl and so for the most part the a lot of developers coming from the account space world ethereum they have to think a little differently in order to solve that and the analogy i like to use here is when we went from single core processors to multi-core processors you'd see the marketing dual core quad-core eight cores 16 cores that meant software under the hood actually had to be written a little differently to take advantage of parallelism and if you didn't do it you could have 16 cores 32 cores but only one would actually be engaged one thread would be engaged in that application unless the application was rewritten for it it's a bit disingenuous then to say oh well intel's a scam and arm's a scam and amd's a scam they lied to us they said 16 cores but i only use one core well that's a software contingent thing those are there you can access them there are design patterns to do that there's we've got a lot better today than we were when the dual core world came out but you still need to introduce parallelism into your code in order to do that the parallelism pattern for extended ut excel people are just learning right now so we're going to keep writing blog posts we're going to keep doing tutorials we're going to keep adding code we'll make sure to include these things in the next iteration of the plutus pioneers program and showcase actual running stuff on the test net and on the main net and then over time people are going to learn a lot more about these things but what's nice about this model is that you have a high degree of flexibility so the way that we design the programmability of cardano is that we understand where you can use accounts and where e utxo makes sense and the eu takes so side of the world you get resource determinism you get a whole bunch of guarantees about execution or the lack thereof and a much easier analysis of behavior of the contracts these are fundamental design trade-offs that make a lot of sense when you talk about verifying the behavior of the flows of value in a smart contract which is where most of the bugs are and why you have reentrancy bugs and all kinds of things that have caused the dow hack and multi-sig bugs and access control bugs where hackers can come in and steal all your money okay so then on the accounts side it does other things for you and we created something called the chimeric ledger design it's running in full on catalyst because we implement in yorman gander this way and we have a lightweight version of it running for staking rewards on the cardonal main net as these design paradigms of extended uso become more mature and a feedback loop occurs for the developers some of the tricks that the developers use to implement parallelism will eventually be rolled into the language itself and into the e-utxo model in addition to that we're working with a partner called mune to do an emulated account layer instead of doing a full chimeric ledger implementation now that probably is enough for most app developers but it can turn into an on-chain full accounts model like what the chimeric ledger has to say about it and actually by building an emulated layer and testing it out and thinking about it i think we can do a follow-up paper that actually has an even better merger between those two systems okay it's not a requirement to write applications now in fact many people don't even need to think about it and they're going to be happy with those design patterns but maybe they need another one but the point here was flexibility to be able to use different computing models on side chains and the main chain and also the ability to outsource computation to parallel actors that's how you achieve scalability when your stake pools can be become batchers and serializers and processors and that trust model is okay for your application and the application developer has to make a decision which model to embrace and that's really breaks down to your operating cost your trust model and also your latency and settlement times that you want to tolerate the base ledger provides a combination of trade-offs and if that's not good enough you go somewhere else and the point is the system allows you to seamlessly wire all those components together for one unified user experience and the user is fully aware in that unified user experience what is the trust model that's the point that's what we've been investing so much time in and thought on now over time as i said design patterns abstractions tools and other things will be constructed and there seems to be this bizarre narrative that's been floating around the space of well come september 12th if when you flip the switch five million applications don't magically turn on and there's two trillion dollars of activity on the chain the next day it's a failure it's just insane to me it's bizarre it's a completely new system it's a completely new model the reason why we're turning on that functionality now is that it's safe to do so we understand what it can do and it massively improves the underlying utility of the system but that doesn't obviate the need for a lot of these applications the community has just started building or thinking about from testing them on the test nets and then eventually rolling those into the main network and when they think that's necessary just the existence of these features on the main net makes things that we're already doing as a company for ethiopia and for other things a lot easier for us the same for the foundation the same for amergo and many other projects but that baseline functionality is of course going to be enhanced and augmented and many new bells and whistles will come the introduction of hydra for example and that's a road that takes months and so every three months there's going to be an hfc event for the probably the foreseeable future and that's going to continue adding more capabilities and functionality to the system and you'll get more and more sophisticated applications and of course with that sophistication if the tooling is built correctly you're still going to be able to maintain a pretty good level of quality and certification and certainty that the application is right the other thing is our scaling solutions that we've thought about for boss show we believe are much faster to market than a fully sharded chain so hydra is not some concept that's way out there and its application will only occur in 2025 or something like that many components of what hydra is going to do to massively scale up concurrency and usability and lower cost can be implemented throughout all of 2022 okay so as this model becomes more popular the tooling becomes popular designs are starting to solidify it's going to be fairly seamless to use these isomorphic state channels and batch processing and other things to basically massively accelerate the applications that's the point and that's the design and that's why we built extended utxo the way we did there are certainly a lot of syntactic sugars and conveniences that you can put into the system in order to make the system potentially a little easier for base level programming but then you lose some of the theoretical and applied certainty about the capabilities of the system the security of the system so you have to always balance these things and there's of course trade-offs there if you like any of that then just take the ethereum style model and that's coming in short order for cardona we'll have a lot to say about that at the summit and when that's there you can run solidity smart contracts in the entire tooling ecosystem of ethereum just better faster and cheaper on cardano and those contracts can call pluto smart contracts on the main chain that infrastructure will be built out as well okay that's the point about being a superset and maximizing functionality i am still at an absolute loss over some of the media coverage over the last two weeks it's just bizarre to me a incredible articles making wild assertions that first we can only process one transaction per block we've demonstrated as have completely independent people building things on the system that that's just a lie it's just not true and it's very clear that a new model requires a different notion of concurrency and we've pointed that out and we're writing extensive documentation now to kind of show people that and eventually tutorials and trainers and other things and then there's pedagogical code that inevitably will be present so why even say these things yet apparently people felt it necessary to to do that and to use twitter as a primary resource and quote tweets in articles like for example adriana's article into which was just a bizarre hit piece then the other narrative we've seen is that there was somehow a massive price collapse of cardano the market collapsed and proportionately everything collapsed together there was no case where everything was green and one thing went down and that thing went down because we somehow failed to deliver something that's another bizarre notion that i just don't understand and actually that decrypt article seemed to imply that which was materially false and absolutely unfair and third apparently this bizarre idea that we've made an assertion that every single thing is finished september 12th we've never said that there's more to the road map there's basho and voltaire and obviously there's still shelly iterations that we're doing like proxy keys and partial delegation and other things and we as a project are moving as quickly as we can with us and the partners together and obviously there's going to be lots of refinements and construction for gogan and a nice feedback loop that is required the point here is that there's a large group of developers who are capitalized passionate at a large group of consumers who are excited and want to be part of this and that process is going to take months to work its way through but they're there they're building and come the 12th they can actually start deploying the outputs of their ideas on both the testnet and the mainnet where and when that makes sense to them and then of course the main net is going to periodically go through a series of upgrades to gain more and more capabilities and make the ease of development easier for people the surrounding infrastructure the pluto's application backhand is also going to do that and of course we're going to continue writing blog posts and the community is going to continue writing blog posts and there's already a tremendously helpful community if you look at the stack exchange and you look at the discord and there's lots of dialogue on a daily basis about these things we've gotten incredible participation with the plutus pioneers program well over 3 000 students and you'll meet a lot of them at the summit a lot of them registered for that so what's the problem you have a large community you have a vibrant development ecosystem we make deadlines and hit those deadlines the 12th is the 12th that's happening there's expectation management that things do take time and you need a feedback feedback loop for things to work properly there's clearly a commitment to education here and a commitment to understanding this model and that's been published time and again and the ecosystem continues to grow yet for some reason that's not enough for some people in this space i think that brings up my final point and it's a meta point about the nature of this industry in my entire professional life and all the people i talked to were in other industries i've never encountered an industry where things are so viciously polarizing and inhuman you're not allowed to make a mistake and if you do make a mistake you are evil a criminal a sociopath a pathological liar a horrible human being the products you build must always be corrupted and bad so if you say something is supposed to come out in a month and it takes two months even though there was a delay if it ordinarily people would look at and say well is it good or bad because there was a delay that must be bad always and every time you have an accomplishment the accomplishment doesn't matter this is not just applied to cardano it seems to be applied universally from bitcoin maximus attacking others italics said in 2014 that ethereum 2 would come out in 2016. five years later they're not quite there yet but they had very good reasons for that yet that is top line news and repeatedly mentioned again and again and again as if all the work that's been done and the ecosystem that's been grown is completely invalid shelly shipped cardano is fully decentralized we're the largest most valuable and most secure proof of stake protocol in the world that is an enormous accomplishment if if that was the only thing accomplished we'd built a better bitcoin and bitcoin's worth a trillion dollars that's something but apparently that's nothing even though the work is public the hundred plus papers which don't matter all the code which doesn't matter all the people use stake pool operators running the system you don't matter according to these critics and if it was just twitter trolls fine but there doesn't seem to be any benefit of the doubt or commercial courtesy i see criticism from vc's hedge fund managers criticism from fellow companies and it doesn't stop at we disagree with their or we think that their design is not competitive or their technology is not as good as ours it's my competitor is a criminal and deserves to be in jail i've never seen anything like this the ceo of microsoft sasha and adella doesn't say boy google's a bad company and everybody working at google they're criminals we can't wait for the regulators to shut them down they may say hey we think there are problems with google and and we think maybe data surveillance is too bad there or there's pr or maybe google docs suck compared to microsoft office and here's why and we support this feature and they don't support that feature but they don't delegitimize and dehumanize people in this particular way yet that is present in this industry why is that important well it's important because this behavior is limiting our industry's ability to grow and thrive and it's creating an ultimately adversarial bend where it makes no sense at all if you are a bitcoin maximalist and you don't like proof of stake what we are launching september 12th should be something you get extremely excited about because you are getting for free a beta test of a path for bitcoin to have smart contracts if it's successful on cardano with value at risk at almost 100 billion dollars every hacker in the world is going to try to break it and understand it and dig deep in it all these applications that are built those design patterns if bitcoin was to upgrade to have extended utxo they would get all of that r d for free so why then as a maximalist would you look at this as a scam or a bad thing all it's doing is testing out something that could be tremendously beneficial to their ecosystem the way we do side chains the way we think about proof-of-stake the way we think about building out a proof-of-stake ecosystem and cross-chain communication is tremendously valuable to the people at polkadot because the similarities of their consensus algorithm and ours and frankly tremendously valuable to economic research which may suggest for example you don't need to bond or slash for proof of stake to work and question some of the design decisions that are being proposed for ethereum too if at the very least kind of force you to double justify them because that's such an onerous requirement so the mere existence of cardano provides a beautiful data point for the polka dot ecosystem in the ethereum ecosystem and everything we do is out in the open there's no proprietary code there's no hidden repositories there's no patents there's no trust us no we actually release it again and again and again and we release it at the highest standards of rigor we write mathematical proofs we write computer proofs we go through the peer review process the work we do sometimes slow sometimes fast is permanently part of the cryptocurrency ecosystem for others to use for example we came up with the concept of nipah pals a graduate student named dionysus zindros was working with aguilos kiosis our our chief scientist did that that is now the basis for derivative work like fly client the existence of that made fly client possible that's a net benefit to the entire ecosystem so i can't understand the maximalism from that point what's the scam that we're just going to be around forever doing scientific research building products products that work at scale products that move every single day billions of dollars of value products that are now programmable catalyst is the largest decentralized organization on the planet over a billion dollars of values there tens of thousands of people participate on a daily basis in some capacity with catalyst it's been used to fund over 150 ventures and more are coming the longer that goes on the more get funded the more interesting input becomes and it's a great data point for anyone interested in decentralized governance and a great data point on how to do the voting effectively what's the scam in that that that data doesn't deserve to exist they should just shut it all down bitcoin is the only truth i don't understand this and these articles that come out i mean even today a tweet from weiss which has historically actually been somewhat friendly to the cardano project said smart contracts are allegedly coming september 12th i i just i don't get that how do you make a statement like that on twitter as supposedly an independent non-biased ratings agency like there's some sort of great lie or conspiracy and we're actually hiding the truth how would we hide that come september 12th either smart contracts run on cardno or not what type of colossal embarrassment would it be if we could run smart contracts when it's asserted that there are we're not some fly-by-night thing we've been around for six years this project has delivered time and time again whether people want to admit it or not over 70 000 nfts have been already issued on cardano in the tweet they made an assertion that only after the hard fork happens allegedly nfts are possible i guess that's news to all the members of the cardano community who have purchased and issued nfts on cardano for example this artwork right here sent from a young gal named jess that's on cardano so you tell me allegedly about these things i don't understand it i just don't understand it it's it's just it's just extraordinary to me and it makes me want to not even attend any industry events or do anything inside the industry what is the industry actually providing at this point there's really no leadership for the core of the industry on a lot of the innovation side it's just maximalism and debates about tired old arguments we looked instead to the real world the farmers the people who have supply chains the governments that want to do e-voting the people who want to do water rights and energy rights and do capacity exchange the telcos that want to revolutionize themselves they don't actually treat people that way when you go and do business with them they're excited to talk to you and they don't open up and say tell us the token dubai they say here's the problem we have and we want solved and we say that's great we're excited and it just so happens that what we've constructed with cardano and the surrounding infrastructure is part of that solution for them and then they ask how much does it cost how long does it take to get the market how do we maintain it how do we upgrade it standard business questions this is the majority of my day now we have well more than 100 in the commercial pipeline we're going to announce a lot at the conference showcase a lot of these things in fact i sent an email out two weeks ago to my company saying we can't accept any new business until november because we're so backlogged on things that's my day now i get invited to go to all these conferences like consensus and satoshi roundtable and so forth and i keep trying to struggle to find the will to actually go to them because i show up and it's the same tired old dialogue again and again and again and where we find the most kinship and partnership are not the infrastructure providers anymore it's usually the layer two or specific people building specific applications because it turns out there's great collaborations that can be done so i i'm not so bullish about this industry at the moment and i'm not so bullish about how people treat each other in this industry it's pretty sad and the launch of smart contracts on cardano is just an example of that we work for a very long time so some of the members of this team have been working for over four years we did our work we did our homework we did hundreds of presentations dozens of papers so much code was written and a lot of very careful thought went into building a very novel model that's ubiquitously beneficial if anything as an experiment for the industry but we feel is the best industrial model and we put our money where our mouth was we spent tens of millions of dollars to bring this to market and the response to it is basically well we're just going to lie and then we're going to write weird articles with pictures of me looking down and that were taken bulgaria two years ago by a journalist that we invited to go to bulgaria with us and wrote a negative article a two-year anniversary of cardano that's our industry that's the standard of journalistic content that's how crypto twitter works that's how these firms treat each other it doesn't impact us and affect us doesn't affect the cardano community we're always positive it's always warm it's always sunny here in cardano and we got places to be and things to do all it does is it hurts the nascent industry as a whole it slows down the rate of innovation it prevents people from wanting to enter the space it re it reduces the effectiveness of partnerships and frankly it creates interpersonal friction where none should exist if everybody has crossed basic decency thresholds and they've publicly said things like my competitor is a criminal or sociopath and deserves to be in jail tell me two years down the road how do we find a way to work together i always end up having to be the bigger man there's a lot of people in the chain link community on twitter who have said horrible things one particular count has committed criminal conduct and we're looking into seeing if we can file a criminal complaint there's never been any effort to slow that down or disavow that they always hide behind the decentralized community and then people always ask why aren't you working with them aren't you collaborating well there's been some offers and we're going to try because it's for the greater good of the ecosystem that collaboration happens but it's bittersweet to be honest with you because anything we announce will be attacked any effort we we push for will there will be some friction and the employees who dedicate their careers and their time to doing these things will be attacked as well and they have to go to work every day and work really hard think really hard and then they check twitter they check reddit and they say i'm so excited to announce this and then every comment is negative says it's a scam how dare you work with them you can't even in this day and age ask a basic question like solana started surging i don't know too much about the project a lot more today than i did when i tweeted but i tweeted at the founders who are very gracious by the way hey congratulations please tell me more and i received a few pms with some documentation and we've been going through it and looking at proof of history and these other things then there was a whole wave of criticism saying how dare you even talk to us you're just trying to use us as a launching pad to improve the value of your token and this that and the other i don't get it why why what's the point what do you gain from that at just attacking and attacking and attacking and attacking and defensive and defensive do you get any upside is there any value to living in constant toxicity and cynicism and and to the project founders what do you gain by curating a community that does that honestly all you do is you make it so that people don't want to collaborate and talk to each other and work together and where do we get as an industry that's supposedly open source which means we all talk to each other work together that floss ethos the free the librie the open source ecosystem what do we get and gain from that nothing absolutely nothing so where are we at well come september 12th it's a party it's a great celebration for our community and for the industry for our community because of their patience and all the things that they've had to endure to get here and the criticism they've personally taken to get here and i recommend we don't spike the football and say see we told you so as satisfying as that can be it's only exacerbating an existing problem that the industry as a whole has rather just be humble about it and say we did it and celebrate it together that's my only plea to my community that i work with and i so admire and love to our employees and contractors and development partners the more than 12 companies that write infrastructure every day for cardano and all the big ecosystem fluids pioneers everybody help us test this it's a thank you to all of them a celebration for all of them a brief moment where we all can take a deep breath relax for a second and just admire the fact that such a significant milestone has been achieved and to the industry it's another experiment data point about how to decentralize computation properly with a different model a unique model distinctly different from anything that's been seen before and we know that because we built it took four years to build it nobody else was crazy enough to try this and there's been a lot of failed attempts to do this and half attempts to do this inside the bitcoin ecosystem 12 years of trying in that respect so this is ubiquitously valuable in that respect so that's the day and then what happens there has to be morning after september 13th monday wake up it's business as usual lots of things will happen network will probably be under a lot more load because people play around with things and test things and that's fine and a few weeks later the latest and greatest version of the plutus application back-end will be released and we'll have the summit and there'll be a huge spotlight on the ecosystem as a whole and that'll be a great event and then you wake up the next day and you keep working that's how greatness is achieved not taking yourself too seriously being humble putting in the work but yet taking the time to reflect upon what's been achieved this has been a six year marathon started in 2015 for me now it's 20 21.