Intersect, Repos, and Cardano Product Backlog

Hi everyone, this is Charles Hoskinson broadcasting live from warm, sunny Colorado. As promised, this is the chosen video to talk a little bit about Intersect and how things are going to start transitioning over from the Input Output repositories to Intersect. So, hold onto your hats; it's going to be a long video. This will be a whiteboard video, and I’m going to share my screen here in just a moment. But first, let me bring up some links for you so that people can follow along and take a look at some things.

Okay, first things first, let’s talk a little bit about Intersect. If you’ve been following the discussions we’ve had about a Cardano members-based organization, we named the MBO Intersect. If the community doesn’t like that name, they can change it. But basically, this is the aggregation hub for a lot of different things. In the whiteboard video, I’m going to discuss the kinds of things that Intersect, as an MBO, does and how it’s governed.

Now, the most important thing that’s relevant to the heart and soul of Cardano are the GitHub repositories. If you take a look at these repos, you can see there are an enormous number of people that work on them regularly: Cardano Node, Daedalus, Plutus, Drestia, and so many others. I think we have 636 repositories. A subset of these are directly connected to the heart and soul of Cardano, and this is kind of the staging ground for almost 15 or 16 companies to work together to build Cardano, the protocol. It’s a very decentralized group, but we’ve been taking the lead in organizing that repository.

What’s going to happen is that it’s going to transition over to Intersect throughout the summer, and a whole governance structure is going to be put around that. The purpose of this video is to talk a little bit about how this transition is going to work and some of the things that are going to happen. It’s easier to draw; it slows me down a little bit and also allows us to have no ambiguity, which is very important. Right now, we have the IO repos, and they’re going to move over to the Intersect repos. In Input Output, we have this concept of tribes, and each tribe is connected to a different concept, like smart contracts, for example, or core technology.

Smart contracts cover things like Plutus and Marlowe, extended UTXO, and these types of things. Core technology covers the core node network consensus. We have tribes like identity, side chains, and so forth. There are collections of them, and they do lots of stuff. If you take a look at the product function of Cardano, all these things come together to form a Cardano product backlog.

It’s the list of all the things that each of these individual tribes can constitute. Hundreds of engineers across 15 companies are working on this in some capacity. We’ve been calling that backlog continuity and GCD. The kinds of things getting Cardano done are related to the roadmap of Shelley, Basho, Voltaire, and Goguen. Continuity is just kind of the daily affairs: better, faster, cheaper, less memory consumption, and improvements, API enhancements, and all these types of things that form a backlog.

Throughout the summer, working groups are going to be established at Intersect. These working groups will take over the function of these tribes. The leadership of these tribes on our end will join the working groups, but then what will happen is that the community can come in from many different places—NFTs, DEXs, Cardano DeFi—and the leadership of these things can join those working groups. The current backlog that we have will be moved over to those working groups as a suggestion, and those groups will decide what’s in and what’s out. As the repos move over, meaning the code moves over, the group of engineers that work here, plus others, will get credentialed here and will start executing in 2024 the new backlog.

We have a PI system that goes quarterly. We take a look at the backlog and make decisions based on what is achievable in a three-month period. There’s a pre and post, and they make commitments and then do a post-mortem to see what actually got done. There’s usually a release cadence of two to eight weeks, depending upon the nature of the project within that PI cycle, to see how things are getting done. Once the code repos move over, there’s a new notion of credentialing, and these working groups will allow the product function to become completely decentralized.

The backlog here basically becomes the backlog there, and these working groups can decide things that go into the backlog in addition to or to replace the suggested backlog that gets thrown over the fence. You can divide this into categories: requires soft or hard fork, does not require a fork. In both cases, SIPS will probably be written for significant, meaningful things, but some things will not require a fork, while others will. The interplay of 1694 is that what 1694 can do, which is independent of Intersect, is important not to conflate the two. This is kind of the governance of Cardano, while Intersect is an execution vehicle that gets things done.

The government gives permission; for example, when it comes to protocol parameters, hard forks, and all these types of things, it gives permission to do that. Hang on, I’ve just lost my… there we go. Wacom can be a little touchy. It gives permission to do a hard fork and can authorize a SIP to be done. There’s going to be an interplay between the governance of Cardano and the working groups here, and that can be done as a consolidated backlog.

A workflow has to be established, so as Intersect turns online, that’s going to be something throughout the summer, fall, and first and second quarters of next year that starts getting memorialized regarding how that workflow is going to work. Regardless of how that workflow works, it fits very nicely onto a standard product backlog and PI system. The engineers here have their groupings of work, and they can just pull it off the shelf and start doing it. The various pieces of infrastructure that are relevant to the repositories will get done. What’s nice is you now have democratic consent, so you have a structure that gives permission to ratify a SIP and also to initiate a hard or soft fork.

This is a big thing, and there needs to be a lot of community participation. The point of a members-based organization is that it’s very easy for community members to join. Over 200 have already joined Intersect, and many more will. That organization is just kind of growing, and the point is it’s owned, operated, and run by the members. It’s a not-for-profit members-based organization, and the goal is just to bring everybody together and have a place where they can actually talk about things.

It goes from the working groups to the SIPS to the actual implementations, and there’s an authorizing function either for an aggregation of things or for individual things, depending upon the nature of those things. We already have a really good, hard-fought operating system for how to think about product and how to structure working groups. Step one is just to port that system over here as a starting point, and then the system will have to be customized to meet the needs of a members-based organization like Intersect. This is an open process. Now, there is a question of how we attract open-source commitments and open-source developers.

We want to create a situation where a lot of external contributors, once these Intersect repos are open, can make contributions. One of the ideas we’re discussing with partners like Emurgo and the foundation, as well as the members of Intersect, is the idea of four courses. Once you complete them, you get certificates that are posted on the Cardano blockchain. These would cover Haskell, Agda, open-source best practices, and a mastering Cardano course. The basic idea is to establish a technical baseline showing that how to read and actually work on the code.

You establish a baseline for how everybody works together in the system and a baseline for how Cardano works under the hood: the network protocol, extended UTXO, how Plutus works, how the APIs work, etc. If designed correctly, these types of things should get you up to speed as a core developer. If you can demonstrate enough technical skill, you should deserve some credentials. I like this because it’s open and self-certifying, but it’s merit-based. You have a mechanism to prove you have the skills and knowledge, and anyone can do it.

There’s no gatekeeping function, and once it’s done, you get a blockchain certificate similar to how the Cardano Foundation is starting to do blockchain certificates for their educational materials. This translates to getting credentials. However, you have to be very careful with these projects, as getting credentials typically is a gatekeeping function. Whoever controls that is the Iraqis of open-source projects: he who controls the spice controls the universe. If there’s a collection of great content that we can build together, anyone who wants to start working on Cardano has access to it.

You can also look at the existing pool of developers and grandfather in some who have already been making major contributions to the system. There would be an or, and they just have to decide what that or looks The working groups and technical leadership of Intersect will ultimately have the final say on who gets credentialed and who doesn’t. What’s nice here is that you have a very good working structure where you have a consent engine, and every single ADA holder now has a say on whether they think something is alright. You have an execution engine that has a structure to allow anyone to become an open-source participant and contribute to it. You have a way of creating knowledge transfer so that there are people coming and going, but you always have content for that.

Then you have an operating model that we figured out after the last six years. This operating model is very hard-fought, but it can reliably do releases in a two to eight-week period and structure an accountability model where PI can be public. Every quarter, you can kind of see what’s going on, and you have one universal backlog. This builds on a lot of the systems we’ve already, as a community, figured out together, the SIP process, for example. It builds on that, so basically, we have a nice way of defining standards and change management.

It builds on how we’ve handled a universal backlog and can learn from things like how Hydra has been a very successful open-source program. Then we get to do some cool new things, like blockchain-based certificates to get credentials. You can drive some of the change of Cardano onto the blockchain itself. Now, there’s a big open question of alternative clients and blueprints. Some people want to do a Rust client; I saw on Twitter that some people want to do TypeScript, and that’s great.

But the issue is: what is the canonical Cardano? A major innovation that’s occurring right now with the SIP 1694 work, the Conway era that’s being pushed out with Sanchonat, is that we actually wrote the specifications for these not in prose-like LaTeX but actually in Agda. Six years of research has gone into how to do this, and a lot of software has been written to build workflow for this. The power of Agda is that it’s implementation-agnostic. What that means is that you have blueprints that you can do code extraction from, and you can use that to test and certify.

That’s why there’s this idea of an Agda class: how to do that is a very specialized skill. It doesn’t necessarily require a huge amount of time, and it doesn’t necessarily require a PhD, but you just have to learn it. We have a model that we’ve developed over six years that is IP of the Cardano ecosystem. It’s something we’ve all learned together, we’ve open-sourced it, and it’s a unique thing we’ve brought to the table. This is the heart and soul of how you can safely move to a polyglot ecosystem when you have alternative clients.

You call that polyglot; it basically means many languages. We want to have lots of alternative clients, but we want those clients to conform to standards and quality so that regardless of the client you choose, you have a reasonably the same level of quality assurance that you have from the canonical Haskell client that we’re currently using. This also means that you can ensure certain properties, the upgradability of these clients, so that the hard fork combinator, for example, which gives very easy hard forks, is preserved throughout the entire system. When the government of Cardano says an upgrade actually happens, it’s not a developer over here on Rust or TypeScript or the Haskell side holding the network hostage and not adhering to the will of the Cardano government. This is another big project: to get all of Cardano covered with formal specifications.

We have one level of spec; this is kind of the Agda, and over here is kind of the LaTeX world. Overall, we have math blueprints that are implementation-agnostic, but this is a particularly useful representation. One of the things we’re going to be doing throughout 2024, as part of the GCD agenda, is getting end-to-end coverage of all of the prior work so that there’s one set of universal blueprints and then creating an entire certification pipeline so that third-party clients can certify against it. Then all the users get a guarantee that clients are interoperable with each other. It’s a big house of work; there will be a specialized working group at Intersect to facilitate this, and the formal methods tribe will work very closely with them and gradually work their way through it.

It’s also an opportunity to simplify some of the blueprints of Cardano because, for example, there have been a lot of conversations about retiring some or all of the logic in Byron to massively simplify the design and reduce the complexity of the code. This would mean there’s a much faster time to market for the Rust and TypeScript clients, and there shouldn’t be any consequences to the users of Cardano because those are legacy concerns. These are all some of the conversations. There’s a blueprint conversation, a certification conversation, and a product management conversation about where we should go with Cardano. We have a continuity and GCD backlog that we put together as a suggestion, and as it moves over to the members-based organization, there’s a debate and discussion among all the members, and that becomes a universal backlog.

That universal backlog can be ratified by the government of Cardano, which would be 1694 if ratified. This is kind of a high-level summary of a very complicated thing. Effectively, you’re changing pilots in a fighter jet; it’s moving very fast, and you’re having to do some very complicated maneuvers. But we’re moving to an incredibly decentralized working group, and what’s so cool about Intersect is that it’s made to operate and run by you, the community. We can build buckets and institutions and fill them with some ideas and capital to bootstrap and get it started, but ultimately, this is the organization the community uses to interface with things the code and many different processes.

It’s also a thing that can become a custodian of knowledge, like things required to certify open-source developers and other things. This process can grow and evolve over time because we only have one client, the Haskell and Agda. But if there’s a Rust client, then you can add a Rust course. If you take that, then you get access to the Rust repos, assuming how to read the specs and the best practices and understand Cardano. Of course, these courses will change over time because every year the protocols evolve and get more advanced, and people have to stay up to date, just like doctors or lawyers have to stay up to date.

There’s some notion of continuing education that occurs, and you either prove it by practice or prove it by certification, or both. This is the current thinking that’s in execution. There’s a whole team here to transition the repos, and there’s a whole team on the other side to receive them. There’s a huge backlog effort to get everything bow-tied up, and there’s already a governance and operating system for that backlog, and they’re executing through a PI system on a quarterly basis. This is all in flight; the fighter jet is in the air.

Over here is where the pilots change out. We’re not going anywhere; no engineers are being pulled, no leadership is being pulled. We’re just now adding a force multiplication. It’s going from one set of people to another set of people, and those things are basically growing up. The goal is to give the government of Cardano some options.

Let me change this over here real quickly. Let’s use a different color—how about purple? I’m not sure why that’s… there we go. We want to give some options to the government. Okay, so buckets to fund, and then you have execution vehicles.

The basic idea is that you allow them to run for a transition period, and then the government will look at these things: the accountability, speed, strategy, and resources required. They can vote on whether they want to continue that or vote if they want to pivot that significantly. This also allows a tremendous amount of diversity at the client level, so you can have very different philosophies and people, but it preserves and protects the kinds of things that ensure that Cardano continues to have uptime. It’s been up for over 2,100 days, so you need a SIP process, a proper change management system, and a universal backlog that there’s consensus around. You need to understand that the people doing the work adhere to a certain level of quality, but you need that to be open and follow best practices of an open-source project.

You need a pretty good operating model so that we can consistently continue to get upgrades. One of the things that would be quite tragic is if, during this transition to Intersect, bureaucracy creates a reality where things slow down considerably. The way things are structured right now is that they’ve actually been speeding up quarter by quarter, and there’s been more reliability coming in quarter by quarter. Even though we’re doing more complicated things, things are being delivered in a reasonable time window. I don’t want to lose that as an ecosystem; I think we need to continue moving quickly.

But we do need to be moving with many voices because there are a lot of different opinions about what should be priorities, and sometimes those are contradictory or divergent opinions. It’s very important to have working groups where those opinions can come together, have a discussion, and get all these things done. Of course, there needs to be a way of creating legitimacy for those opinions—not the "trust me, bro" approach, but a vote.