This year, the Pier Two team had the opportunity to attend a variety of events at [Devconnect](https://devconnect.org/) which took place in Istanbul. We are excited to share our experience as well as the next steps for our light client project, Lantern. Devconnect was a week-long gathering organised by the community, featuring a series of events spread out across various locations in Istanbul. The daily agenda was packed with sessions that provided valuable insights into where the ecosystem is heading and what challenges lie ahead. These were provided through presentations, interactive workshops, and captivating panel discussions.

For the Pier Two team, the highlights of the week were conversations around Ethereum's next steps towards scalability, security, and decentralization. Despite varying projects and objectives, everyone's dedication and contributions were evident. All were driving towards the shared goal of enhancing the Ethereum ecosystem. Among the topics that were discussed, we highly valued the discussions around light clients. Light clients are a critical tool that allows users to interact with Ethereum by providing greater privacy, and censorship resistance.

**About Lantern**
This leads us to describe more about our project, Lantern — a C# implementation of a light client for Ethereum. Throughout the conference, we engaged in high-level technical discussions that validated our approach for the light client’s design. These discussions have paved a pathway for us to move forward in the Lantern project. At one of the Devconnect events, the Pier Two team also had the opportunity to [give a presentation about the Discv5 project](https://www.youtube.com/watch?v=LM_uqsSt2F4) and how it relates to the development of Lantern. In the presentation, we explained the challenges of running full nodes for Ethereum, why it requires a lightweight solution, and how Lantern plans to fulfill that. In summary, there are two key technologies that enable this — sync committee and Verkle trees. By relying on these two key technologies, we envision a client that is lightweight enough to run on any device and can provide the following peer-to-peer capabilities:
- Broadcasting transactions to the Ethereum network
- Downloading specific portions of state (e.g. account balances and smart contract related data) from the network
- Sending ETH from one address to another
- Interacting with locally stored smart contract state
- Fetch light client data from beacon nodes to verify if data is canonical

These capabilities make the client ideal for wallet-related use cases. 

**Our Next Steps**
As a starting point this year, we implemented the [Node Discovery (v5)](https://github.com/Pier-Two/Lantern.Discv5/tree/main) protocol in C#. This protocol will be used for building a client that is able to receive light client data directly from beacon nodes. In the beacon chain, there is a group of 512 validators known as the sync committee. The group is responsible for signing the header of a new block every 12 seconds to agree on its correctness. By only tracking the participation of a subset of validators as opposed to the entire validator set, the beacon chain can be followed on resource constrained devices. The signature created by the sync committee can be used to verify whether data such as account balances or smart contract data is part of the canonical chain.

A key characteristic of this client is its fully peer-to-peer nature. Therefore, as long as the nodes in the beacon chain are operating correctly, the light client data would be available for access. The consensus layer network is being run by multiple client implementations, including Lighthouse, Prysm, Lodestar, Nimbus, and Teku. As of today, only Lodestar and Nimbus have implemented the light client specification, which allows their clients to serve light client data over Libp2p. Lodestar and Nimbus only account for 5% of the consensus layer's share. In the coming months, we expect the availability of light client data to improve as other clients adopt the light client specification.

**Conclusion**
Devconnect was an invaluable experience, and the atmosphere in Istanbul was nothing short of inspiring. We return to our project with a clear direction and a determination to make Lantern an essential tool for the public. Stay tuned for updates as we progress, and we hope to see many of you at the next Ethereum event!

One small step for censorship resistance, one giant leap for accessibility

# The CBDC Pilot Report

Reviewing the recent report on the Australian Central Bank Digital Currency Pilot (**Pilot**) run by the Reserve Bank of Australia (**RBA**) and the Digital Finance Cooperative Research Centre (**DFCRC**), my mind goes to two statements. The first from Shakespeare and the second from the political scientist Arthur Bentley.

As a general rule and in the context of Macbeth contemplating a significant decision, Shakespeare portrays Macbeth seeking third-party opinions before deciding whether or not to kill Duncan, in order to become king. Shakespeare writes:

*…I have bought Golden opinions from all sorts of people[.]* - Macbeth, Scene 1, Act 10

Arthur Bentley, while talking specifically about the epistemological study of politics, helpfully summarises why relative limits on inquiries into things do not always make sense:

*Ought we not draw a distinction in advance between politics and other sorts of
social activity so that we can have our field of study defined and delimited at the
outset? The answer is no. Many a child making paper toys has used his scissors too
confidently and cut himself off from the materials he needs. We shall plunge into any
phenomena belonging to the roughly recognised field of government. If any of these
things lead us to interesting paths we shall follow them, heedless of definition.* - Arthur. F. Bentley, The Process of Government (1908), p 199

In essence, it is good to ask questions, to a lot of people, and end up with either an answer you did or didn’t expect or that gives you more useful information - like has been done with the CBDC Pilot. It is correct to solicit widely regarding questions we consider about how to design society, even when replacing good (or current) kings.

## The Pilot

The Pilot’s focus was to engage deliberately with the Australian industry to explore use cases for a CBDC. It was rather limited in scale, with 14 industry participants. The four use case categories that emerged surrounded markets, digital money, smarter payments, and inclusion and resilience in the digital economy. 

Separate from the discrete analysis of what was covered by the Pilot, the inquiry enlivens a broader debate about whether the Government (or statutory bodies) should be further endowed with managing economic interactions while driving fiscal and monetary policy. It probably doesn’t hurt, at least to some degree. Private industry is wonderful at innovation, and the Government is good at making capital available at times of need, although it does need to be rate-limited in its oversight of technicalities (or simply, leave the market be for a while until it understands better and engages with studies like these). This could be by implementing a broader sandbox environment, with legislative exemptions and allowances for innovation, affording the market further time to ferment deeper use cases.

The various use cases covered in the project highlighted a range of areas where
tokenised money could add value to the Australian economy and Australia’s contribution to the global economy, including by facilitating programmable payments, atomic
settlement in tokenised asset markets and offline payments. The project also canvassed opportunities for a CBDC to support the development of new forms of privately-issued digital money (including tokenised bank deposits or CBDC-backed stablecoins) which could address some of the business needs identified in the use case submissions. In this sense, a CBDC could be viewed as an enabling complement to, rather than a substitute for, private sector innovation.

The result of the CBDC Pilot is CBDC’s are useful across most of the modalities tested. Now we need to consider whether it makes our markets more purposeful, reliant, resilient, and complementary to the human experience. The findings seem to be yes, it can make both new and existing markets more efficient and have some ancillary benefits associated with increasing access to the digital economy across our society. 

> **The use cases examined in the project suggested that broad access to a CBDC could support (directly and indirectly) the creation of new or more efficient markets.**

Interestingly, most industry participants preferred that pilot CBDC balances and transactions were private, while a few valued the benefits of transparency that came from the DLT implementation (e.g. to allow independent verification of financial capacity or reserve backing of stablecoins).

## What was Tested

Taking only select examples from the Pilot, the below use cases were tested.

### Atomic Settlements

Imperium Market’s pilot project tested two banks that were paired to trade tokenised debt securities. These trades were settled using a pilot version of the CBDC based on an atomic settlement model. Imperium Markets facilitated the trade by ensuring that payment was made at the same time as the delivery of the securities, a process known as delivery versus payment (DvP). They used the 'allowance' feature on the pilot CBDC platform, which works similarly to a one-time direct debit for a specific amount, authorised by the end users. Imperium Markets did not need to handle or store the pilot CBDC during the transaction. This meant that the end users weren't exposed to any counterparty risk, shifting the legislative substrate as we know it.

The potential for limiting the number of custody swaps when exchanging assets until all criteria are met is good for business. Dropping assets between isolated systems of intermediaries seems to be becoming less relevant. Industrialised trust, through robust systems (whether blockchain, CBDC-backed assets in a uniform environment, or otherwise) could impact capital structures in the same way interest rates do (a lot).

Clearing and settlement in the traditional markets is generally T+2. That is a lot of relatively latent capital, where complex actors may layer and disperse this production risk, but most can not. Closing that gap redistributes capital quicker to market participants rather than market intermediaries. Economies need to move, and participants drive capital transfers in such disparate and increasingly discrete ways, that mixed market economies will benefit from greater availability of dry powder. Intermediaries that simply hand money around, due to lack of a fulsome alternative, aren’t the end state of our economic system. Remember, at one point, Feudalism seemed self-evident. 

### GST Automation

The GST Automation Pilot use explored by the Commonwealth Bank of Australia and Intuit  showcased automated GST collection and payment using a pilot CBDC, initiated by merchants through an in-app invoice. The GST was auto-calculated and directed to an account for the ATO, while the rest was simultaneously paid to the merchant. This demonstrates the CBDC's potential to enhance GST collection efficiency, reduce businesses' compliance tasks, and possibly eliminate the need for Business Activity Statements.

Making tax interactions seamless and compliance makes sense. This, and similar innovations, would be efficiency multipliers in the market, reducing administrative overheads for tax compliance and enabling a redistribution of resources.

### Tokenised Invoices

Wrapping the CBDC, to re-issue a CBDC onto a third-party platform (in most instances in the Pilot, participants were using a private and permissioned Layer 2) outside the immediate Ethereum base layer ecosystem will likely become less relevant once Chainlink’s cross-chain interoperability (**CCIP**), or something similar, becomes ubiquitous.

In the tokenised invoices use case, a wholesale car dealer issued a tokenised invoice, which was partly sold to a financier for optimised working capital. When due, the buyer settled the invoice with a stablecoin backed by Pilot CBDC, auto-calculating and simultaneously disbursing payments to both the supplier and financier. Using a DLT platform with features like Chainlink's CCIP can enhance supply chain finance transparency, efficiency, and reduce errors and counterparty risks.

### Wholesale Distribution

Novatti lead a pilot illustrating the use of pilot CBDC to back the issuance of a stablecoin on a public DLT platform. The CBDC was 'escrowed' in a hash time locked contract, ensuring the stablecoin's value was verifiably backed. The wrapped stablecoin had usage restrictions for approved end users for the purposes of the Pilot.

Wrapping a CBDC will likely be the norm, as it allows fluid movement of tokens across ecosystems, which enhances interoperability and liquidity (and therefore capital efficiency). Chainlink’s CCIP, once fully realised, may limit the requirement for this to be as ubiquitous as the Pilot would suggest. But we will see.

## Legal and Regulatory Findings

During the Pilot, which was structured as a contractual claim on the RBA involving real customers, participants had to meet all legal and regulatory standards before implementing their use cases. Regulatory bodies ASIC and AUSTRAC facilitated this, even providing several exemptions due to the research-based nature of the project. However, this process highlighted certain uncertainties regarding how CBDCs and new business models that use them would fit into existing legal and regulatory frameworks. The Pilot CBDC was launched as a contractual obligation of the RBA, rather than being rooted in a legislative context. Further complexities arose from the Pilot CBDC's utilisation in tokenised digital asset settlements, with confusion surrounding the regulatory status of these digital assets.

> **In many instances there was uncertainty around the legal and regulatory treatment of the digital assets, in particular whether they were regulated ‘financial products’ under the Corporations Act 2001. This had implications for the regulatory status of the use cases.** - Pilot CBDC Report, page 7


Aside from the underlying financial services, including Chapter 7 of the *Corporations Act 2001* (Cth), considerations, rather than the contractual protections offered under the Pilot, it is likely that, among other things, legislative amendments would be required to include customer-held CBDC redress rights under the Financial Claims Scheme administered by APRA.

## Takeaways

Importantly, almost all Pilot CBDC use cases seemed to work and be more efficient than solutions available today.  Moving past the practical applications of the technology, we move to the deeper layer of appropriate regulation and changes required to see the use cases enshrined or empowered by law.

Government is (or should be) reflective of the aggregate actions and views of the constituents empowering it, giving appropriate weight to expert opinions when establishing frameworks to govern how we interact, create, adapt, and live. Emphasising the importance of broad inquiry and consultation, the overarching lesson from the Pilot is that while digital innovations like CBDCs promise enhanced efficiency and functionality, their real-world application requires careful consideration of existing frameworks, potential disruptions, and the broader impact on society.

The CBDC Pilot Report - An Analysis

The topic of client diversity within the Ethereum community has been broadly addressed to increase awareness. As of today, there are 4 implementations of an execution layer client and 5 implementations of a consensus layer client being used in production environments. This is because an Ethereum node is made up of two software clients: the execution layer client (EL) and the consensus layer (CL). With this level of diversity in implementations, the Ethereum network is more resilient to potential bugs or issues that may arise in one client implementation. If one client experiences a software bug or issue, the other clients will continue to function as expected, preventing a network-wide failure. However, for this approach to work, it is crucial that no single client has more than **33% of the total network**.

As a result, the Ethereum community has been actively working towards promoting client diversity and encouraging users to switch to different clients if they are currently using a client that has a large share of the network. The diversity of clients among Ethereum users is heavily influenced by the choices made when running an Ethereum node. If a particular client is found to be more convenient than others, users are likely to continue using it.

However, what if the barrier for running a client was reduced dramatically in terms of hardware requirements? This is one of the main motivations behind the development of [Portal Network](https://ethereum.org/en/developers/docs/networking-layer/portal-network/). The following article describes how Portal Network can help increase client diversity by enabling lightweight access to Ethereum.

## **What is Portal Network?**

Accessing the Ethereum network by running a full node can be a significant barrier for many users due to the hardware requirements. These requirements are fixed and will only increase over time as state growth is permanent until Ethereum has migrated to statelessness. The Portal Network addresses this issue by giving users high degree of flexibility on how much hardware resources they want to use for accessing Ethereum. It achieves this by having four new networks (or sub-protocols): History Network, Beacon Chain Light Client Network, State Network, and Transaction Gossip Network.

Users can run Portal Network clients on resource-constrained devices to access the Ethereum protocol from any of these four networks. Each network is designed to be independent and offers a different set of functionalities. For example, to check the balance of an address, the user would need to connect to the State Network. The design of each network being separate allows users greater customisation with their participation, meaning if they only wish to check their address balance, there is no need to join networks other then the State Network.

## **How Portal Network can help Execution Layer Client Diversity**

There are various sources available for estimating the levels of client diversity within the execution and consensus layer. However, despite improvements in client diversity in both layers over the recent years, there is still room for improvement in the execution layer.

One possible solution to improve client diversity is by providing users with a high level of flexibility. This can be achieved by eliminating the need for running a full node for the purpose of querying state or historical data, or to submit transactions, thereby reducing costs for users who are currently paying node providers for accessing on-chain data. Over an extended period, this method could help improve client diversity levels, as running a Portal Network client can be more convenient for users to run than a full node.

In the long term, once all four networks are fully functional and optimisations have been implemented for achieving performance levels comparable to RPC providers, it may change the infrastructure landscape for dApps or other protocols that depend on Ethereum. If the Portal client is coupled with EVM execution, it will become possible for users to solely rely on the Portal Network for verifying new transactions, providing easier access to the blockchain for more users.

However, one might be concerned whether Portal Network could be vulnerable to the same issue posed by the lack of client diversity. To address this, there are currently three different official implementations being built, namely Trin (Rust), Fluffy (Nim), and Ultralight (Javascript). Once all four networks are live, users will be able to select from any implementations, further increasing the level of client diversity and making the network more resilient against failures due to software bugs.

## Path Towards **Long Term Sustainability**

For Portal Network to have long term sustainability, there needs to be a large number of nodes running the different implementations, distributed across the network. Based on the current statistics from [ethernodes.org](http://ethernodes.org/), there are an estimated 6,825 nodes running in the execution layer. The total count of these nodes can vary, largely determined by the specific reasons why individuals choose to run a full node. If the purpose is to run a validator, then running a full node is an essential requirement. 

The goal with Portal Network is to scale up and support potentially millions of nodes, without having any upper limit to the overall count. In fact, one of the core design objectives behind the development of the Portal Network is to ensure that the overall network benefits as the number of nodes continues to grow. This is because each node has a collective responsibility of only storing a portion of data in their relevant network. With the addition of new nodes, their individual responsibilities can be reduced, as the load of data they handle can be further distributed amongst the expanding network. This capability is not currently available in the execution layer because each node is required to store full the history and state.

In order to achieve such a large node count, it would be necessary to develop a diverse array of tools, each one incorporating a Portal Network client and tailored for a specific purpose. 

## About Lantern

Pier Two is creating Lantern (C#) with the hope of also supporting the Portal Network. Lantern being built in C# could be beneficial for the Nethermind client as it can be embedded directly into its codebase. This would enable users who are running the Nethermind client to sync with the network by fetching historical data from Portal. This also furthers the development for [EIP-4444](https://eips.ethereum.org/EIPS/eip-4444) which it allows execution layer clients to prune history older than 1 year. Once this is implemented, the Portal Network can be used as a decentralised alternative for allowing new nodes to sync using History Network, and thereby reducing the hardware requirements for running a full node.

Portal Network and Client Diversity

Light clients allow users to interact with blockchains without having to run a full node. This provides greater security, privacy, and resiliency compared to a traditional client-server model. For a blockchain to support light clients, it needs to have a consensus mechanism that allows users with limited hardware resources to accurately maintain a view of the chain. Additionally, at the networking layer, full nodes must be able to respond to requests from light clients in a peer-to-peer manner.

However, with this design, an issue arises when the number of light clients in the network increases. With more light clients requesting data, there would be a greater burden on full nodes to serve those additional requests. As a result, this could potentially deteriorate the network's overall health. In practice, this was a noticeable scalability problem that was inherent with the design of [LES (Light Ethereum Subprotocol)](https://github.com/ethereum/devp2p/blob/master/caps/les.md). While full nodes can choose to serve requests from light clients, they lack any incentive to do so, given that light clients do not contribute (only request data) to the network. 

To allow users to interact with Ethereum on resource constrained devices without sacrificing decentralisation, a fundamentally different approach needs to be applied. This article explores problems in the ecosystem concerning access to Ethereum, and the development efforts happening to address these.

## Problems in the Ecosystem

If users want to interact with Ethereum in a decentralised manner, they have **no option other than running a full node**. For the majority, this would be a large undertaking because it involves running both an [execution client](https://ethereum.org/en/developers/docs/nodes-and-clients/#execution-clients) and a [consensus client](https://ethereum.org/en/developers/docs/nodes-and-clients/#consensus-clients). The necessary hardware for this setup demands enough storage space and RAM. These requirements will only continue to increase until Ethereum has completed its migration to [statelessness](https://ethereum.org/en/roadmap/statelessness/). 

However, assuming these requirements have been met, the node then needs to catch up with the network’s view of the chain. It does this by verifying individual blocks until it reaches the most recent one. This process is called syncing and it can take a long time to complete, depending on the chosen method. There have been advancements made by client teams to provide alternative methods for syncing that are faster and usually take a couple of hours. 

Even so, this approach doesn't provide a **comparable user experience** to traditional apps and services. This is primarily the reason why centralised RPCs have been very useful. They enable users to interact with Ethereum by abstracting away the need for setting up a node. However, this convenience comes at the cost of decentralization. By losing decentralization, other key benefits that empower users with sovereignty, such as unrestricted access and privacy, are also lost. It is entirely possible for the RPC provider to limit or reject any number of queries.

These are the problems that are highly worth pursuing solutions for. By having a local solution which allows direct access to Ethereum, the protocol becomes available to anyone globally with just a smartphone device and access to internet. This not only democratizes access but it also paves the way towards a reality where applications can operate in a truly peer-to-peer fashion, utilizing Ethereum as a fundamental source of truth.

## What is Portal Network?

This is the direction being taken by Piper Merriam who is leading the efforts to build the [Portal Network](https://www.ethportal.net/). The Portal Network’s goal is to allow users to access Ethereum-related data on devices with as little computational resources as possible. It is designed to be scalable in a way that as more participants join, the overall network benefits rather than degrades. In terms of the underlying architecture, it consists of four sub-protocols, where each is independent of one another. This includes:

* History Network - Access to historical data such as previous blocks, headers, and receipts
* State Network - Account balances, and smart contract storage
* Transaction Gossip Network - Broadcasting transactions to be included in future blocks
* Canonical Transaction Index Network - Retrieving a transaction based on its hash value

It is important to highlight that participants are not restricted to just one network but are free to join any number of the above mentioned based on their needs and preferences. They would also have full control over how much resources (e.g. RAM, CPU, and storage space) they want to contribute to the network.

The construction of these sub-protocols is possible using the Discv5 protocol which is a peer-to-peer mechanism used by nodes for discovering other nodes within the network. The [Discv5 protocol](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md) is relatively lightweight and allows for building custom protocols. For more information about how Discv5 works, there is a three-part explainer dedicated for this, accessible using the link [here](https://piertwo.com/blog/discv5-explainer-series-1/).

## Stateless Roadmap

As mentioned previously, Ethereum will be transitioning to a stateless model in the future. Under this model, nodes will have flexibility to decide how much “state” they want to store. For reference, the state size in Mainnet is approximately 900GB and is increasing as users deploy more smart contracts, or as existing contracts occupy more storage slots. To address this indefinite growth, this is a much needed upgrade as it can reduce burden of storage requirement on full nodes. 

Under this model, however, if a node is fully stateless, it will be unable to fulfil queries from applications for retrieving account balances or smart contract related data. This opens a path for Portal Network clients that are participating in individual sub-protocols to act as state providers. Furthermore, when a new node wants to sync with the network, it can rely on the history sub-protocol for fetching historical data.

## How Lantern can fit into this?

In an effort to expand access to light clients, and provide redundancy, the ongoing efforts out of Portal Network are exciting. Today, there are teams working on three different implementations, including [Trin (Rust)](https://github.com/ethereum/trin), [Fluffy (Nim)](https://github.com/status-im/nimbus-eth1/tree/master/fluffy), and [Ultralight (Typescript)](https://github.com/ethereumjs/ultralight).

Diversity of programming languages helps to expand developer ecosystems that are part of that language. Having redundancy across implementations also promotes optimisations and helps discover new ways to approach designs. This is where Lantern comes in. To support the development of Portal Network, there can be another implementation built in C# that would focus on allowing users to interact with Ethereum (e.g. verifying on-chain information and sending transactions) without needing centralised RPCs. This implementation would exist as an open-source project.

It's universally agreed that UX in the space needs improvement. This will be a core focus of Lantern. An effort to improve the UX of interacting with Ethereum without sacrificing decentralisation. And some of that work [has begun](https://piertwo.com/blog/discv5-explainer-series-1/). Pier Two have implemented Discv5 in C#, as a library that can be used for implementing the Portal Network specs. The goal will be to expand and fully support all four sub-protocols.

Much of the mindshare today tends to focus on common examples, where problems related to finance, collectibles, assets or identity are being addressed, in fully developed nations. Where energy, compute and political agendas are relatively consistent. As we zoom out, and think about what Ethereum has the potential to unlock, we must also think about scalability, resources and scarcity. Light clients will play a really pivotal role in realising that potential, and providing access to everyone with an Internet connection. Specifically, to provide the maximum number of people direct access to the network, as seamlessly as possible.

Why Light Clients matter

# Pier Two is in Active Grant status, supported by the Ethereum Foundation Ecosystem Support Program (ESP) for implementing Discv5 in C#.

Nodes without full chain history and state data are likely to have poor peer connections and may be disconnected frequently when they cannot provide the requested data to their peers. Not all nodes are created equal. “Light client”, in the context of Ethereum, typically refers to the DevP2P LES network. LES ([Light Ethereum Subprotocol](https://github.com/ethereum/devp2p/blob/master/caps/les.md)) is a protocol for Ethereum blockchain that allows low-power devices, such as smartphones or IoT devices, to access the blockchain network securely and efficiently, without having to download and store the entire blockchain, and without compromising on the security of the network. The DevP2P LES network is designed using a client/server architecture. Servers are service providers and store all of the data. Clients request the data and are purely consumers of this data. With growing demand on Ethereum, including L2s and L3s, if there are not enough servers, the clients cannot get the data required. We need something more scalable, and more reliable.

> **“Like light client technology...I want every Ethereum wallet 2-years from now to be actually verifying the chain and not just be dependent on a central infrastructure provider telling it what the hashes are.”** - Vitalik Buterin, Shapella Hard Fork launch event, April 12, 2023

The Discovery v5 protocol (Discv5) is a mechanism used by the Beacon chain to discover peers in the network. In building and specifying the Kademlia DHT, built on top of Discv5, the “[Portal Network](https://www.ethportal.net/)”, and contributing researchers, are working to address the problem outlined above. By designing decentralised peer-to-peer networks which together provide the data and functionality necessary to expose the standard JSON-RPC API, each additional stateless client that joins the network adds additional capacity to the network. This would expose the APIs necessary for standard "wallet" applications, run on resource constrained devices and doesn't require syncing. Referred to as Functional Light Clients (FLCs). Importantly, these networks are not critical to the operation of the core Ethereum protocol. They simply provide a view into the protocol.

The primary challenge for FLCs is to develop a system that enables each node to store a portion of the data and for the network to provide a way for nodes to locate the nodes that contain the data they require. Whether that is using bloom filters to reduce the amount of data needed to be transmitted between nodes or enabling wider adoption of blockchain technology in developing countries where high-powered devices are not readily available, or to make Ethereum more accessible to users with limited storage or processing power.

For the purposes of the current Ethereum Foundation Ecosystem Support Program (ESP), this is where Pier Two is contributing to implement on Discv5, and has received a grant. Now in an “Active” state.

As part of the Discv5 Grant, the following deliverables for Pier Two are as set:
- A comprehensive and documented understanding of the Discv5 components and design rationale.
- Review the "discv5-wire.md" and understand the wire protocol
- Study the "discv5-theory.md" and familiarize with the algorithms and data structures
- Read the "discv5-rationale.md" to understand the design rationale behind Discv5
- Document and publish the process/findings as Discv5 educational blog posts
	- [Article 1](https://piertwo.com/blog/discv5-explainer-series-1)
	- [Article 2](https://piertwo.com/blog/discv5-explainer-series-2)
	- [Article 3](https://piertwo.com/blog/discv5-explainer-series-3)
- A working implementation of the Discv5 wire protocol.
	- Implement the Discv5 wire protocol as defined in discv5-wire.md
	- Implement the required algorithms defined in discv5-theory.md
	- Test the implementation to ensure it conforms to the specification
	- Refactor and optimize the code as necessary: https://github.com/Pier-Two/Lantern.Discv5

- A production-ready implementation of the Discv5 library in C#.
- Integrate the test vectors defined in discv5-wire-test-vectors.md
- Perform comprehensive testing to ensure the library is production-ready
- Refactor and optimize the code as necessary

Pier Two has been researching the topic of light clients for some time, and will continue to find ways to contribute to important ongoing research. For those interested, you can find a link to an independent article published back in March 2022, “[World of Light Clients](https://piertwo.com/research/world-of-light-clients-ethereum)”. We look forward to sharing more information on this topic, and others in the future.

Follow us on [Twitter](https://twitter.com/PierTwo_com).

Pier Two receives a Grant from the Ethereum Foundation

Have you ever wondered how Ethereum nodes discover each other within the network? If so, this series is for you!

In this first blog post, we explain the Discv5 protocol, focusing on node identity and record management. Our goal is to explain the various components of the protocol in an accessible way, even if you're not familiar with cryptography or networking. Some prior knowledge of Ethereum basics may be helpful for following along.

We'll cover a wide range of topics, from encryption to topic advertisement and lookup, using examples to illustrate key concepts. By the end of this series, you'll have a solid understanding of how Discv5 works and why it's vital for the Ethereum ecosystem.

## **Intro to Discv5**

One of the key components of Ethereum is its peer-to-peer communication system, which relies on a set of defined networking protocols. A networking protocol consists of rules that dictate how communication between two computers should happen. In the context of Ethereum, these protocols determine how nodes send or receive messages. For instance, one protocol specifies how nodes can discover each other on the network and exchange information about their capabilities and status. Another protocol defines how nodes can request and deliver blocks and transactions to each other. By following these protocols, nodes can communicate in a secure and compatible way. One of these protocols is Discv5, which stands for Discovery Protocol V5.

### **What is Discv5?**

To understand what Discv5 is and how it relates to Ethereum, we can think of Ethereum as two layers: execution and consensus. In both layers, there are a collection of networking protocols that work in parallel for processing incoming requests and sending responses. In the execution layer, there is Discv4 (the previous version of Discv5) and DevP2P (a stack of protocols). The consensus layer uses Discv5 and LibP2P (a stack of protocols). 

Discv5 is one of these protocols that enables nodes to discover each other to form a peer-to-peer network. Nodes using Discv5 can act as both a provider and a consumer of data and can communicate directly with other nodes without intermediaries. This allows for decentralisation, which means that there is no central authority or server that controls the network or its resources. 

It was originally designed for Ethereum, but it can be applied to any network that needs a mechanism for discovering peers (nodes) and for connecting with specific peers that provide certain services. In comparison to its previous version Discv4, Discv5 is the latest version which provides additional features such as encrypted communication using cryptography and allowing nodes to broadcast information about their services  in a scalable way to the network. 

To learn more about its technical details, we will explore how nodes identify and manage each other's records in the network.

## Node Identity and Record Management

To participate in the Discv5 protocol, nodes need to create and maintain an identity that follows a standard format called Ethereum Node Record (ENR). This format is specified in **[EIP-778](https://eips.ethereum.org/EIPS/eip-778#:~:text=This%20EIP%20defines%20Ethereum%20Node%20Records%2C%20an%20open,port%20numbers.%20No%20other%20information%20can%20be%20relayed.)**.

You can think of an ENR as a profile card that has some basic information about a node, like its IP address, port number, and what protocols it supports. We will use the term **record** to refer to an ENR for simplicity. Nodes can share their records with other nodes to make it easier to connect and communicate. The following part explains the contents of a record:

- **`Signature`** - This is a way of proving that the record belongs to a certain node and that it has not been tampered with by anyone else. It is created by using a cryptographic method that involves applying a private key (known only by the node) to generate a unique code (called signature) from the record data. The signature can then be verified by anyone who has access to the public key (which is part of the record) using a matching cryptographic method .
- **`Seq`** - A number that indicates how fresh the record is. Whenever the node updates any of the fields in its record, it increases this number by one. This way, other nodes can tell which record is more recent and avoid using outdated information.
- **`Key/Value pairs`** - A data structure that stores various attributes of the node, such as its IP address, port number, and supported protocols. Each attribute has a key (a short name) and a value (the actual data). For example, the key **`ip`** might have the value **192.168.0.1**. The key/value pairs can be customised to suit different needs and preferences of the node.

The table below shows the common key/value pairs that are used by the Discv5, as defined in the EIP:

| Key | Value  |
| --- | --- |
| id | This is a short label that tells other nodes what kind of identity scheme the node uses. For example, "v4" means that the node uses a version 4 identity scheme. The identity scheme defines how nodes create and verify their identities. |
| secp256k1 | This is a long string of letters and numbers that represents the node's public key. The public key is part of a cryptographic system that allows nodes to prove their identity and sign messages. The public key can be shared with anyone who wants to communicate with the node. |
| ip | This is a series of numbers separated by dots that identifies the node's location on the internet. For example, "192.168.0.1" is an IP address. The IP address helps other nodes find and connect to the node over the internet. |
| tcp | This is a number between 0 and 65535 that specifies the TCP port used by the node. TCP stands for Transmission Control Protocol, which is a method of sending data over the internet reliably and in order. The TCP port helps other nodes establish a TCP connection with the node for data exchange. |
| udp | This is a number between 0 and 65535 that specifies the UDP port used by the node. UDP stands for User Datagram Protocol, which is another method of sending data over the internet quickly and without waiting for confirmation. The UDP port helps other nodes send UDP packets to the node for data exchange. |
| ip6 | This is an alternative way of writing an IP address using hexadecimal digits and colons instead of decimal digits and dots. For example, "2001:db8::1" is an IP6 address. IP6 stands for Internet Protocol version 6, which is a newer version of IP that can support more devices on the internet than IP version 4. |
| tcp6 | This is similar to tcp, but it works with IP6 addresses instead of IP4 addresses. |
| udp6 | This is similar to udp, but it works with IP6 addresses instead of IP4 addresses. |

While **`id`** key is mandatory, all other keys are optional, meaning they may or may not be there in a node’s record. As mentioned earlier, this record format is flexible by allowing adding custom key-value pairs specific to network or application. This lets nodes in the network include more information as needed.

You might wonder about the purpose of different keys, such as **`ip6`**, **`tcp6`**, and **`udp6`**. In some networks, multiple devices may share the same public IP address, which can make it difficult to connect to specific devices. This problem is solved by including both IPv4 and IPv6 addresses in the record. This ensures that nodes can be reached from both types of networks, making it easier for devices to connect to each other.

### Kademlia-Inspired **Routing Table**

In this section, we will learn how nodes keep track of other nodes in the network. Each node maintains a data structure called a **routing table**. The design of this table is inspired by the Kademlia DHT (Distributed Hash Table) that is commonly used by distributed file sharing or data storage systems. The Kademlia DHT stores key-value pairs, where the key is the hash of the content and the value is the actual content (e.g. file). The keys in the DHT are organised based on the closeness of node IDs. We will discuss this more in a moment, however, it is important to mention how the routing table in Discv5 differs from Kademlia DHT and why it is used in the first place. 

In the Discv5 protocol, nodes neither store files nor provide storage to other nodes. Instead, they maintain a routing table with records (ENRs) that are relayed according to the requested distances from other nodes. This process enables the discovery of new nodes through a scalable data structure that grows with the number of nodes. No matter the size of the network, any node can be found using a lookup mechanism. Now we can look at the routing table and what is involved in maintaining it.

A routing table is like a phone book that stores the records of other nodes. If we recall from the previous section, each record has an ID that uniquely identifies the node. Nodes with similar IDs are considered **neighbours** because they are close to each other in the network's logical structure, not in terms of physical distance. To measure the closeness between two nodes, we use a mathematical operation called **XOR** to calculate a "distance" between their IDs. XOR stands for **exclusive or**, which is a logical operation that takes two inputs and returns **true** if only one of its inputs is **true**. If both inputs are **false**, the result is also **false**. We can use XOR to compare two node IDs and determine their relative distance within the network's logical organisation. Let's see an example with two node IDs:

```markdown
Node A = 10
Node B = 12
```

To calculate the XOR of these node IDs, we first write them in binary form (using only 0s and 1s). This means converting the values 10 and 12 to their binary equivalents:

```markdown
Node A = 1 0 1 0
Node B = 1 1 0 0
```

Next, we compare each pair of bits (0 or 1) at the same position and write down a 0 if they are the same or a 1 if they are different:

```markdown
 1 0 1 0
 1 1 0 0
---------
 0 1 1 0
---------
```

So, the XOR distance between Node A and Node B is 0110, which is 6 in decimal notation. A larger XOR distance indicates that more bits are different, while a smaller XOR distance signifies that the nodes are closer to each other. This is how the routing table stores and retrieves records of other nodes based on their proximity.

### **Record Management**

Building upon the routing table concept, we now delve into the specifics of how nodes manage the information about other nodes within the network. The routing table is organized into multiple buckets, each containing up to 16 node records. Each node can have up to 256 buckets in its routing table, with each bucket corresponding to a different distance range from the node, as measured by the XOR of their public keys.

The buckets are ordered by increasing distance from the node, so the first bucket contains the closest neighbours and the last bucket contains the farthest ones. Within each bucket, the node records are sorted by their last seen time, with the most recently seen ones at the back and the least recently seen ones at the front. This organisation allows the node to keep track of which neighbours are active and which ones are stale.

When a node learns about a new neighbour through a message or a lookup, it attempts to add the neighbour's record to the appropriate bucket based on distance. However, if the bucket is already full, the node does not simply overwrite an existing record. Instead, it checks if the least recently seen record in that bucket is still alive by sending a **PING** message (these will be explained in the next blog post). If a response is received within a certain timeout period, the record is considered still valid and remains in place. If no response is received or an error message is obtained, the record is deemed invalid or unreachable and gets replaced with the new record.

The routing table is dynamic and constantly updated as nodes join or leave the network or change their attributes. To maintain an accurate view of its neighbours, a node periodically refreshes its buckets by performing lookups for random targets within each distance range. This process enables the node to discover new nodes that have joined or moved closer to its vicinity and remove old nodes that have left or moved farther away.

## **Continue Reading**

In this blog, we've covered the basic concepts of Discv5, such as how nodes create and verify their identities, how they store and update information about other nodes using Ethereum Node Records and routing tables, and how they manage the records they share with the network. These concepts are essential for understanding the discovery and communication processes of Ethereum nodes.

In our next post, we will explore how nodes communicate with each other using encryption and discuss the types of messages they relay to one another. Here's the link to **[part two](https://piertwo.com/blog/discv5-explainer-series-2)**.

The Discv5 protocol is implemented in the software of nodes according to the developer specification. For those curious, the official specification resides in the [**Devp2p GitHub repository**](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md).

We'd love to hear your thoughts on this series or any questions you may have. Feel free to share your comments below or reach out to us on social media. If you're interested in contributing to our C# implementation of Discv5, visit our [**GitHub repository**](https://github.com/Pier-Two/Lantern.Discv5/tree/main) for more information.

Furthermore, if you're interested in learning more about Ethereum or blockchain technology, here are some resources you might find helpful:

- [Learn Hub | ethereum.org](https://ethereum.org/en/learn/)
- [GitHub - ethereumbook/ethereumbook: Mastering Ethereum, by Andreas M. Antonopoulos, Gavin Wood](https://github.com/ethereumbook/ethereumbook)
- [Ethereum Stack Exchange](https://ethereum.stackexchange.com/)

Discv5 Explainer Series: #1

In our last discussion, we introduced the Discv5 protocol and explained how nodes manage their identity and their neighbours. By using an identity system, nodes are able to establish secure and verifiable connections with other nodes in the network. As we move forward, we'll dig deeper into how nodes find each other and communicate securely in this follow-up article. We'll examine the different types of messages and packets that are relayed in the network, how connections start, and how communication sessions are set up using Discv5. By the end of this article, you'll have a clear understanding of how nodes discover each other and communicate in a peer-to-peer manner.

## **Messages and Packets Overview**

### Messages

The Discv5 protocol supports various message types that nodes can send or receive. Each type of message has a matching pair: one for requesting something and one for responding to it. For example, a node can ask another node for its metadata (information about itself) and get an answer with the metadata. When a node receives a message, it uses a number called the **message ID** to identify what type of message it is. Then, it sends back the appropriate type of message. These messages play a crucial role in maintaining the network's connectivity, allowing nodes to locate suitable peers, share information, and synchronise data. The table below lists all the types of messages that Discv5 supports and their IDs.

| Message Type | Message ID  | Description |
| --- | --- | --- |
| PING Request | 1 | A message that is sent periodically to verify if the other node is still active. It also shares updated information about the sender's status and how to reach them. |
| PONG Response | 2 | A message sent back after receiving a PING request, confirming that the other node is working and sharing its own status and contact information. |
| FINDNODE Request | 3 | A message sent to locate other nodes that are a specific XOR distance away from the recipient node's ID, helping to establish and maintain connections within the network. |
| NODES Response | 4 | A message sent in reply to a FINDNODE or TOPICQUERY request, containing a list of nodes that meet the requester's criteria. By sharing this information, it strengthens connections, expands the network reach, and ultimately improves network discovery. |
| TALKREQ Request | 5 | A message sent with a particular set of rules for data sharing between nodes, tailored for a specific application or service. This allows nodes to connect and use different services offered by other nodes, making communication efficient and supporting a variety of applications within the network. |
| TALKRESP Response | 6 | A message sent as a reply to a TALKREQ message, providing the requested information or a simple acknowledgment if the receiving node doesn't support or recognise the requested communication rules. |
| REGTOPIC Request | 7 | A message sent to express interest in a specific discussion topic, requesting a ticket if available. This allows nodes to join communication channels focused on particular subjects, connecting them with other nodes that share similar interests and fostering topic-based interactions within the network. |
| TICKET Response | 8 | A message sent in reply to a REGTOPIC request, providing either a ticket and wait-time for registration or immediate registration confirmation. This manages topic sign-ups efficiently by organizing interested nodes, allowing them to join topic-based communication channels efficiently and maintaining an orderly network structure. |
| REGCONFIRMATION Response | 9 | A message sent after a successful topic registration, informing the requester about their registration status and sharing relevant details about the topic channel. This ensures that the nodes are aware of their participation in the topic-based communication channel. |
| TOPICQUERY Request | 10 | A message sent to search for nodes registered to a specific communication topic, helping nodes find others with similar interests. |

### Packets

One of the key aspects of secure communication between nodes in the Ethereum network is how messages are encrypted and structured. The Discv5 protocol does not send messages as plain text; instead, it uses a shared session key to encrypt them. This session key is established through a handshake mechanism that we will explain later. 

Secure communication within the Ethereum network relies on the proper use of packets to handle messages. These small units of data contain the essential information for maintaining encrypted and authenticated communication. The Discv5 protocol, in particular, uses specific packet structures to ensure that communication between nodes stays both secure and reliable.

Each packet in the Discv5 protocol has two main parts: a header and an optional encrypted and authenticated message. The header component contains an **Authdata** section which can have a variable size, depending on the packet type. In essence, the header provides the necessary information for the receiving node to correctly process and handle the packet. There are three main packet types in the Discv5 protocol:

1. **Ordinary message packet:** This packet type securely sends encrypted and authenticated messages between nodes. In ordinary message packets, the Authdata section contains the source node ID, which identifies the sender and ensures the communication remains intact.
2. **WHOAREYOU packet:** If the recipient of an ordinary message packet encounters issues with decryption or authentication, they can send a **WHOAREYOU** packet. These packets ask the sender for additional information to confirm their identity so they successfully decrypt or authenticate the content in the packet. In **WHOAREYOU** packets, the Authdata section has an ID nonce (a random value) and the requesting node's `**Seq**` number, both essential for identity verification.
3. **Handshake message packet:** After a **WHOAREYOU** packet is sent, handshake message packets are used to set up a new session between nodes and securely transmit both the handshake-related data and the original encrypted/authenticated message. In handshake message packets, the **Authdata** section includes a fixed-size authdata-head component, an ID signature, a temporary public key, and, if needed, an optional node record. Together, these components ensure a secure and authenticated connection between nodes.

Now that we've covered the various message and packet types, it's important to understand how nodes actually relay packets to each other.

## **UDP Communication**

In our previous blog, we discussed that a protocol is a set of rules for sending and receiving data across a network. The Discv5 protocol uses the User Datagram Protocol (UDP) to send packets between nodes. UDP is a simple and efficient way to send data without needing a dedicated connection, making it great for tasks like online gaming or streaming that need quick response times.

For more background, we will briefly explain how the internet protocol works. The Internet Protocol (IP) is the basis of the Internet Protocol Suite and plays an important role in moving data across networks worldwide. It works by giving a unique IP address to each device so they can exchange data packets. UDP and another protocol called the Transmission Control Protocol (TCP) are built on top of IP, providing different features for data transmission. IP is widely used in many applications, from web browsing and email services to file transfers and streaming, making it a key part of the modern internet structure.

UDP is all about speed and simplicity, while TCP focuses on reliable and connection-oriented data transfers. With TCP, devices first create a dedicated connection before exchanging data, which ensures that data packets are delivered reliably. However, this extra reliability can lead to slower response times and increased processing overhead.

For Ethereum's node discovery process, where the goal is to quickly find and communicate with other nodes in the network, UDP is a better choice. Although it doesn't guarantee that data packets will always be delivered or that data transmission will be reliable, UDP's speed and simplicity make it more suitable for this purpose.

Below are key points about UDP communication in Discv5, which nodes rely on for efficient and reliable communication:

1. **Packet size limits:** Discv5 has maximum and minimum packet size limits to ensure efficient and reliable communication. The protocol is designed to handle most messages within these size constraints, preventing excessive data transfers that could slow down the network or lead to packet loss.
2. **Short timeouts:** Low-latency communication is essential for maintaining an efficient network. As a result, short timeouts are placed on request/response interactions to ensure quick and smooth communication between nodes. These timeouts help prevent nodes from waiting indefinitely for responses and allow them to adapt to changing network conditions.
3. **Response destination:** When a node sends a response to a request, the response is sent back to the original requester's address, ensuring that the communication loop is properly closed. This practice helps maintain the integrity of the communication process and allows nodes to keep track of their interactions with other nodes in the network. 

### **Bootstrapping Process**

When a node starts for the first time, it is preloaded with a list of known records (ENRs). These records provide the node with an initial set of nodes to connect to within the Ethereum network, serving as a starting point for discovering additional peers. To begin the node discovery process, the node sends a **PING** message to these known nodes, checking their status and availability.

The **PING** message serves as a lightweight way to determine if a node is active and reachable, helping to establish an efficient network topology. It is important to note that the nodes in the preloaded list may not always be online or available, so the discovery process is essential for maintaining a reliable and up-to-date network.

Before nodes can communicate securely using the Discv5 protocol, they must establish a shared set of session keys. This is achieved through a process called a **handshake**, which involves an authenticated key agreement protocol based on elliptic curve cryptography. The handshake is a series of steps that nodes follow to confirm each other's identity and create the necessary cryptographic keys for secure communication. The handshake is essential in Discv5 because it prevents unauthorised access to the messages exchanged between nodes, protects sensitive information, and ensures the reliability of their communication.

In the following section, we will detail the specifics of the handshake mechanism, explaining each step and its importance in maintaining secure communication . 

### Handshake Mechanism

Before two nodes can communicate, they must establish a secure session for exchanging information. The handshake process ensures that both nodes can trust each other, communicate securely, and verify each other's identities. This process is particularly important in decentralized networks, where nodes need to establish trust without relying on a central authority. 

**Initiating the Handshake**

Suppose Node A wants to communicate with Node B, for instance, to send a **FINDNODE** message. First, Node A checks if it already has the session keys from previous interactions. If it has the session keys, Node A encrypts its request with those keys. If not, Node A initiates the handshake by sending a message packet with random content. This message serves as a signal to Node B that a new secure communication session needs to be established.

In response, Node B sends a challenge in the form of a **WHOAREYOU** packet, which includes a unique ID nonce value and the sequence number of Node A's record, if available. The nonce value helps Node A to trace the challenge back to the initial request packet, while the sequence number allows Node A to determine if it needs to send an updated record to Node B.

**Responding to the Challenge**

Upon receiving the challenge, Node A confirms that Node B is active and ready to engage in the handshake. Node A then resends the initial request as a handshake message packet containing three parts: an ID signature, a temporary public key, and its node record, if necessary.

The ID signature is created to prove that Node A controls the private key associated with its node record. This signature prevents the replay of the handshake, ensuring that each handshake is unique and secure.

The temporary public key is generated by Node A as part of the Diffie-Hellman key agreement, a cryptographic method that allows two parties to establish a shared secret over an insecure channel. This temporary key will be used by both nodes to derive the shared session keys for secure communication.

**Deriving Session Keys**

Node A and Node B derive new session keys using a Diffie-Hellman key agreement and the HKDF key derivation function, a method to extract and expand cryptographic keys. This process involves Node A generating a temporary private key and its corresponding public key while Node B uses its static private key. The combination of these keys allows both nodes to create a shared secret that will be the basis for the session keys.

The session keys are divided into initiator and recipient keys, ensuring that each node has a unique key for encrypting and decrypting messages. This enhances the security of the communication by preventing unauthorised access to the exchanged information.

**Completing the Handshake**

Node B receives the handshake message packet and verifies the ID signature against Node A's public key. After successful verification, Node B attempts to decrypt and authenticate the message using the newly derived session keys. If the decryption and authentication are successful, Node B considers the new session keys valid and replies to the message, for example, with a **NODES** message.

Finally, Node A receives the response message and authenticates/decrypts it using the new session keys. If this process is successful, both nodes have verified each other's identity and can now communicate securely using the established session keys.

This handshake process is vital for maintaining secure communication sessions between nodes in decentralized networks. It allows nodes to verify each other's identities, establish trust, and ensure that information exchange is more secure and reliable, even in the absence of a central authority.

### Continue Reading

In this blog, we covered key aspects of Discv5's node communication like messages, packets, and UDP for efficient, reliable data exchange. We also discussed the bootstrapping process for new nodes to join the network and the secure handshake mechanism ensuring network integrity and security.

In our final post, we will discuss how topics are structured, the role they play in enhancing the discovery process, and how nodes manage their subscriptions. Here's the link to **[part three](https://piertwo.com/blog/discv5-explainer-series-3).** 

As always, we'd love to hear your thoughts on this series or any questions you may have. Feel free to share your comments below or reach out to us on social media. If you're interested in contributing to our C# implementation of Discv5, visit the official **[GitHub repository](https://github.com/Pier-Two/Lantern.Discv5/tree/main)** for more information. 

If you're looking for more resources on Ethereum or blockchain technology, you can refer to the list provided in our **[first blog post](https://piertwo.com/blog/discv5-explainer-series-1)**.

Discv5 Explainer Series: #2

In our previous articles, we introduced you to the Discv5 protocol, covering topics such as node identity, record management, and communication. However, in order to fully understand the protocol, it's important to delve deeper into the role of topics in improving the discovery process. Topics provide a way for nodes to group related information and make it easier for other nodes to find and subscribe to that information. In this article, we will explain **topics** in detail, discussing their background, how nodes manage them using registration and queue, and how they are advertised across the network. By the end of the blog, you'll have a complete understanding of the Discv5 protocol and how it works in practice.

## Background

In a peer-to-peer network like the one used by Ethereum, nodes often need to find and connect with other nodes that offer specific services or share common interests. This is important for keeping the network running and enabling tasks like transaction validation, sharing blockchain data, and executing smart contracts. To make this process easier, Discv5 uses **topics** – simple strings that act like labels to show the services or interests provided by a node. These topics give a clear and standardised way to describe a node's abilities, making it simpler to find and interact with other nodes.

When a node makes itself visible under a specific topic, it is said to **place an ad** for itself. Depending on the application's needs, a node can advertise multiple topics, a single topic, or even no topics at all. Every node taking part in the Discv5 discovery protocol acts as an advertisement medium. This means they accept topic ads from other nodes and later return them to nodes looking for the same topic. By organizing various services and interests, topics improve the effectiveness of finding and connecting with other nodes in the network, ultimately supporting the overall performance and strength of the system.

Now, we will look at how these topics are stored and managed within the Discv5 protocol.

## **Topics and Management**

### **Topic Table and Queue**

At the core of topics in Discv5 is the **topic table**, a data structure that stores advertisements for a multitude of topics. Each topic within the table is associated with its own list of ads, known as the **topic queue**. As the name suggests, topic queues operate on a First-In-First-Out (FIFO) basis with a limited length, ensuring that the oldest ads are removed to make room for new ones when the queue reaches its full capacity.

To better understand this concept, let's visualise a topic table containing three different topic queues for topics T1, T2, and T3. The queue for topic T1 is filled to capacity and cannot store any more ads until the older ones are removed to free up space. In contrast, the queues for topics T2 and T3 may have varying levels of capacity available to accommodate new ads.

Each time a new ad is added to a topic queue, it is placed at the end of the queue. As ads are removed based on their age, the oldest ad is removed from the front of the queue, allowing the subsequent ads to move up in the queue. This FIFO mechanism ensures that each advertisement has a fair chance to be discovered and utilised by nodes searching for services or interests associated with a particular topic.

By organizing ads in this manner, the topic table not only maintains an efficient system for storing and managing topic advertisements but also provides a standardised method for nodes to search and discover services offered by their peers (connected nodes) in the network.

### **Ticket System**

In Discv5, a ticket system is implemented to manage ad registrations efficiently. Ads are placed in a queue and remain there for a fixed 15-minute period. This system ensures that ads get equal exposure and helps maintain a fair advertisement process.

When a node wants to place an ad, it has to wait for a specific duration before the ad is accepted. This waiting time depends on the availability of slots in the ad table and the topic queue. For more context, a slot is a position in the ad table where an ad can be displayed. To keep track of this waiting time, the node receives a ticket that indicates how long it should wait before its ad gets approved. The node must hold onto this ticket and present it once the waiting time has elapsed.

The waiting time for ad registration is determined by these rules:

1. If the ad table is full, the node must wait for a slot to open up based on the oldest ad's remaining lifetime. This ensures that ads are replaced in a timely manner and provides equal opportunity for new ads.
2. If the topic queue is full, the node's waiting time is calculated as the target-ad-lifetime minus the lifetime of the oldest ad in the queue. This helps maintain a balance between different topics and prevents any single topic from dominating the queue.
3. If there are available slots in the table and the topic queue, the ad can be placed immediately without any waiting time.

Tickets are issued by nodes and contain essential information to ensure proper ad registration. These tickets serve several purposes: they confirm that the node requesting the ticket is the one attempting to use it, ensuring a fair process for all participants. They also validate that the ticket is applicable for only one topic, preventing misuse for multiple ads or topics. Additionally, tickets can be used within the registration window, ensuring timely ad placements, and they cannot be used more than once, preventing any unfair advantage or repeated ad exposure.

To enhance security and protect ticket information, details such as waiting time and the node's ID can be encrypted using a secret key. Employing this method guarantees that only the intended node can access and use the ticket for ad registration.

### **Advertisement Protocol**

Now, let's take another look at some messages from a previous blog post to explain how the process of topic advertisement works in Discv5. The advertisement protocol in Discv5 is a mechanism that allows nodes to discover and connect with others interested in the same topics. We'll use an example from the **[theoretical section](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-theory.md#advertisement-protocol)** of the official Discv5 specification to illustrate this process. 

Imagine three nodes, A, B, and C, interacting with each other. Node A provides a service related to a topic called "Transaction relayer." To advertise its service, Node A decides to use Node C as the advertising medium. Meanwhile, Node B is on the lookout for services related to transaction relaying. Let's examine the step-by-step interaction between nodes A and C as they work together to advertise the topic:

1. **Node A initiates registration**: Node A initiates the advertisement process by sending a **REGTOPIC** message to Node C, requesting to register the topic "Transaction relayer." At this stage, Node A doesn't include a ticket in the message, as it hasn't received one yet.
    
    ```
    Node A -> Node C: REGTOPIC ["Transaction relayer", ""]
    ```
    
2. **Node C responds with a ticket**: Node C processes the request and sends a **TICKET** message to Node A, providing a ticket and a waiting time. The ticket is an identifier that Node A will use in subsequent interactions with Node C, while the waiting time indicates how long Node A should wait before proceeding to the next step.
    
    ```
    Node C -> Node A: TICKET [ticket, wait-time]
    ```
    
3. **Node A waits and re-sends registration**: Node A waits for the specified waiting time and sends a new **REGTOPIC** message to Node C. This time, Node A includes the received ticket in the message to verify its identity and to show that it has followed the required waiting time.
    
    ```
    Node A -> Node C: REGTOPIC ["Transaction relayer", ticket]
    ```
    
4. **Node C sends a new ticket**: Node C verifies the ticket and replies with another **TICKET** message, providing a new ticket and a new waiting time. Node A stores this new ticket to be ready for a confirmation call. This step is necessary to further ensure the fairness of the advertisement process and to prevent spamming.
    
    ```
    Node C -> Node A: TICKET [ticket, wait-time]
    ```
    
5. **Node C confirms registration**: After the registration window ends, Node C selects Node A as the registered node for the topic "Transaction relayer," based on factors such as ticket priority and available spots. Node C adds Node A to its topic queue and sends a **REGCONFIRMATION** message to Node A to confirm the successful advertisement registration.
    
    ```
    Node C -> Node A: REGCONFIRMATION ["Transaction relayer"]
    ```
    

During this process, nodes A and C exchange **REGTOPIC**, **TICKET**, and **REGCONFIRMATION** messages, along with waiting times to manage the advertisement registration. When Node B searches for the topic "Transaction relayer," it will send a **FINDNODE** message to Node C, which will then provide information about Node A's registration. As a result, Node B can connect to Node A and access the desired service.

### **Conclusion**

In this final part of our Discv5 Explainer Series, we covered how topics are used to enhance the discovery process in the Ethereum network and how nodes manage their subscriptions. Topics play a crucial role in maintaining a dynamic and healthy network, enabling nodes to efficiently find and connect with others that share their interests. 

When you combine all the various components discussed in the previous blogs, such as node identities, routing tables, and secure communication mechanisms, Discv5 emerges as a powerful and robust protocol for enabling efficient, secure, and scalable peer-to-peer communication within the Ethereum network.

We hope you enjoyed this series and now have a better understanding of the Discv5 protocol. If you have any questions or comments, please share them below or reach out to us on social media. For those interested in contributing to our C# implementation of Discv5, visit the **[GitHub repository](https://github.com/Pier-Two/Lantern.Discv5/tree/main)** for more information.

Thank you for following along, and happy learning!

Discv5 Explainer Series: #3

*NOTE: a version of this post was originally published on 4th June 2021. Details are likely to be superseded by more recent information. Check [Arbitrum Docs](https://developer.arbitrum.io/inside-arbitrum-nitro) for more recent updates.*

Arbitrum is a layer 2 scaling solution for Ethereum. The following read will give you a thorough understanding of the mechanisms that power [Arbitrum](https://arbitrum.io/) to give you more comfort and understanding in an L2 environment.

## Architecture Summary

*   As Arbitrum (L2) exists as a scalability solution for Ethereum (L1), the Arbitrum architecture naturally exists in part on L1 and in part on L2.
*   The component of Arbitrum that exists on L1 is the EthBridge, which is a set of Ethereum contracts.
*   The EthBridge is responsible for refereeing the Arbitrum Rollup protocol, as well as maintaining the inbox and outbox of the chain.
*   The inbox and outbox of a chain is what allows users, L1 contracts, and full nodes to send their transactions to the chain as well as observing the outcome of those transactions.
*   The Arbitrum Virtual Machine is the gateway between L1 and L2, and the function provided by the EthBridge.
*   The AVM is what is capable of reading inputs, and executing computations on these inputs to produce outputs.
*   ArbOS is run on top of the Arbitrum Virtual Machine, and is responsible for ensuring the execution of smart contracts on the Arbitrum chain.
*   ArbOS exists completely on L2, and runs EVM contracts just like how they would be run on Ethereum.

![Layer 1 to Layer 2.](https://storage.googleapis.com/mycelium-bucket/diagram_1_9294177094/diagram_1_9294177094.png)

> *A high level view of the Arbitrum architecture.*

## The Rollup protocol

*   The order of messages in the chain’s inbox determines the result of the transactions.
    *   Subsequently, anyone who is watching the inbox is able to know the results of the transactions, simply by self-executing them.
*   The Rollup protocol is responsible for confirming the results of transactions that have effectively already occurred.
*   Users that participate in the protocol are called validators; if a validator stakes Eth into a deposit contract, they become a staker and can stake on blocks in the Rollup chain.
    *   Both the roles of validator and staker are permissionless.
*   In terms of security, only a single honest validator is required to force the correct execution of the chain.
    *   This gives the rollup chain the same degree of trustlessness as the Ethereum main chain.
    *   Arbitrum assumes the existence of at least one honest validator.
*   The Rollup protocol acts upon the Rollup chain, which is a chain of rollup blocks existing separately to the Ethereum chain.
*   It is the role of the validators to propose new blocks to be added to the chain.
*   Every block that is proposed will eventually be confirmed or rejected by the protocol.
*   Each block is made up of a number of fields, and apart from the block number field, the data given in each of these fields are assertions made by the block’s proposer which may or may not be correct.
    *   If any of the asserted fields are incorrect, the protocol will eventually reject the block.
*   Once a block is proposed it receives a deadline for confirmation.
*   If a validator disagrees with the block, they should propose their own correct block which earns the honest validator some reward when they eventually end up in a fraud proof with the incorrect block.

## Staking

*   For a staker to add a rollup block to the chain, they must place their stake on the block they are adding.
*   Staking is permissionless, anyone can stake on any block where staking is possible.
*   Once you are staked on a block you cannot withdraw your stake until that block has been confirmed.
*   When you stake a block, you are confirming that a block is correct, AND every block in the chain between the most recent confirmed block and the block that you are staking is correct.
*   If a block that you stake on is incorrect, or a block in the chain between the most recent confirmed block and the block that you are staking on is incorrect, you will forfeit your stake.
*   If you are not staked on a block, you can stake on the most recent confirmed block.
*   If you are staked on a rollup block, you can extend your stake up to any successor of the block you are staked on.
*   The amount needed to stake is dynamic.
    *   There is the base stake amount that is specified as a parameter in the Arbitrum chain, which will be used most of the time.
    *   As a security measure to prevent an attacker slowing down the network despite losing their stake, the stake amount is multiplied by a factor that increases exponentially with respect to time, since the deadline for the first unresolved node passed.
    *   This is to increase the cost of such an attack during the length of the attack.
    *   This increase in stake is temporary and only occurs when the chain is making slow progress to confirm blocks.

## Challenge protocol

*   When two stakers are staked on different blocks, where one block is not a successor of the other, there will be a particular block that they disagree on and a challenge will occur.
*   Most of the challenge occurs on the Arbitrum chain and is refereed by a L1 contract.
*   The challenge consists of an interactive, multi-round dissection game that occurs on L2, and a one-step proof that is executed on L1.
*   The staker who proposed the disputed block is defending their assertion against a disagreeing staker.
*   The defending staker is essentially claiming that by starting at the preceding block, after some N instructions are executed by the virtual machine, the preceding block state will have been advanced to the state given in their proposed block.
*   The defending staker (Alice) will make the first move in the dissection game by dissecting the N instruction into K parts of size N/K.
    *   Note that the subsections are not equal in size with respect to the number of steps, but rather equal in size with respect to the amount of Arbgas consumed.
    *   Also note that each segment will naturally have a start-point and an end-point (this is trivial, but makes the next dot-point easier to understand).
*   The opposing staker (Bob) will also dissect the set of instructions into K parts of size N/K, except one of Bob’s K segments will have a different end-point to Alice’s corresponding segment’s end-point.
    *   This is essentially Bob identifying the segment with which he disagrees.
*   Bob will then perform the same action as Alice’s initial step and dissect one of the segments into K subsegments of size N/K, and send this segment with identified subsegments back to Alice.
*   Alice then performs the same action as Bob’s initial step and identifies a subsegment where she disagrees with its end-point.
*   This dissection process continues until Alice and Bob have identified a single instruction on which they disagree.
*   This instruction is sent to a L1 contract which executes it and decides the winner of the dispute.
*   The loser of the dispute will have their stake confiscated, part of which will be burnt - to avoid an attacker hedging their bets - and the remainder of the stake will be given to the honest staker as a reward.
*   During the whole dissection process, the L1 contract that is refereeing the game does not know any information about the instructions themselves, it is only checking that each player is following the rules of the dissection game.
*   During the dispute, all other validators are able to determine for themselves the result of the dispute before the dispute is finalised; meaning that essentially a soft fork occurs and the validators can continue to submit rollup blocks on the correct chain.
*   The challenge period has an imposed time limit of roughly one week per staker.
*   Each staker must make all of their moves within their week allocation or they will lose the dispute.
    *   Think of a chess clock.

![Multi-round](https://storage.googleapis.com/mycelium-bucket/diagram_2_5b56f13345/diagram_2_5b56f13345.png)

> *A demonstration of the multi-round, interactive disection game played by two stakers during the challenge protocol. In reality, the disputed assertion will have many more instructions (rows with squiggles in it) and thus more rounds will be played but the principle is the same.*

## Validators

*   A validator is a node on the Arbitrum chain that has chosen to watch the activity of the Rollup protocol and advance the state of the chain.
    *   Not all nodes act as validators.
*   Offchain Labs expect validators to follow either an active, defensive, or watchtower strategy; although, this is not enforced by the protocol.
    *   An ‘active validator’ is continuously working to advance the chain by proposing new blocks. Only one honest active validator is required per chain; increasing the number of active validators does not increase the efficiency of the chain.
    *   A ‘defensive validator’ watches the Rollup protocol and acts only when they witness dishonest behaviour, at which point they will either propose a correct block - or if a correct block has already been proposed - they will stake on that block.
    *   A ‘watch tower validator’ watches the Rollup protocol just like a defensive validator, but if they witness dishonest behaviour they don’t propose or stake on a correct block themselves, rather they simply alert other validators to do so.
*   Offchain labs will run an active validator on their flagship Arbitrum chain.
*   For most of the time, defensive and watchtower validators will not need to do anything, hence an attacker is never aware of how many defensive validators there are.
*   Although anyone can be a validator, it is expected that the main parties who will choose to become one will be parties who have significant assets invested in the chain, or parties who are hired to be validators by those with significant investments.

## Full Nodes

*   A full node on Arbitrum has the same role as a full node on Ethereum; they track the state of the chain and allow others to interact with the chain
*   Due to a built-in AVM emulator, a full node is able to treat the chain as a matter of computing outputs from inputs, without any knowledge of the actual Rollup protocol.
*   A full node can serve as an aggregator on the chain, further increasing cost efficiency for the users.
*   Arbitrum includes the facility to collect fees from users to compensate the full node for the costs incurred while acting as an aggregator.
*   Full nodes can also compress transactions to further decrease the L1 calldata cost.
    *   The full node submits the compressed transaction to the chain’s inbox, where arbOS receives it and uncompresses the transaction.
*   A full node will typically incorporate both compression and aggregation, i.e. it will submit a batch of compressed transactions to the chain’s inbox.

## Sequencer Mode

*   When an Arbitrum chain is launched, there is the option to launch with a sequencer.
*   A sequencer is a full node that has additional privileges for the ordering of transactions in the chain’s inbox.
*   These privileges allow the sequencer to instantly guarantee the result of a transaction.
*   When an Arbitrum chain is launched with a sequencer, the chain inbox is effectively split in two:
    *   One inbox will operate as it normally would if there was no sequencer, i.e. nodes can send messages to the inbox which will be tagged with a block number and timestamp.
    *   The second inbox will be controlled by the sequencer, and only the sequencer can send messages to this inbox
*   When the sequencer sends messages to their inbox, they can specify the block number and timestamp with which to tag the message.
    *   This includes block numbers and timestamps from the past, up to a specified delta_blocks, blocks in the past and delta_seconds, seconds in the past.
    *   These delta values will typically correspond to roughly ten minutes of wall-clock time.
*   Now when arbOS checks the inbox, it will receive the message with the lowest block number, which may be at the head of either the regular inbox or the sequencer inbox.
*   The limit on how far back the sequencer can backdate blocks is tied to the number of confirmation blocks needed to achieve finality on Ethereum.
    *   If x amount of blocks are needed to achieve finality on Ethereum, then the sequencer will backdate by x blocks so that it knows exactly which transactions will precede its current transaction.
*   When sequencer mode is activated on an Arbitrum chain, transactions that are submitted to the sequencer will achieve finality x blocks faster than it would if there was no sequencer, but transactions that are submitted to the regular inbox will achieve finality x blocks slower than if there was no sequencer.
    *   This is regarded as a positive trade-off due to the large practical difference when comparing instant and five minute finality with five minute and ten minute finality.
*   However, a malicious sequencer is able to take advantage of these privileges to a certain extent.
    *   A malicious sequencer is able to censor user’s transactions by simply not including them to the sequencer inbox, forcing the user to send the same transactions to the regular inbox after realising that they have been censored.
    *   The sequencer would also have the power to frontrun user’s transactions.
*   The initial Arbitrum chain will launch with a sequencer that is run by Offchain Labs.
*   There has previously been some successful research into developing a decentralised fair sequencer algorithm by a team at Cornell Tech, and with some more work this will form the long term solution for Arbitrum.

![with and without a sequencer](https://storage.googleapis.com/mycelium-bucket/diagram_3_764350ae06/diagram_3_764350ae06.png)

> *The difference in the chain’s inbox when the sequencer mode is enabled / disabled.*

## ArbGas / Fees

*   ArbGas operates similarly to Ethereum gas in that it is used to measure the cost of computation on the Arbitrum chain.
*   However, there is no hard ArbGas limit imposed on an Arbitrum chain, and ArbGas can be consumed much faster than Ethereum gas.
*   A key role of ArbGas is to provide a predictable measure of how long it would take to validate the result of a computation.
*   Every rollup block includes a claim about the total amount of ArbGas consumed in the chain, meaning that the difference between the current block’s claim and the previous block’s claim should be a valid indicator of how much ArbGas is consumed in the current block.
*   This way, a validator who is checking a block’s validity can set their gas limit to this amount, and if they run out of ArbGas before they reach the end of the block, they can be certain that they have identified an invalid block and successfully challenge it.
*   Users are charged fees when they submit their transactions to the chain.
*   If the user sends their transaction to an aggregator, a portion of the fees is automatically paid to that aggregator for costs they incurred.
*   The remainder of the fees is sent to a network fee pool which is used to pay for services that ensure that the chain operates securely.
*   Fees are charged for L2 transactions, L1 calldata, computation, and storage.
*   Fees are paid in Eth.

## Summary

*   Arbitrum is a L2 scalability solution developed by Offchain Labs: an optimistic rollup that uses a multi-round interactive challenge protocol.
*   The flagship Arbitrum chain was released to developers on the 28th of May, and will be open to users once a quorum of projects has been reached.
*   From the User’s point of view, interacting with the Arbitrum chain will be an identical experience to interacting with Ethereum.

For an even deeper dive on Arbitrum, see the [docs here](https://developer.offchainlabs.com/docs/inside_arbitrum).


Stay up to date with Pier Two’s latest news and updates.

One small step for censorship resistance, one giant leap for accessibility

The CBDC Pilot Report - An Analysis

Portal Network and Client Diversity

Why Light Clients matter

Pier Two receives a Grant from the Ethereum Foundation

Discv5 Explainer Series: #1

Discv5 Explainer Series: #2

Discv5 Explainer Series: #3

Arbitrum Explained In 10 Minutes

Subscribe to our newsletter

One small step for censorship resistance, one giant leap for accessibility

The CBDC Pilot Report - An Analysis

Portal Network and Client Diversity

Why Light Clients matter

Pier Two receives a Grant from the Ethereum Foundation

Discv5 Explainer Series: #1

Discv5 Explainer Series: #2

Discv5 Explainer Series: #3

Arbitrum Explained In 10 Minutes