In our previous articles, we introduced you to the Discv5 protocol, covering topics such as node identity, record management, and communication. However, in order to fully understand the protocol, it's important to delve deeper into the role of topics in improving the discovery process. Topics provide a way for nodes to group related information and make it easier for other nodes to find and subscribe to that information. In this article, we will explain topics in detail, discussing their background, how nodes manage them using registration and queue, and how they are advertised across the network. By the end of the blog, you'll have a complete understanding of the Discv5 protocol and how it works in practice.
Background
In a peer-to-peer network like the one used by Ethereum, nodes often need to find and connect with other nodes that offer specific services or share common interests. This is important for keeping the network running and enabling tasks like transaction validation, sharing blockchain data, and executing smart contracts. To make this process easier, Discv5 uses topics – simple strings that act like labels to show the services or interests provided by a node. These topics give a clear and standardised way to describe a node's abilities, making it simpler to find and interact with other nodes.
When a node makes itself visible under a specific topic, it is said to place an ad for itself. Depending on the application's needs, a node can advertise multiple topics, a single topic, or even no topics at all. Every node taking part in the Discv5 discovery protocol acts as an advertisement medium. This means they accept topic ads from other nodes and later return them to nodes looking for the same topic. By organizing various services and interests, topics improve the effectiveness of finding and connecting with other nodes in the network, ultimately supporting the overall performance and strength of the system.
Now, we will look at how these topics are stored and managed within the Discv5 protocol.
Topics and Management
Topic Table and Queue
At the core of topics in Discv5 is the topic table, a data structure that stores advertisements for a multitude of topics. Each topic within the table is associated with its own list of ads, known as the topic queue. As the name suggests, topic queues operate on a First-In-First-Out (FIFO) basis with a limited length, ensuring that the oldest ads are removed to make room for new ones when the queue reaches its full capacity.
To better understand this concept, let's visualise a topic table containing three different topic queues for topics T1, T2, and T3. The queue for topic T1 is filled to capacity and cannot store any more ads until the older ones are removed to free up space. In contrast, the queues for topics T2 and T3 may have varying levels of capacity available to accommodate new ads.
Each time a new ad is added to a topic queue, it is placed at the end of the queue. As ads are removed based on their age, the oldest ad is removed from the front of the queue, allowing the subsequent ads to move up in the queue. This FIFO mechanism ensures that each advertisement has a fair chance to be discovered and utilised by nodes searching for services or interests associated with a particular topic.
By organizing ads in this manner, the topic table not only maintains an efficient system for storing and managing topic advertisements but also provides a standardised method for nodes to search and discover services offered by their peers (connected nodes) in the network.
Ticket System
In Discv5, a ticket system is implemented to manage ad registrations efficiently. Ads are placed in a queue and remain there for a fixed 15-minute period. This system ensures that ads get equal exposure and helps maintain a fair advertisement process.
When a node wants to place an ad, it has to wait for a specific duration before the ad is accepted. This waiting time depends on the availability of slots in the ad table and the topic queue. For more context, a slot is a position in the ad table where an ad can be displayed. To keep track of this waiting time, the node receives a ticket that indicates how long it should wait before its ad gets approved. The node must hold onto this ticket and present it once the waiting time has elapsed.
The waiting time for ad registration is determined by these rules:
- If the ad table is full, the node must wait for a slot to open up based on the oldest ad's remaining lifetime. This ensures that ads are replaced in a timely manner and provides equal opportunity for new ads.
- If the topic queue is full, the node's waiting time is calculated as the target-ad-lifetime minus the lifetime of the oldest ad in the queue. This helps maintain a balance between different topics and prevents any single topic from dominating the queue.
- If there are available slots in the table and the topic queue, the ad can be placed immediately without any waiting time.
Tickets are issued by nodes and contain essential information to ensure proper ad registration. These tickets serve several purposes: they confirm that the node requesting the ticket is the one attempting to use it, ensuring a fair process for all participants. They also validate that the ticket is applicable for only one topic, preventing misuse for multiple ads or topics. Additionally, tickets can be used within the registration window, ensuring timely ad placements, and they cannot be used more than once, preventing any unfair advantage or repeated ad exposure.
To enhance security and protect ticket information, details such as waiting time and the node's ID can be encrypted using a secret key. Employing this method guarantees that only the intended node can access and use the ticket for ad registration.
Advertisement Protocol
Now, let's take another look at some messages from a previous blog post to explain how the process of topic advertisement works in Discv5. The advertisement protocol in Discv5 is a mechanism that allows nodes to discover and connect with others interested in the same topics. We'll use an example from the theoretical section of the official Discv5 specification to illustrate this process.
Imagine three nodes, A, B, and C, interacting with each other. Node A provides a service related to a topic called "Transaction relayer." To advertise its service, Node A decides to use Node C as the advertising medium. Meanwhile, Node B is on the lookout for services related to transaction relaying. Let's examine the step-by-step interaction between nodes A and C as they work together to advertise the topic:
Node A initiates registration: Node A initiates the advertisement process by sending a REGTOPIC message to Node C, requesting to register the topic "Transaction relayer." At this stage, Node A doesn't include a ticket in the message, as it hasn't received one yet.
Node A -> Node C: REGTOPIC ["Transaction relayer", ""]Node C responds with a ticket: Node C processes the request and sends a TICKET message to Node A, providing a ticket and a waiting time. The ticket is an identifier that Node A will use in subsequent interactions with Node C, while the waiting time indicates how long Node A should wait before proceeding to the next step.
Node C -> Node A: TICKET [ticket, wait-time]Node A waits and re-sends registration: Node A waits for the specified waiting time and sends a new REGTOPIC message to Node C. This time, Node A includes the received ticket in the message to verify its identity and to show that it has followed the required waiting time.
Node A -> Node C: REGTOPIC ["Transaction relayer", ticket]Node C sends a new ticket: Node C verifies the ticket and replies with another TICKET message, providing a new ticket and a new waiting time. Node A stores this new ticket to be ready for a confirmation call. This step is necessary to further ensure the fairness of the advertisement process and to prevent spamming.
Node C -> Node A: TICKET [ticket, wait-time]Node C confirms registration: After the registration window ends, Node C selects Node A as the registered node for the topic "Transaction relayer," based on factors such as ticket priority and available spots. Node C adds Node A to its topic queue and sends a REGCONFIRMATION message to Node A to confirm the successful advertisement registration.
Node C -> Node A: REGCONFIRMATION ["Transaction relayer"]
During this process, nodes A and C exchange REGTOPIC, TICKET, and REGCONFIRMATION messages, along with waiting times to manage the advertisement registration. When Node B searches for the topic "Transaction relayer," it will send a FINDNODE message to Node C, which will then provide information about Node A's registration. As a result, Node B can connect to Node A and access the desired service.
Conclusion
In this final part of our Discv5 Explainer Series, we covered how topics are used to enhance the discovery process in the Ethereum network and how nodes manage their subscriptions. Topics play a crucial role in maintaining a dynamic and healthy network, enabling nodes to efficiently find and connect with others that share their interests.
When you combine all the various components discussed in the previous blogs, such as node identities, routing tables, and secure communication mechanisms, Discv5 emerges as a powerful and robust protocol for enabling efficient, secure, and scalable peer-to-peer communication within the Ethereum network.
We hope you enjoyed this series and now have a better understanding of the Discv5 protocol. If you have any questions or comments, please share them below or reach out to us on social media. For those interested in contributing to our C# implementation of Discv5, visit the GitHub repository for more information.
Thank you for following along, and happy learning!
