Explanation series - A unified stack for scalable and reliable P2P communication

This post is part of a series focusing on Waku's features. The goal is to help developers understand how they can leverage Waku for their own applications and projects.

Libp2p is the de facto standard for building peer-to-peer networks. It is a network stack that enables developers to build peer-to-peer applications in many languages, platforms, and contexts. Yet libp2p is not sufficient when building chat and other applications in a decentralised manner.

We have already learned how Waku enables privacy-preserving DoS protection on gossipsub networks and reliable communication on end-user devices.

Let’s explore how Waku delivers a comprehensive peer-to-peer technology stack, bringing turnkey reliability and scalability to decentralised applications.

Content addressing

A unique property of Waku is how content addressing is handled. Due to the low latency, broadcast and small payload properties required, Waku approaches content addressing in a unique manner. Contrary to IPFS or blockchain systems, which solely use hashes to identify and retrieve data in the network, Waku uses a metadata string, or content topic, that senders set on their messages.

This content topic can then be used to select what messages to process locally for relay nodes or to filter a subset of messages to forward to edge nodes when using filter protocol.

Content topics are defined on the Waku Message wire format, alongside other properties such as timestamp and RLN proofs.

Reliability and scalability

Reliability and scalability are two sides of the same coin. Scalability refers to a network’s ability to remain reliable while it grows in several directions, such as the number of nodes, message traffic, etc.

We can loosely define reliability with two intertwined properties:

• Messages are transmitted between users in a timely fashion
• Information on whether a message was received is available

As presented in our Message Reliability article, reliability can be split into three layers:

• Transport
• Peer-to-peer
• End-to-end

TCP, WebSocket, libp2p, and gossipsub provide transport reliability, thanks to sensible default configurations applied by Waku libraries. Peer-to-peer and end-to-end reliability are achieved by Waku protocols.

Tunable peer-to-peer reliability

Waku improves peer-to-peer reliability over vanilla libp2p-gossipsub. First, by caching messages using Waku store, it allows nodes to query whether a message was seen on the network. Waku sync enables efficient synchronisation of recent messages between nodes. Other mechanisms are also used to improve reliability, such as regular pings to assess whether a node is online or applying redundancy when using light protocols.

This is necessary because libp2p has been hardened in contexts (IPFS, Ethereum) where the libp2p node is mostly online. Waku takes it a step further by making end-user devices peer-to-peer.

End-to-end reliability

Again, libp2p was hardened in contexts where the application has some intrinsic end-to-end reliability: in a blockchain, nodes will know a block is missed as each block refers to the previous one. Similarly, they will learn from missed transactions as all transactions are referred to the next block. In IPFS, the DHT helps understand the expected content across the network, mitigating any potential reliability issues.

Those features are unavailable when building a generalised communication protocol with a clear separation of routing and application layers. This separation preserves privacy and censorship resistance, as message routing does not have access to application data or logic. To address this, Waku provides end-to-end reliability protocols: Minimal Viable Data Sync (MVDS) and Scalable Data Sync (SDS), which applications can integrate to ensure messages are not lost.

Refer back to our Message Reliability article to learn more about reliability in Waku.

Scalability

Reliability protocols allow Waku to apply scalability strategies with confidence.

Waku’s goal is to provide a censorship-resistant communication network. Using decentralisation as a tactic to achieve this requires maximising participation in the Waku Relay network. This is why it is important for end-users to be able to run a relay node at home, on their laptop, or NAS. In this context, bandwidth is the scarcest resource.

To remediate this, we split the relay network into different pubsub topics, or gossipsub sub-networks. We refer to these as shards. This allows the traffic of all Waku users to be segregated into more manageable traffic lanes.

Light protocols can then be leveraged so that a node can relay on a shard (high bandwidth cost) and use light protocols (low bandwidth cost) on other shards, maximising decentralisation while minimising bandwidth usage. When coupled with RLN Relay, this ensures the global bandwidth threshold of the network does not impede its growth and adoption.

The challenges then come from peer discovery: how can a node find peers for specific shards?

Peer discovery and management

Waku combines several peer discovery protocols to enable fast and secure connectivity.

On all platforms, Waku peer exchange is used to connect more peers with a simple request to a neighbour. With the addition of rendezvous, discovering nodes in the appropriate shards is enhanced. However, those methods are susceptible to Sybil attacks, where an attacker could drive a peer to only connect to malicious nodes. This is prevented by the usage of discv5, a decentralised peer discovery mechanism, which, as a downside, is slower. Waku also enhanced discv5 to filter nodes according to their shards and capabilities, allowing for efficient connection to the right set of peers.

Waku’s peer discovery system is an example of how it bundles several protocols to provide the best of both worlds to the end user. Another notable functionality is enhanced peer management. Waku libraries manage peers depending on the platform to ensure that the right type of peers, whether by shards or by service (light protocols), are available. Peers that do not perform well on a Waku protocol (e.g. a message sent by light push is not seen by store peers) are then recycled in favour of new peers.

These protocols, working together, unlock Waku's full utility as an all-in-one reliable and scalable message routing and peer discovery technology stack.

Turnkey message routing and peer discovery

Waku is a modular protocol suite designed to provide a seamless peer-to-peer communication experience across various environments and platforms. This modularity is not meant to create complicated choices for developers but rather to deliver a robust protocol stack that works out of the box, ensuring scalability and reliability.

One could argue that using libp2p directly in an application would enable a more bespoke networking solution. However, the separation of application and routing brought by using Waku is a feature—in other words, Waku can be used to plug into a general routing network, which brings shared resources, anonymity for app users, privacy preservation, and censorship resistance.

Thanks to collaborations with Status and partners like RAILGUN, Waku's protocol suite offers a diverse set of proven strategies integrated into its SDKs. This allows developers to leverage a single, versatile SDK that adapts to different application contexts without requiring the understanding and import of multiple protocols and libraries.

Waku's library handles common issues like frequent disconnections, unreliable networks, and peer discovery latency on end-user devices. It also supports building large-scale networks with tens of thousands of nodes.

As a work in progress, the Waku messaging API will serve as the first layer of this vision. It will enable developers to create peer-to-peer applications by simply understanding the concept of content topics and RLN memberships, streamlining development and fostering easy adoption.