Learn more about how The Graph uses Waku to build a decentralised data querying layer for blockchains
Indexing blockchain data can be challenging for projects with complex smart contracts. Managing and querying the vast data generated has become a significant challenge as the blockchain ecosystem proliferates.
The inherent properties of blockchain pose unique challenges when it comes to retrieving and querying data effectively. Some of the challenges include:
Blockchain finality: refers to the irreversible acceptance of a transaction or block into the canonical chain, which can complicate the querying process.
Chain reorganisations refer to situations where the canonical chain is revised due to competing blocks. This block revision, in turn, introduces uncertainties that must be accounted for.
Uncled blocks refer to valid blocks not included in the canonical chain. These blocks complicate retrieving accurate and consistent query results from blockchain data.
These peculiarities of blockchain architecture make querying data time-consuming and conceptually complex, requiring specialised solutions to navigate seamlessly.
What is our solution? The Graph.
Gateway to querying web3 data - The Graph
The Graph aims to be the querying layer for blockchains. It intends to create a decentralised protocol for indexing and querying blockchain data. The Graph is similar to a middleware layer that indexes data from blockchains and storage networks, making it easily accessible and queryable without much hassle.
There are various segments within The Graph Protocol:
1. Data Indexing Segment:
- Subgraphs: The open-source GraphQL schemas that define the data requirements and mapping rules.
- Indexers: The nodes responsible for processing blockchain data based on the subgraph specifications and storing the indexed data in a queryable format.
- Curators: The participants who signal the importance of specific subgraphs by staking tokens and incentivising indexers to prioritise indexing those subgraphs.
2. Data Querying Segment:
- Query Node: The stateless component that serves as the entry point for querying the indexed data routing GraphQL queries to the appropriate indexers.
- Graph Explorer: The graphical user interface allowing developers and users to explore and interact with the indexed data, execute GraphQL queries, and visualise the subgraph schemas.
3. Network and Governance Segment:
- The Graph Network: The decentralised network of indexers, curators, and other participants contributing to indexing and querying blockchain data, operating on the Waku messaging protocol.
Proof of Indexing
Proof of Indexing is a mechanism The Graph uses to ensure the integrity and reliability of the indexed data within its ecosystem. It is closely related to the core functionality of The Graph Protocol.
Proof of Indexing works by having indexers periodically submit a proof that they have correctly indexed the data specified by the assigned subgraphs. These proofs are cryptographic hashes that summarise the indexed data, allowing other nodes in the network to verify the correctness of the indexing process without having to re-index the entire data themselves.
The role of Proof of Indexing is twofold:
Data Integrity: By requiring indexers to submit proofs, The Graph ensures that the indexed data is accurate and consistent across the decentralised network of indexers. This helps maintain the reliability and trustworthiness of the indexed data, which is crucial for applications built on top of The Graph.
Indexer Accountability: Proof of Indexing holds indexers accountable for their work. Indexers who fail to submit valid proofs or are caught submitting incorrect proofs can face penalties, such as slashing their staked tokens or removal from the indexing pool. This incentivises indexers to perform their indexing duties honestly and accurately.
Core developer teams in The Graph ecosystem
Multiple teams work on various focus areas within The Graph. GraphOps is one of the teams focusing on data indexing. GraphOps leverages Waku for a unique use case in 'subgraph radios', which we will explain in detail.
Subgraph radios are a feature introduced by Graphcast that allows real-time streaming of data updates and events from The Graph's indexed subgraphs. They bridge The Graph's indexed data and Graphcast's event processing framework, making subgraph radios a key component of Graphcast's real-time data streaming and event processing capabilities.
What is Graphcast?
The Graph is a decentralised indexing layer that does not rely on centralised communications. Graphcast was introduced to provide uncompromised communications in The Graph's indexing solution.
Before the GraphOps team introduced Graphcast, the majority of the indexing solutions used three ways to communicate:
1. Onchain communications: Expensive and inefficient regarding scalability as the users need to pay gas for every state change.
2. Old-fashioned group chat: It is highly dependent on Web2 communications, which have a single point of failure and higher security risks.
3. Automated bots: Several indexer bots running on centralised servers to automate communication on event signalling.
All three solutions listed above could have been more efficient. Hence, we wanted a more robust and decentralised communications infrastructure to fix several issues. The core contributors of GraphOps mention that Waku is an obvious choice for solving this problem. Continue reading to learn more about how Graphcast uses Waku.
How does The Graph use Waku?
Waku is one of the foundational technologies used to build Graphcast, a domain-specific gossip network for indexers.
Waku as an infrastructure for decentralised indexing
To maintain the decentralised nature of its infrastructure, The Graph leverages Waku, a family of decentralised Web3 communications protocols, for communication between its indexers. Indexers are responsible for processing blockchain data and creating queryable subgraphs.
Waku enables indexers to communicate and sync data peer-to-peer, eliminating the need for centralised servers or intermediaries. This approach aligns with the decentralisation principles and censorship resistance underpinning Web3.
By utilising Waku, The Graph ensures that its indexing infrastructure remains decentralised, resilient, and resistant to single points of failure. This not only enhances the security and reliability of The Graph's services but also upholds the core values of Web3, fostering a truly decentralised ecosystem for data indexing and querying.
The Waku implementation used for Graphcast
Waku provides several implementations that apply to various use cases based on resource availability. Graphcast uses Waku's Rust bindings. In Graphcast's Waku integration, all downstream radios act as Waku relay nodes. Graphcast uses its unique pubsub topic. The GraphOps team also operates their own nwaku fleet in addition to the already-existing Waku nodes that support Graphcast's pubsub topic.
Subgraph versioning with Waku
GraphOps core contributors also mentioned they face difficulties with subgraph versioning. These difficulties revolve around syncing the subgraph data based on the correct versions.
The core contributors are also interested in exploring js-waku- a light node implementation of Waku - to create a web interface that allows sending messages to the Graphcast network to sync the subgraph data based on the correct versions.
This is another fantastic use case that Waku provides to Graphcast, providing private, robust, and reliable communication.
Closing words
The Graph is one of the most widely used projects in the Web3 space. Waku's unique use case of decentralising their indexers with a peer-to-peer network is one of a kind, and only a limited number of Web3 infrastructure projects implement such a high degree of decentralisation in their indexers. TheGraph has introduced a novel use case to the space, and the Waku team is happy to help other projects decentralise their communications infrastructure.
Join our Discord community, and our core contributors will be happy to assist you in getting started with Waku.