How to query Web3 data using The Graph Protocol and Subgraphs

Whether you are building a dApp or you belong to the businesses and enterprises that consume blockchain data— you must be looking for an effortless way to access the blockchain data. The Graph, as one of the most preferred blockchain data indexing protocols, allows applications in the web3 space to easily query specific data from the blockchain’s decentralized database and utilize it for different purposes. If you want to learn about on-chain data querying on The Graph protocol, this article will show you how to query Web3 data using the Graph protocol, its custom APIs, and Subgraphs.

Understanding The Graph Protocol and its importance in the Web3 space

The Graph is a decentralized data indexing and query protocol that organizes complex blockchain data into simple, extractable format. The protocol allows data consumers like dApp developers and web3 businesses to efficiently and securely query specific data from a range of blockchains. Anyone can use Subgraph to index and query Web3 data. Let’s look at the example below to further understand The Graph protocol’s importance in Web3 space.

The prominent NFT project; Bored Ape Yacht Club, stores all of its data on the Ethereum blockchain, creating challenges for anyone to read the data directly. Like, you can query basic information about the owner of a certain NFT or Ape, fetch the content URI of Apes based on their unique ID, or check the total NFT supply. These operations are easy because they are supported directly on the smart contracts through programming. But, if we talk about more advanced and real-world queries, such as querying apes owned by some specific address and filtering Apes based on their characteristics– these operations are not directly enabled through smart contracts. To retrieve these types of data related to aggregation, search, and non-trivial filtering, you have to individually analyze each transaction and then use Token ID and IPFS hash to read the metadata, and finally, you can aggregate the data. This approach usually takes hours and sometimes even days for a dApp to retrieve data on their interface.

Additionally, businesses can set up their own server that processes bulk transactions, save information to the database, and build API endpoints for web3 data querying. However, this alternative is resource intensive, demands high maintenance, and is exposed to issues such as single point of failure and security vulnerabilities.

The Graph solves all of these challenges with its decentralized data indexing protocol that is designed to index bulks of blockchain data and allow for efficient querying of web3 data using Subgraphs. Since its inception, The Graph network has focused on removing all these tradeoffs that blockchain dApp developers and data consumers face till now. The Graph provides them with a core infrastructure that decentralizes the API and query layer in the decentralized web known Web3. With the Graph, any business can create Subgraphs to query data from various networks supported on the Graph protocol using the protocol’s query language– GraphQL.

That was all about Graph Network’s introduction. Now, to understand how to query web3 data using Subgraphs, you need to learn about multiple roles that uphold the Graph network. These roles allow the Graph ecosystem to facilitate efficient web3 data querying with Subgraphs and other services. Let’s learn more about these roles:

Indexers: Indexers are the backbone of The Graph’s proof-of-stake network. They operate nodes on the network through which data is indexed and served to the queries. Indexers choose to index Subgraphs based on their curation signal. To become indexers, they have to stake their GRT tokens, aggregate delegation from delegators, and they also should have end-to-end familiarity with how various roles work on The Graph Network and its indexing tech stack.

Curators:  Curators are the community members, data consumers, or even Subgraph developers who use their intense Web3 knowledge to examine and signal the Subgraphs that Graph Network should index. To do this, Curators stake GRT in the Subgraph’s bonding curve and get their Graph Curation Shares (GCS). Curators can access various Subgraphs through the Graph Explorer, and from there, they make signaling decisions. Curators who are able to signal on high-quality Subgraphs are entitled to receive a certain share from query fees generated by that specific Subgraph.

Delegators:  Delegators are the network participants on the Graph that delegate (or stake) their GRT for Subgraphs of their choice. Subgraphs with more delegators easily get attraction from the indexers, and hence Indexers prefer to index those subgraphs first. Delegators do not directly participate in the network. Hence, they do not need to run a Graph Node, still, they contribute to the network’s security and earn a certain portion of the rewards and query fees that a Subgraph receives. The total number of queries a Subgraph can process is highly dependent on its Indexer’s own stake plus delegated stake, which means a higher stake allows Subgraphs to process more queries.

Also Explore: Blockchain Data Indexing: Navigating its working and challenges

How to query web3 data on The Graph Protocol

You can query on-chain data in various ways, including the do-it-yourself options available at The Graph Network or leveraging the managed services offered by a trusted Subgraph service provider. Let’s understand more about these options:

Do-it-yourself Method:

1. Subgraph development with Subgraph Studio:

The Graph Network offers ‘Subgraph Studio’, a platform allowing you to create and publish your Subgraphs on the decentralized Graph network in a permissionless way. Building your own Subgraph means you can customize the queries as required and allow other dApp consumers to query on-chain data from it. Once made live, your Subgraph will start extracting data from the blockchain network, process the raw data, and store the indexed data in a way that makes data querying extremely easy via GraphQL language.

To launch your Subgraph in Studio, you need to have all the prerequisites in place for Subgraph development and deployment. This includes scalable nodes, a web3 wallet, and certain prerequisites like Graph CLI and docker installed in your machine. Once you deploy the Subgraph, you can use it to pull data from the blockchain but to decentralize its infrastructure and make the Subgraph available to various dApps, the Subgraph, indexers, curators, and delegators will authenticate your Subgraph and decide if it should be published on the Graph or not.

With Subgraph Studio, you can publish Subgraph on limited blockchain networks, including  Ethereum, Further, to deploy and maintain your Subgraphs in Studio, you must understand the tokenomics of the Graph protocol, which indexers, delegators, and curators uphold. Also, Subgraphs need to have their custom payment contracts through which micropayment happens on the data consumers’ side and is provided to the Subgraph provider. Creating and managing these GRT payment contracts is another challenge that Subgraph developers can face.

2. The Hosted Service Subgraphs:

Hosted Service is a Graph-owned service that allows for hosting of Subgraphs that are not supported on Graph decentralized network, but they are available on-demand. dApp developers can use hosted Service to deploy Subgraphs that can index a range of supported blockchain networks, index data from them, and make available for different queries via graphQL. Since the hosted Service is known as free Graph Node Indexer, developers do not need a dedicated Graph node or wallet to deploy their Subgraphs. Instead, they can use a GitHub account to create a hosted service account to get started.

As we know, Indexing a Subgraph on The Graph’s decentralized network is a complex process. For example, you need indexers to index your Subgraph, you need enough approval from the curators, and then you need delegators to delegate GRT for your Subgraph. Plus, you need to be well-versed with all the technical and programming aspects of Subgraph creation and deployment. Hence, projects that are new and they cannot attract indexers , curators, delegators for their Subgraph, Hosted Service will work. That’s because the Graph Network hosts these Subgraphs on its centralized infrastructure, handling indexing, delegation, etc., on their behalf.

Hosted service Subgraphs works like centralized servers, which may lead to issues like the single point of failure and privacy concerns. And, if you choose to migrate your Subgraph, there’s a set of challenges, such as you need to be familiar with Graph Studio, dedicated tech stack, and the whole concept of indexing, delegation, and curation that also includes GRT staking and running production-level nodes. Also, Hosted Service does not support Subgraph deployment for AppChains and dedicated indexers. Which means if you want to use Subgraph to query data from any standalone chain, then migrating from Hosted Service to decentralized Graph network is the only way to go.

Despite all these, Hosted Service is opposed to The Graph’s motive of decentralizing the data indexing process. Therefore, the Hosted Service on the Graph protocol is going to be deprecated soon, as announced by The Graph’s official documentation. Knowing this, a lot of dApps have already started to migrate their Subgraph from hosted Service to Graph’s decentralized network.

3. Web3 data querying with already published Subgraphs 

The above two options are for projects that look for customized data queries and retrieval of very specific on-chain data from the blockchain network of their choice. Further, DApps that rely on general-purpose data querying can use the existing Subgraphs published on the Graph Explorer to fulfill their data requirements. For example, if you want to query data using Liverpool Subgraph, go to that project, click on the “Query” button, and fetch the unique query URL from there. The URL will look like below:

https://gateway.thegraph.com/api/[api-key]/subgraphs/id/FDD65maya4xVfPnCjSgDRBz6UBWKAcmGtgY6BmUueJCg

With this URL, you can query data from Liverpool Subgraph instantly. Similarly, you can do with other Subgraphs. As you can see, the URL requires an API. You can generate this API key in the Subgraph Studio. This will cost you certain fees you need to pay in GRT. Get more information about fees and billing here.

Managed Services:

Managed service is another great way for businesses to leverage The Graph’s technology with or without GRT token delegation/staking and managing Subgraph infrastructure on their end. This refers to a solution provider that simplifies Subgraph creation and deployment with advanced tooling and stack, allowing developers to focus on building great applications with access to real-time data. Managed services offer everything required for Subgraphs– from reliable deployment infrastructure to plug-play dev tools and high-availability servers. This way, any business— whether technical or non-technical can rely on managed services to create their own custom Subgraphs.

How Zeeve can help?

Zeeve makes the life of dApp developers easier by tackling all the challenges they come across while querying web3 data using Subgraphs. Zeeve offers the following Subgraph services to enable the development, deployment, and maintenance of Subgraphs.

Support for shared and dedicated Indexers: The shared indexing services at Zeeve are designed for dApps that require a defined number of Subgraphs deployment and data query requests. For them, Zeeve hosts a full-fledged Graph node on its own and makes this infrastructure available to many Subgraphs. This means that various Subgraphs can be deployed and managed on a single infrastructure. While this approach is cost-efficient and less complex for developers, shared indexing infrastructure may not be suitable for projects that need to deploy multiple Subgraphs, index a huge volume of blockchain data, and thereby, serve a bulk of specific queries on a regular basis.

This type of project can use Zeeve’s Dedicated Indexers services so that there should not be any limitation on the number of Subgraphs they can launch or the number of queries their Subgraphs handle. Basically, Zeeve will handle everything, from setting up your dedicated Graph node and the server, getting the infrastructure ready, and then deploying your Subgraph seamlessly. Knowing that maintaining a dedicated Graph node is a bit of a hassle, where you need to constantly upgrade the infrastructure, track the performance, and ensure high availability— Zeeve takes responsibility for end-to-end management of your nodes on our enterprise-grade infrastructure for performance, scalability, and resilience.

Custom Subgraph development: With custom Subgraph development, Zeeve helps to create custom-fit Subgraphs for their dApps while handling everything related to the Subgraph’s development and deployment . Zeeve eliminates the hassle of hosting nodes, delegating GRT tokens, or programming smart contracts by providing a low-code infrastructure that enables Subgraph creation and hosting in a few clicks. Zeeve’s Subgraph is supported for a range of popular blockchain networks, including those yet to be available on the Graph Network itself.

Monitoring, maintenance, and more: Monitoring your Subgraph is important to maintain its high-end performance and resilience while eliminating downtime issues. For this, Zeeve offers a graphical dashboard that monitors your Subgraphs on vital parameters such as block height, space/storage consumed, and standard uptime.

Also, if you want to migrate your Subgraph from a hosted service to Zeeve managed services, we provide complete support for this, making migration a quick and hassle-free approach. If you are looking to build or deploy a custom Subgraph or you want to migrate your existing Subgraph to Zeeve’s high-availability infrastructure, we are ready to help! For more information on Zeeve Subgraphs services or to discuss your project-specific requirements, talk to our experts in blockchain data indexing.