-
Notifications
You must be signed in to change notification settings - Fork 370
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add page * update * how it works * Update docs/bos/queryapi/how-works.md Co-authored-by: Morgan McCauley <morgan@mccauley.co.nz> --------- Co-authored-by: Morgan McCauley <morgan@mccauley.co.nz>
- Loading branch information
1 parent
9082533
commit fc2568f
Showing
4 changed files
with
144 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
--- | ||
id: bos-components | ||
title: Components handling Historical data | ||
sidebar_label: Components & Historical data | ||
--- | ||
|
||
Building components that handle historical blockchain data require dedicated solutions that manage the data and reduce the latency of requests, as it's not possible to scan the whole blockchain when a user makes a request. | ||
|
||
A simple solution for developers building on NEAR BOS is using [QueryAPI](intro.md), a fully managed solution to build indexer functions, extract on-chain data, store it in a database, and be able to query it using GraphQL endpoints. | ||
|
||
## QueryAPI | ||
|
||
:::tip | ||
Learn more about QueryAPI in this [QueryAPI Overview](intro.md) article. | ||
::: | ||
|
||
#### Tutorials | ||
|
||
For a technical implementation deep-dive, check these QueryAPI tutorials: | ||
|
||
- [Posts Indexer tutorial](../tutorial/indexer-tutorials/posts-indexer.md): this indexer creates a new row in a pre-defined database for every new BOS post found on the blockchain. | ||
- [Hype Indexer tutorial](../tutorial/indexer-tutorials/hype-indexer.md): this indexer creates a new row in a pre-defined database for every new BOS post or comment found on the blockchain that contains either `PEPE` or `DOGE` in the contents. | ||
- [BOS Feed Indexer tutorial](../tutorial/indexer-tutorials/feed-indexer.md): this indexer keeps track of new posts, comments, and likes on BOS, so a social feed can be rendered quickly. | ||
|
||
## GraphQL queries | ||
|
||
Using [QueryAPI's GraphiQL](index-function.md#mutations-in-graphql) tab, you can access the GraphiQL Explorer that provides a user friendly GraphQL playground, where you can view and create queries and mutations based on the DB schema that you defined for the indexer. | ||
|
||
![QueryAPI Indexer Dashboard](/docs/assets/QAPIgraphiql.png) | ||
|
||
You can easily set some fields and select the returning data | ||
that you want, and the tool will build a query on the mutation panel on the right. | ||
Then you can copy the resulting query, either in your JavaScript code so that you pass actual | ||
data manually, or you pass in the mutation data object as a second parameter. | ||
|
||
For example, if you go and add a new mutation, click <kbd>+</kbd>, then you can do a bunch of actions here, such as creating, deleting, or inserting posts into your table. | ||
|
||
![Playground](/docs/assets/QAPIScreen.gif) | ||
|
||
If you want to test your mutation, using [Debug Mode](index-function.md#local-debug-mode) you can add a specific | ||
block to the list, and then play it to see how it works. | ||
Based on the indexer logic you defined, you'll get a call to the GraphQL mutation with the object | ||
and data passed into it. | ||
|
||
:::tip Video Walkthrough | ||
|
||
**Tip:** watch the video on how to [create mutations in GraphQL](https://www.youtube.com/watch?v=VwO6spk8D58&t=781s). | ||
|
||
::: | ||
|
||
## Generate a BOS component using Playground | ||
|
||
Creating a BOS component from a GraphQL query is simple when using QueryAPI's GraphQL Playground. Just follow these steps: | ||
|
||
- go to the GraphiQL tab | ||
- select the query that you want to use | ||
- click on the <kbd>Show GraphiQL Code Exporter</kbd> button | ||
- get some default code here, copy it, | ||
- go to the BOS sandbox, paste it. | ||
|
||
|
||
This will set up some boilerplate code to execute the GraphQL query, add the query that you had | ||
in your playground and then call that query, extract the data and render it using the | ||
render data function. | ||
|
||
Once you have the BOS component code, you can test it out by going to [the sandbox](https://near.org/sandbox), | ||
pasting the generated code, and then selecting <kbd>Component Preview</kbd>. | ||
Next, you can create a nice UI over this boilerplate code, and publish your new BOS component. | ||
|
||
### Component Examples | ||
|
||
- [Activity Feed widget](https://near.org/near/widget/ComponentDetailsPage?src=roshaan.near/widget/user-activity-feed&tab=source) running on [near.org](https://near.org) | ||
|
||
<!-- | ||
- Example of BOS component using BigQuery | ||
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
id: how-it-works | ||
title: How QueryAPI works | ||
sidebar_label: How it works | ||
--- | ||
|
||
QueryApi is a streaming indexer implementation that executes custom indexing logic written by developers on the NEAR blockchain. | ||
QueryApi allows hosted execution of complex queries (ones that can’t be answered by a [simple RPC](../../5.api/rpc/introduction.md) or [Enhanced API](https://docs.pagoda.co/api) call), data hosting, and its exposure via GraphQL endpoints. | ||
|
||
|
||
## Components involved | ||
|
||
The QueryApi implementation integrates many different components in a single and streamlined solution. | ||
In a high-level overview, the main components are: | ||
|
||
:::info Components | ||
`NEAR Protocol` -> `NEAR Lake` -> `Coordinator` -> `Runner` -> `Database` -> `API` | ||
::: | ||
|
||
|
||
### Detailed overview | ||
|
||
An in-depth, detailed overview of the QueryApi components: | ||
|
||
[![QueryAPI](/docs/qapi-components.png)](/docs/qapi-components.png) | ||
|
||
|
||
### Description | ||
|
||
- **Protocol:** the underlying NEAR Layer-1 Blockchain, where data `Blocks` and `Chunks` are produced. | ||
- **NEAR Lake:** an indexer which watches the Layer-1 network and stores all the events as JSON files on AWS S3. Changes are indexed as new `Blocks` arrive. | ||
- **Coordinator:** the QueryApi coordinator indexer filters matching data `Blocks`, runs historical processing threads, and queues developer's JS code to be indexed with these matched blocks. | ||
- **Runner:** executes the user's indexer code, which outputs the data to the database. | ||
- **Database:** a Postgres database where the developer's indexer data is stored, using a logical DB per user, and a logical schema per indexer function. | ||
- **API:** a Hasura server running on Google Cloud Platform exposes a GraphQL endpoint so users can access their data from anywhere. | ||
|
||
## Who hosts QueryAPI | ||
|
||
[Pagoda Inc.](https://pagoda.co) runs and manages all the infrastructure of QueryAPI, including NEAR Lake nodes to store the data in JSON format on AWS S3. | ||
|
||
- NEAR Lake indexing is hosted on AWS S3 buckets. | ||
- Coordinator is hosted on GCP. | ||
- Runners are hosted on AWS SQS and AWS Lambda. | ||
- Hasura GraphQL API server is hosted on GCP. | ||
|
||
:::caution Pricing | ||
QueryAPI is currently free. Pagoda doesn't charge for storage of your indexer code and data as well as running the indexer, but usage pricing will be introduced once QueryApi is out of beta. | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.