Skip to content

Commit

Permalink
add QueryApi context object information (#1500)
Browse files Browse the repository at this point in the history
* add context

* add images
  • Loading branch information
bucanero authored Sep 29, 2023
1 parent 1cd6457 commit c4383c2
Show file tree
Hide file tree
Showing 9 changed files with 347 additions and 1 deletion.
345 changes: 345 additions & 0 deletions docs/bos/queryapi/context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,345 @@
---
id: context-object
title: QueryAPI Context object
sidebar_label: Context object
---

## Overview

The `context` object is a global object made available to developers building indexers with [QueryAPI](intro.md). It provides helper methods for developers to interact with the resources spun up alongside their indexer, such as their GraphQL Endpoint and their database. There are also helper methods to allow specific API calls.

:::caution Under development

The formatting and changes in this document are still in progress. These changes are not fully featured yet but will be by the time they hit production. Specifically, [auto-complete](#auto-complete) and the `context.db.TableName.methodName` format.

:::

## Main Methods

:::tip
All methods are asynchronous, hence why all examples have the `await` keyword in front of the function call.
:::

### GraphQL

When an indexer is published, the SQL DDL written under the `schema.sql` tab is used to spin up a Postgres database. This DB is integrated with Hasura to provide a GraphQL endpoint. This endpoint can be used to interact with your database.
Making calls to the Hasura GraphQL endpoint requires an API call, which is restricted in the environment. So, the GraphQL method allows calls to the endpoint related to the indexer.

:::tip

The GraphQL method was previously the only way to interact with the database. Now, the [DB methods](#db) provide a more accessible functionality. The GraphQL method is better suited for more complex queries and mutations. Also, more information about GraphQL calls can be [found here](https://graphql.org/learn/queries/).

:::

#### Input

```js
await context.graphql(operation, variables)
```

The operation is a string formatted to match a GraphQL query. The variables are any data objects used in the query.

#### Example

```js
const mutationData = {
post: {
account_id: accountId,
block_height: blockHeight,
block_timestamp: blockTimestamp,
content: content,
receipt_id: receiptId,
},
};


// Call GraphQL mutation to insert a new post
await context.graphql(
`mutation createPost($post: dataplatform_near_social_feed_posts_insert_input!){
insert_dataplatform_near_social_feed_posts_one(
object: $post
) {
account_id
block_height
}
}`,
mutationData
);
```

The above is a snippet from the social feed indexer. In this one, we have a mutation (which mutates or changes data in the database, instead of a query that merely reads) called `createPost`. The mutation name can be anything. We specify a variable `post` and execute a graphQL method, which inserts the `post` object and returns `account_id` and `block_height` from the newly inserted object. Finally, we pass in `mutationData` as the variable, which is automatically linked to `post` since it's the only field.

:::tip

You can find other examples of `context.graphql` in the [social_feed indexers](../tutorial/indexer-tutorials/feed-indexer.md).

:::

### Set

Each indexer, by default, has a table called `indexer_storage`. It has a field for `key` and `value`, functioning like a key-value store. This table can, of course, be removed from the DDL before publishing. However, it kept the `set` method as an easy way to set some value for a key in that table. This method is used in the default code to set the `height` on each invocation.

#### Input

```js
await context.set(key, value)
```

#### Example

```js
const h = block.header().height;
await context.set("height", h);
```

### FetchFromSocialApi

Calls to APIs are restricted in QueryAPI because making calls usually require establishing identity. QueryAPI does not support secrets yet (e.g., identity tokens) as those would end up stored on the blockchain, which is not secure.
However, social API doesn’t require a secret, and has some potential uses. So, QueryAPI supports calls to it through this method.

#### Input

```js
await context.fetchFromSocialApi(path, options)
```

The path is a resource from the social API that the developer is targeting. Options are where the rest of the call is placed (`headers`, `method`, `body`, and so on).

#### Example

```js
const response = await context.fetchFromSocialApi("/index", {
method: "POST",
headers: {
["Content-Type"]: "application/json",
},
body: JSON.stringify({
action: "post",
key: "main",
options: {
limit: 1,
order: "desc",
},
}),
});
```

:::tip

This snippet is from the [social_lag indexer](https://near.org/dev-queryapi.dataplatform.near/widget/QueryApi.App?selectedIndexerPath=morgs.near/social_lag&view=editor-window).

:::

## DB

The DB object is a sub-object under `context`. It is accessed through `context.db`. Previously, the GraphQL method was the only way to interact with the database. However, writing GraphQL queries and mutations is pretty complicated and overkill for simple interactions. So, simpler interactions are instead made available through the db sub-object. This object is built by reading the schema written by the developer, parsing its information, and generating methods for each table. See below for what methods are generated for each table. The format to access the below methods is as follows: `context.db.[tableName].[methodName]`. Concrete examples are also given below.

:::info Note

One thing to note is that the process where the code is read is not fully featured.
If an `ALTER TABLE ALTER COLUMN` statement is used in the SQL schema, for example, it will fail to parse.
Should this failure occur, the context object will still be generated but `db` methods will be unavailable. An error will appear on the page saying `types could not be generated`. A more detailed error can be viewed in the browser's console.

:::


### DB Methods

These DB methods are generated when the schema is read. The tables in the schema are parsed, and methods are set under each table name. This makes using the object more intuitive and declarative.

If the schema is invalid for generating types, then an error will pop up both on screen and in the console. Here's an example:

![auto complete](/docs/queryapi/autocomp-error.png)

### Methods

:::note

Except for [upsert](#upsert), all of the below methods are used in [social_feed_test indexer](https://near.org/dev-queryapi.dataplatform.near/widget/QueryApi.App?selectedIndexerPath=darunrs.near/social_feed_test&view=editor-window). However, keep in mind the current code uses the outdated call structure. An upcoming change will switch to the new method of `context.db.TableName.methodName` instead of `context.db.methodName_tableName`.

:::

### Insert

This method allows inserting one or more objects into the table preceding the method call. The inserted objects are then returned back with all of their information. Later on, we may implement returning specific fields but for now, we are returning them all. This goes for all methods.

#### Input

```js
await context.db.TableName.insert(objects)
```

Objects can be a single object or an array of them.

#### Example

```js
const insertPostData = {
account_id: accountId,
block_height: blockHeight,
block_timestamp: blockTimestamp,
content: content,
receipt_id: receiptId
};
// Insert new post to Posts table
await context.db.Posts.insert(insertPostData);
```

In this example, we insert a single object. But, if you want to insert multiple objects, then you just pass in an array with multiple objects. Such as `[ insertPostDataA, insertPostDataB ]`.

### Select

This method returns rows that match the criteria included in the call. For now, we only support explicit matches. For example, providing `{ colA: valueA, colB: valueB }` means that rows where `colA` and `colB` match those **EXACT** values will be returned.

There is also a `limit` parameter which specifies the maximum amount of objects to get. There are no guarantees on ordering. If there are 10 and the limit is 5, any of them might be returned.

#### Input

```js
await context.db.TableName.select(fieldsToMatch, limit = null)
```

The `fieldsToMatch` is an object that contains `column names: value`, where the value will need to be an exact match for that column. The `limit` parameter defaults to `null`, which means no limit. If a value is provided, it overrides the `null` value and is set to whatever was passed in. All matching rows up to the limit are returned.

#### Example

```js
const posts = await context.db.Posts.select(
{
account_id: postAuthor,
block_height: postBlockHeight
},
1
);
```

In this example, any rows in the `posts` table where the `account_id` column value matches `postAuthor` **AND** `block_height` matches `postBlockheight` will be returned.

### Update

This method updates all rows that match the `whereObj` values by setting the `updateObj` values. It then returns all impacted rows. The `whereObj` is subject to the same restrictions as the select’s `whereObj`.

#### Input

```js
await context.db.TableName.update(whereObj, updateObj)
```

#### Example

```js
await context.db.Posts.update(
{id: post.id},
{last_comment_timestamp: currentTimestamp});
```

In this example, any rows in the `posts` table where the `id` column matches the value `post.id` will have their `last_comment_timestamp` column value overwritten to the value of `currentTimestamp`. All impacted rows are then returned.

### Upsert

Upsert is a combination of insert and update. First, the insert operation is performed. However, if the object already exists, the update portion is called instead. As a result, the input to the function are objects to be inserted, a `conflictColumns` object, which specifies which column values must conflict for the update operation to occur, and an `updateColumns` which specifies which columns have their values overwritten by the incoming object’s values. The `conflictColumns` and `updateColumns` parameters are both arrays. As usual, all impacted rows are returned.

#### Input

```js
await context.db.upsert(objects, conflictColumns, updateColumns)
```

The Objects parameter is either one or an array of objects. The other two parameters are arrays of strings. The strings should correspond to column names for that table.

#### Example

```js
const insertPostDataA = {
id: postId
account_id: accountIdA,
block_height: blockHeightA,
block_timestamp: blockTimeStampA
};


const insertPostDataB = {
id: postId
account_id: accountIdB,
block_height: blockHeightB,
block_timestamp: blockTimeStampB
};
// Insert new post to Posts table
await context.db.Posts.upsert(
[ insertPostDataA, insertPostDataB ],
[‘account_id’, ‘id’],
[‘block_height’, ‘block_timestamp’);
```

In this example, two objects are being inserted. However, if a row already exists in the table where the `account_id` and `id` are the same, then `block_height` and `block_timestamp` would be overwritten in those rows to the value in the object in the call which is conflicting.

### Delete

This method deletes all objects in the row that match the object passed in. Caution should be taken when using this method. It currently only support **AND** and exact match, just like in the [select method](#select). That doubles as a safety measure against accidentally deleting a bunch of data. All deleted rows are returned so you can always insert them back if you get back more rows than expected. (Or reindex your indexer if needed)

#### Input

```js
await context.db.TableName.delete(whereObj)
```

As stated, only a single object is allowed.

#### Example

```js
await context.db.delete_post_likes(
{
account_id: likeAuthorAccountId,
post_id: postId
}
);
```

In this example, any rows where `account_id` and `post_id` match the supplied value are deleted. All deleted rows are returned.

### Auto Complete

:::tip
Autocomplete works while writing the schema and before publishing to the chain. In other words, you don't need to publish the indexer to get autocomplete.
:::

As mentioned, the [DB methods](#db-methods) are generated when the schema is read.
In addition to that, TypeScript types are generated which represent the table itself. These types are set as the parameter types. This provides autocomplete and strong typing in the IDE. These restrictions are not enforced on the runner side and are instead mainly as a suggestion to help guide the developer to use the methods in a way that is deemed correct by QueryAPI.


:::info Types

You can also generate types manually. Clicking the `<>` button generates the types. It can be useful for debugging and iterative development while writing the schema.

![auto complete](/docs/queryapi/autocomp-types.png)

:::

#### Compatibility

By default, current indexers have a `context` object included as a parameter in the top-level `async function getBlock`. This prevents autocomplete, as the local `context` object shadows the global one, preventing access to it. Users need to manually remove the `context` parameter from their indexers to get the autocomplete feature. For example:

```js
async function getBlock(block: Block, context) {
```
should become
```js
async function getBlock(block: Block) {
```
#### Examples
Here are some screenshots that demonstrate autocomplete on methods, strong typing, and field names:
![auto complete](/docs/queryapi/autocomp1.jpg)
![auto complete](/docs/queryapi/autocomp2.jpg)
![auto complete](/docs/queryapi/autocomp3.jpg)
![auto complete](/docs/queryapi/autocomp4.jpg)
![auto complete](/docs/queryapi/autocomp5.jpg)
3 changes: 2 additions & 1 deletion website/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -686,8 +686,9 @@
},
"bos/queryapi/intro",
"bos/queryapi/how-it-works",
"bos/queryapi/index-functions",
"bos/community/indexers",
"bos/queryapi/index-functions",
"bos/queryapi/context-object",
{
"Tutorials": [
"bos/tutorial/indexer-tutorials/posts-indexer",
Expand Down
Binary file added website/static/docs/queryapi/autocomp-error.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp-types.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp2.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp3.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp4.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/docs/queryapi/autocomp5.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit c4383c2

Please sign in to comment.