Skip to content

Commit

Permalink
add demo
Browse files Browse the repository at this point in the history
  • Loading branch information
IanRFerguson committed Mar 26, 2024
1 parent dd4de91 commit 7457e19
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 19 deletions.
Binary file added .demo/stepOne.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .demo/stepThree.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .demo/stepTwo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 15 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,18 @@ Since Polars leverages Rust speedups, you need to have Rust installed in your en

## Usage

```
# Instantiate a connection to BigQuery
from klondike import BigQueryConnector
bq = BigQueryConnector(
app_creds="/path/to/your/service_account.json"
)
# Read data from BigQuery
sql = "SELECT * FROM nba_dbt.staging__nyk_players"
df = bq.read_dataframe_from_bigquery(sql=sql)
# Write data to BigQuery
bq.write_dataframe_to_bigquery(
df=df,
table_name="nba_dbt.my_new_table",
if_eixsts="truncate"
)
```
In this demo we'll connect to BigQuery, read data, transform it, and write it back to the data warehouse.

First, connect to the BigQuery warehouse by supplying the `BigQueryConnector()` object with the relative path to your service account credentials.

Next, supply the object with a SQL query in the `read_dataframe_from_bigquery()` function to redner a `DataFrame` object:

<img src="./.demo/stepOne.png">

Now that your data is pulled into a local instance, you can clean and transform it using standard Polars functionality - [see the docs](https://docs.pola.rs/py-polars/html/reference/dataframe/index.html) for more information.

<img src="./.demo/stepTwo.png">

Finally, push your transformed data back to the BigQuery warehouse using the `write_dataframe_to_bigquery()` function:

<img src="./.demo/stepThree.png">

0 comments on commit 7457e19

Please sign in to comment.