Skip to content

Commit

Permalink
Create a new page
Browse files Browse the repository at this point in the history
Signed-off-by: Mecoli1219 <michaellai901026@gmail.com>
  • Loading branch information
Mecoli1219 committed Nov 22, 2024
1 parent 6bd04c9 commit 8f86c44
Show file tree
Hide file tree
Showing 6 changed files with 99 additions and 44 deletions.
3 changes: 1 addition & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -624,11 +624,10 @@
import_projects = [
{
"name": "flytesnacks",
"source": flytesnacks_local_path or "https://github.com/Mecoli1219/flytesnacks",
"source": flytesnacks_local_path or "https://github.com/flyteorg/flytesnacks",
"docs_path": "docs",
"dest": "flytesnacks",
"cmd": [
["git", "-C", f"{flytesnacks_path}", "checkout", "origin/jupyter-basic"],
["cp", "-R", f"{flytesnacks_path}/examples", "./examples"],
[
# remove un-needed docs files in flytesnacks
Expand Down
8 changes: 0 additions & 8 deletions docs/user_guide/basics/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Basics

.. include:: /examples

This section introduces you to the basic building blocks of Flyte
using `flytekit`. `flytekit` is a Python SDK for developing Flyte workflows and
tasks, and can be used generally, whenever stateful computation is desirable.
Expand All @@ -22,12 +20,6 @@ workflows
launch_plans
imperative_workflows
documenting_workflows
/examples/basics/basics/basic_interactive_mode.ipynb
shell_tasks
/examples/basics/basics/basic_interactive_mode
https://raw.githubusercontent.com/Mecoli1219/flytesnacks/85966d139a9a3ccdf3323124563960fdd5f5844a/examples/basics/basics/basic_interactive_mode.ipynb
examples/basics/basics/basic_interactive_mode.ipynb
test
examples/basics/basics/basic_interactive_mode
named_outputs
```
33 changes: 0 additions & 33 deletions docs/user_guide/basics/test.ipynb

This file was deleted.

3 changes: 3 additions & 0 deletions docs/user_guide/flyte_fundamentals/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ use cases.
- Develop and deploy workflows to a local Flyte demo cluster.
* - {doc}`⏱ Running and scheduling workflows <running_and_scheduling_workflows>`
- Execute workflows programmatically and schedule them as cron jobs.
* - {doc}`📙 Jupyter notebook interaction <jupyter_notebook_interaction>`
- Develop and debug Flyte workflows interactively in Jupyter notebooks.
* - {doc}`📊 Visualizing task input and output <visualizing_task_input_and_output>`
- Create rich, customizable static reports for increased visibility into tasks.
* - {doc}`🏎 Optimizing tasks <optimizing_tasks>`
Expand All @@ -45,6 +47,7 @@ cluster, see the {ref}`Deployment Guide <deployment>`.
tasks_workflows_and_launch_plans
registering_workflows
running_and_scheduling_workflows
jupyter_notebook_interaction
visualizing_task_input_and_output
optimizing_tasks
extending_flyte
Expand Down
91 changes: 91 additions & 0 deletions docs/user_guide/flyte_fundamentals/jupyter_notebook_interaction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
kernelspec:
display_name: Python 3
language: python
name: python3
---

(getting_started_jupyter_notebook_interaction)=

# Running and developing workflows in Jupyter notebooks

Flyte supports the development, running, and debugging of tasks and workflows in an interactive
Jupyter notebook environment, which accelerates the iteration speed when building data-
or machine learning-driven applications.

```{admonition} Attention
:class: attention
This feature requires the `flytekit` version `1.14.0` or higher.
```

```{admonition} Prerequisites
:class: important
This guide assumes that you've completed the previous guides for
{ref}`Running and Scheduling Workflows <getting_started_run_and_schedule>`.
The code snippets in this guide are intended to be run in a Jupyter notebook.
```

The code of this guide can be found in the [flytesnacks](https://github.com/flyteorg/flytesnacks/blob/master/examples/basics/basics/basic_interactive_mode.ipynb)

## Create an interactive `FlyteRemote` object

In {ref}`Running and Scheduling Workflows <getting_started_run_and_schedule>`, you learned
how to run registered Flyte workflows from a Python runtime using the
{py:class}`~flytekit.remote.remote.FlyteRemote` client.

When developing workflows in a Jupyter notebook, `FlyteRemote` provides an
interactive interface to register and run workflows on a Flyte cluster. Let's
create an interactive `FlyteRemote` object:

```{code-cell} ipython3
:tags: [remove-output]
from flytekit.configuration import Config
from flytekit.remote import FlyteRemote
remote = FlyteRemote(
config=Config.auto(),
default_project="flytesnacks",
default_domain="development",
interactive_mode_enabled=True,
)
```

```{admonition} Note
:class: Note
The `interactive_mode_enabled` flag is automatically set to `True` when running
in a Jupyter notebook environment, enabling interactive registration and execution
of workflows.
```

## Running a task or a workflow

You can run entities (tasks or workflows) using the `FlyteRemote`
{py:meth}`~flytekit.remote.remote.FlyteRemote.execute` method.
During execution, `flytekit` first checks if the entity is registered with the
Flyte backend, and if not, registers it before execution.

```{code-block} python
execution = remote.execute(my_task, inputs={"name": "Flyte"})
execution = remote.execute(my_wf, inputs={"name": "Flyte"})
```

You can then fetch the inputs and outputs of the execution by following the steps
in {ref}`<getting_started_run_and_schedule_fetch_execution>`.

## When Does Interactive `FlyteRemote` Re-register an Entity?

The interactive `FlyteRemote` client re-registers an entity whenever it's
redefined in the notebook, including when you re-execute a cell containing the
entity definition, even if the entity remains unchanged. This behavior facilitates
iterative development and debugging of tasks and workflows in a Jupyter notebook.

## What's next?

In this guide, you learned how to develop and run tasks and workflows in a
Jupyter Notebook environment using interactive `FlyteRemote`.

In the next guide, you'll learn how to visualize tasks using Flyte Decks.
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,8 @@ execution = remote.execute(flyte_task, inputs={"name": "Kermit"})
You can also launch tasks via `flytectl`, learn more in the {ref}`User Guide <remote_task>`
```

(getting_started_run_and_schedule_fetch_execution)=

## Fetching inputs and outputs of an execution

By default, {py:meth}`FlyteRemote.execute <flytekit.remote.remote.FlyteRemote.execute>`
Expand Down Expand Up @@ -342,4 +344,5 @@ In this guide, you learned about how to:
- Run tasks, workflows, and launch plans using `FlyteRemote`.
- Create a cron schedule to run a launch plan at a specified time interval.

In the next guide, you'll learn how to visualize tasks using Flyte Decks.
In the next guide, you'll learn how to develop and run tasks and workflows in
a Jupyter Notebook environment.

0 comments on commit 8f86c44

Please sign in to comment.