Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

366 add cli commands to inspectrestart the environment #409

Merged
merged 39 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
93119d9
ran into __root__ and Pydantic None error
stan-dot Mar 14, 2024
1c22fc8
change the vscode settings to explicit
stan-dot Mar 28, 2024
c6642b7
add tests
stan-dot Apr 2, 2024
686221f
update tests and schema
stan-dot Apr 2, 2024
eec09d7
add polling to environment reset
stan-dot Apr 3, 2024
a992f32
error with patching sleepy test
stan-dot Apr 3, 2024
09e345c
change timeout in the cli
stan-dot Apr 3, 2024
11838c5
lint
stan-dot Apr 4, 2024
32add10
(#366) fix function reference and mocking in cli test
dperl-dls Apr 24, 2024
41ee94c
minor formatting changes
stan-dot Apr 25, 2024
89fcced
fix B008 issue default args
stan-dot Apr 26, 2024
2dd5ec0
update the return line in the rest client
stan-dot May 2, 2024
d829595
chang task into example task
stan-dot May 13, 2024
2a862c4
follow suggestions for error handling
stan-dot May 13, 2024
0e9d60a
fix as suggested
stan-dot May 13, 2024
9f35b3f
revert settings change
stan-dot May 15, 2024
0854b0f
vscode settings restore
stan-dot May 15, 2024
8dc6cbc
revert formatting changes
stan-dot May 15, 2024
4321cd5
settings corrected
stan-dot May 15, 2024
959e8b0
fix import
stan-dot May 17, 2024
9cd2f90
make lint happy
stan-dot May 23, 2024
ffc7412
removed all 'raise' from the cli'
stan-dot May 23, 2024
0beec9c
change more into logging not print
stan-dot May 23, 2024
838cd98
remove logs from cli
stan-dot May 28, 2024
3ec47fb
move to pprint
stan-dot May 30, 2024
a6a8d6f
change the tests for a nicer set of commit messages
stan-dot May 30, 2024
b902ac9
remove redundant test
stan-dot May 30, 2024
6fb6a7a
finally fix the test to query the right endpoints
stan-dot May 30, 2024
9408dc8
make better coverage
stan-dot May 30, 2024
94a7b05
rollback raise changes
stan-dot May 31, 2024
d13234c
fix test expected message string
stan-dot Jun 3, 2024
29d64bf
add docs on env reload
stan-dot Jun 4, 2024
a1c2321
small changes to reload cli logic
stan-dot Jun 4, 2024
9ff48be
add pytest raises to the timout test
stan-dot Jun 11, 2024
41fca37
fix timeout test
stan-dot Jun 11, 2024
4427cdb
remove 1 line to pass codecov
stan-dot Jun 11, 2024
d0d9694
Add test for environment reload failure
callumforrester Jun 12, 2024
260afb3
Merge branch 'main' into 366-add-cli-commands-to-inspectrestart-the-e…
stan-dot Jun 20, 2024
665e396
fix lint
stan-dot Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/reference/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,8 @@ paths:
'200':
content:
application/json:
schema: {}
schema:
$ref: '#/components/schemas/EnvironmentResponse'
description: Successful Response
summary: Delete Environment
get:
Expand Down
17 changes: 11 additions & 6 deletions docs/tutorials/dev-run.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,28 @@
# Run/Debug in a Developer Environment
# Run/Debug in a Developer Environment

Assuming you have setup a developer environment, you can run a development version of the bluesky worker.


## Start Bluesky Worker

Ensure you are inside your virtual environment:
```

```
source venv/bin/activate
```

You will need to follow the instructions for setting up ActiveMQ as in [run cli instructions](../how-to/run-cli.md).

You will need to follow the instructions for setting up ActiveMQ as in [run cli instructions](../how-to/run-cli.md).

The worker will be available from the command line (`blueapi serve`), but can be started from vscode with additional
The worker will be available from the command line (`blueapi serve`), but can be started from vscode with additional
debugging capabilities.

1. Navigate to "Run and Debug" in the left hand menu.
2. Select "Worker Service" from the debug configuration.
3. Click the green "Run Button"

[debug in vscode](../images/debug-vscode.png)

## Develop devices

When you select the 'scratch directory' option - where you have devices (dodal) and plans (BLxx-beamline) in a place like `/dls_sw/BLXX/software/blueapi/scratch`, then the list of devices available will refresh without interfacing with the K8S cluster. Just run the command `blueapi env -r` or `blueapi env --reload`.

With this setup you get a developer loop: "write devices - write plans - test them with blueapi".
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ dev = [
"mypy",
"pytest-cov",
"pytest-asyncio",
"responses",
"ruff",
"sphinx-autobuild==2024.2.4", # Later versions have a clash with fastapi<0.99, remove pin when fastapi is a higher version
"sphinx-copybutton",
Expand Down
59 changes: 57 additions & 2 deletions src/blueapi/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from functools import wraps
from pathlib import Path
from pprint import pprint
from time import sleep

import click
from pydantic import ValidationError
Expand Down Expand Up @@ -181,8 +182,9 @@ def run_plan(
if config.stomp is not None:
_message_template = StompMessagingTemplate.autoconfigured(config.stomp)
else:
pprint("ERROR: Cannot run plans without Stomp configuration to track progress")
return
raise RuntimeError(
"Cannot run plans without Stomp configuration to track progress"
)
event_bus_client = EventBusClient(_message_template)
finished_event: deque[WorkerEvent] = deque()

Expand Down Expand Up @@ -278,6 +280,59 @@ def stop(obj: dict) -> None:
pprint(client.cancel_current_task(state=WorkerState.STOPPING))


@controller.command(name="env")
@check_connection
@click.option(
"-r",
"--reload",
is_flag=True,
type=bool,
help="Reload the current environment",
default=False,
)
@click.pass_obj
def env(obj: dict, reload: bool | None) -> None:
"""
Inspect or restart the environment
"""

assert isinstance(client := obj["rest_client"], BlueapiRestClient)
if reload:
# Reload the environment if needed
print("Reloading the environment...")
try:
deserialized = client.reload_environment()
print(deserialized)

except BlueskyRemoteError as e:
raise BlueskyRemoteError("Failed to reload the environment") from e

# Initialize a variable to keep track of the environment status
environment_initialized = False
polling_count = 0
max_polling_count = 10
# Use a while loop to keep checking until the environment is initialized
while not environment_initialized and polling_count < max_polling_count:
# Fetch the current environment status
environment_status = client.get_environment()

# Check if the environment is initialized
if environment_status.initialized:
print("Environment is initialized.")
environment_initialized = True
else:
print("Waiting for environment to initialize...")
polling_count += 1
sleep(1) # Wait for 1 seconds before checking again
if polling_count == max_polling_count:
raise TimeoutError("Environment initialization timed out.")

# Once out of the loop, print the initialized environment status
print(environment_status)
else:
print(client.get_environment())


# helper function
def process_event_after_finished(event: WorkerEvent, logger: logging.Logger):
if event.is_error():
Expand Down
20 changes: 14 additions & 6 deletions src/blueapi/cli/rest.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from blueapi.service.model import (
DeviceModel,
DeviceResponse,
EnvironmentResponse,
PlanModel,
PlanResponse,
TaskResponse,
Expand Down Expand Up @@ -115,16 +116,23 @@ def _request_and_deserialize(
raise_if: Callable[[requests.Response], bool] = _is_exception,
) -> T:
url = self._url(suffix)
response = requests.request(method, url, json=data)
if data:
response = requests.request(method, url, json=data)
else:
response = requests.request(method, url)
if raise_if(response):
message = get_status_message(response.status_code)
error_message = f"""Response failed with text: {response.text},
with error code: {response.status_code}
which corresponds to {message}"""
raise BlueskyRemoteError(error_message)
raise BlueskyRemoteError(str(response))
deserialized = parse_obj_as(target_type, response.json())
return deserialized

def _url(self, suffix: str) -> str:
base_url = f"{self._config.protocol}://{self._config.host}:{self._config.port}"
return f"{base_url}{suffix}"

def get_environment(self) -> EnvironmentResponse:
return self._request_and_deserialize("/environment", EnvironmentResponse)

def reload_environment(self) -> EnvironmentResponse:
return self._request_and_deserialize(
"/environment", EnvironmentResponse, method="DELETE"
)
10 changes: 7 additions & 3 deletions src/blueapi/service/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,17 +89,18 @@ def get_environment(
return EnvironmentResponse(initialized=handler.initialized)


@app.delete("/environment")
@app.delete("/environment", response_model=EnvironmentResponse)
async def delete_environment(
background_tasks: BackgroundTasks,
handler: BlueskyHandler = Depends(get_handler),
):
) -> EnvironmentResponse:
def restart_handler(handler: BlueskyHandler):
handler.stop()
handler.start()

if handler.initialized:
background_tasks.add_task(restart_handler, handler)
return EnvironmentResponse(initialized=False)


@app.get("/plans", response_model=PlanResponse)
Expand Down Expand Up @@ -132,6 +133,9 @@ def get_device_by_name(name: str, handler: BlueskyHandler = Depends(get_handler)
return handler.get_device(name)


example_task = Task(name="count", params={"detectors": ["x"]})


@app.post(
"/tasks",
response_model=TaskResponse,
Expand All @@ -140,7 +144,7 @@ def get_device_by_name(name: str, handler: BlueskyHandler = Depends(get_handler)
def submit_task(
request: Request,
response: Response,
task: Task = Body(..., example=Task(name="count", params={"detectors": ["x"]})), # noqa: B008
task: Task = Body(..., example=example_task),
handler: BlueskyHandler = Depends(get_handler),
):
"""Submit a task to the worker."""
Expand Down
Loading
Loading