Difference between the two implementations of multi-route

[secsilm]

Here are two implementations of multi route, they seem to have the same effect. Which one do you recommend? Is there any difference in performance?

import time
from types import MappingProxyType as ImmutableDict
from typing import List

from mosec import Server, ValidationError, Worker, get_logger, Runtime

logger = get_logger()


class Model1(Worker):
    """Sample Class."""

    example = ['model1 test'] * 4

    def forward(self, data: dict) -> float:
        logger.debug(f"model1 received {data=}")
        # Customized, simple input validation
        return data


class Model2(Worker):
    """Sample Class."""

    example = ['model2 test'] * 4

    def forward(self, data: List[float]) -> List[float]:
        logger.debug(f"model2 received {data=}")
        return data


class Model3(Worker):
    """Sample Class."""

    example = ['model3 test'] * 4

    def forward(self, data: float) -> dict:
        logger.debug(f"model3 received {data=}")
        return data


if __name__ == "__main__":
    server = Server()

    # append_worker
    # server.append_worker(Model1, max_batch_size=8, route='/model1')
    # server.append_worker(Model2, max_batch_size=8, route='/model2')
    # server.append_worker(Model3, max_batch_size=8, route='/model3')

    # Runtime, ref: https://mosecorg.github.io/mosec/examples/multi_route.html
    model1 = Runtime(Model1, max_batch_size=8)
    model2 = Runtime(Model2, max_batch_size=8)
    model3 = Runtime(Model3, max_batch_size=8)
    server.register_runtime({"/model1": [model1], "/model2": [model2], "/model3": [model3]})

    server.run()

Additionally, I see that the config in the logs is also the same:

2024-10-30T03:31:20.368113691Z DEBUG mosec: parse service arguments conf=Config { path: "/tmp/mosec_f6cb34aa", capacity: 1024, timeout: 3000, address: "0.0.0.0", port: 6000, namespace: "mosec_service", log_level: "debug", runtimes: [Runtime { max_batch_size: 8, max_wait_time: 10, worker: "Model1_1" }, Runtime { max_batch_size: 8, max_wait_time: 10, worker: "Model3_3" }, Runtime { max_batch_size: 8, max_wait_time: 10, worker: "Model2_2" }], routes: [(/model1: [Model1_1], resp(application/json)), (/model2: [Model2_2], resp(application/json)), (/model3: [Model3_3], resp(application/json))] }

[kemingy]

They work exactly in the same way.

If you need to share one of the workers in multi-routes, for example, you may need a general text embedding worker for both text classification and text summarization; then, the second one might look clear. Otherwise, you can use whatever you like.

[secsilm]

OK, thanks.