Managing Compute Resources for Containers

[jeffjunzhang]

(1) In clipper, when we deploy a model container, e.g., a trained pytorch model, how do we set the CPU and memory limits for it? Is this supported by Clipper admin? If not, any solution to achieve this goal?
(2) Even when we start the clipper cluster, can we set a CPU/memory limit on the whole cluster? Also, is there a way to set the CPU/memory limits for query_frontend and mange_frontend as well?

Thank you for your help!

[withsmilo]

Hi, @jeffjunzhang .
Currently clipper_admin does not support it. So our team modified the following yaml files manually for the purpose you mentioned.

• https://github.com/ucbrise/clipper/blob/develop/clipper_admin/clipper_admin/kubernetes/mgmt-frontend-deployment.yaml
• https://github.com/ucbrise/clipper/blob/develop/clipper_admin/clipper_admin/kubernetes/query-frontend-deployment.yaml
• https://github.com/ucbrise/clipper/blob/develop/clipper_admin/clipper_admin/kubernetes/model-container-template.yaml

I will try to add a new feature to manage k8s resources to Clipper. Thanks for your good point.

[jeffjunzhang]

Thank you, @withsmilo
Btw, if I'm using the docker container manager (DockerContainerManager()), which files should I manually modify?

After the modification, we should re-compile all the source codes from scratch, right?

[withsmilo]

@jeffjunzhang ,
Are you testing Clipper on your local PC? If on mac, see https://docs.docker.com/docker-for-mac/#advanced.

[jeffjunzhang]

@withsmilo Thanks again.
I'm testing on a local 24-core server running a Ubuntu.

Basically, say if I have 1 pytorch model to deploy with Clipper, I want to limit the model container with 10 cores+ some memory, and also 10 cores for the query front-end container and management container.

What's the simplest way to achieve these?

[withsmilo]

@jeffjunzhang ,
Clipper is using DockerContainerManager for test only, not production. How about using docker update command to update the resource limit of some containers after deploying Clipper and your models? See https://docs.docker.com/engine/reference/commandline/update/. I think this is the simplest way.