-
Notifications
You must be signed in to change notification settings - Fork 626
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: start chatglm-6b locally err #926
Comments
close for openllm 0.6 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
I executed "TRUST_REMOTE_CODE=True openllm start /usr1/models/chatglm-6b" and model was loaded successfully, swagger was available, but got error using v1/chat/compelitioins,
seems that the transformer version required by openllm and transformer version required by chatglm-b are not compatible.
To reproduce
TRUST_REMOTE_CODE=True openllm start /usr1/models/chatglm-6b
Logs
Environment
Name: bentoml
Version: 1.1.11
Summary: BentoML: Build Production-Grade AI Applications
Home-page: None
Author: None
Author-email: BentoML Team contact@bentoml.com
License: Apache-2.0
Location: /opt/buildtools/python-3.9.2/lib/python3.9/site-packages
Requires: prometheus-client, python-json-logger, pathspec, fs, rich, numpy, packaging, opentelemetry-instrumentation, httpx, opentelemetry-util-http, click-option-group, psutil, opentelemetry-api, opentelemetry-instrumentation-aiohttp-client, opentelemetry-semantic-conventions, jinja2, schema, deepmerge, attrs, opentelemetry-sdk, python-dateutil, simple-di, cloudpickle, pip-requirements-parser, uvicorn, requests, aiohttp, watchfiles, circus, python-multipart, inflection, click, pip-tools, pyyaml, starlette, cattrs, nvidia-ml-py, opentelemetry-instrumentation-asgi
Required-by: openllm
Name: transformers
Version: 4.38.2
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /opt/buildtools/python-3.9.2/lib/python3.9/site-packages
Requires: requests, numpy, regex, huggingface-hub, filelock, safetensors, packaging, tokenizers, pyyaml, tqdm
Required-by: optimum, openllm
Python 3.9.2
System information (Optional)
No response
The text was updated successfully, but these errors were encountered: