Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

使用openai接口规范时遇到stop words 错误 #370

Open
Meshinfo opened this issue Sep 29, 2024 · 2 comments
Open

使用openai接口规范时遇到stop words 错误 #370

Meshinfo opened this issue Sep 29, 2024 · 2 comments
Labels

Comments

@Meshinfo
Copy link

发现一个bug:如果利用openai的api兼容接口,llm使用glm-4,在使用assistant_rag这个demo时候。会遇到rror code: 400 - {'error': {'message': 'stop:最大只能有4个停止词 (request id: 202409291149558009254221372292)', 'type': '', 'param': '', 'code': '1214'}} 的问题。OPENAI规范里确实stop words只能有4个。但是使用qwen-max模型不会有这问题。请问一下,stop words是哪一个生成的?有什么办法能够暂时绕过这个错误吗?

@Meshinfo Meshinfo changed the title metaljacket@meshinfo.cn 使用openai接口规范时遇到stop words 错误 Sep 29, 2024
@Meshinfo
Copy link
Author

研究了一下,最后生成的'stop': ['✿RESULT✿', '✿RETURN✿', 'Observation:', 'Observation:\n', '"], "instruction":'],超过了1个,直接在ultis.py里简单粗暴的修改了:
def merge_generate_cfgs(base_generate_cfg: Optional[dict], new_generate_cfg: Optional[dict]) -> dict:
generate_cfg: dict = copy.deepcopy(base_generate_cfg or {})
if new_generate_cfg:
for k, v in new_generate_cfg.items():
if k == 'stop':
stop = generate_cfg.get('stop', [])
stop = stop + [s for s in v if s not in stop]
### generate_cfg['stop'] = stop[:4]
else:
generate_cfg[k] = v
return generate_cfg
这应该也算个bug吧,希望能修复一下。

@Meshinfo
Copy link
Author

或者修改gen_keyword里,把new_generate_cfg={'stop':['Obersrvation','Oberservation:\n']} 删掉一个元素.

@JianxinMa JianxinMa added the Bug label Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants