Conversation Titles Generated in Incorrect Language #2077
Replies: 4 comments 2 replies
-
Happened to me too. I can reproduce by just writing „hey“ |
Beta Was this translation helpful? Give feedback.
-
@ndrsfel @Fakamoto I can’t reproduce it. What model are you using for generating the title? (I tried 10 times with gpt-4-turbo and gpt-3.5-turbo, everything seems to be working correctly) |
Beta Was this translation helpful? Give feedback.
-
@ndrsfel @Fakamoto This is a prompting issue and will almost always have a degree of variability even with a better titling prompt. The prompt includes the initial user message as well as the AI's response (both truncated to save context), and certain tokens found in either of those or as the result of the cut-off text, can produce this issue. Yes it's ideal to always generate the expected language but this is part of the flaw in working with LLMs for structured output. As you can see, this happens on ChatGPT as well, although they probably have a more robust guardrails system to prevent this when it first started happening. https://www.reddit.com/r/ChatGPT/comments/15xi4y3/gpt_randomly_generating_titles_in_different/ https://www.reddit.com/r/ChatGPT/comments/10lwg5x/chatgpt_behaving_really_odd_spanish_chat_titles/ I'm not categorizing this as a bug but rather as a feature request for added guardrails |
Beta Was this translation helpful? Give feedback.
-
I am getting this error pretty consistently - happy to provide any testing data if it would help. For now, I am going to try to edit the title prompts to force english titles |
Beta Was this translation helpful? Give feedback.
-
What happened?
Occasionally, the titles of conversations are automatically generated in Spanish, despite the conversations being conducted entirely in English. This inconsistent behavior does not align with the language used within the chat sessions and occurs unpredictably. When the issue manifests, it leads to confusion and a disjointed user experience. Ideally, conversation titles should accurately reflect the language predominantly used within the conversation, or at least adhere to the user's default or selected language preferences
Steps to Reproduce
Please note that this issue occurs intermittently and may not be reproducible in every instance of conversation title generation.
Engage in a chat conversation entirely in English (or in a language other than Spanish).
Complete the conversation and allow the system to generate a title for it.
Observe the title of the conversation, noting any instances where it is generated in Spanish, despite the chat being in English.
What browsers are you seeing the problem on?
Chrome
Relevant log output
No response
Screenshots
No response
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions