[BUG] Getting unsupported operand type(s) for +: 'dict' and 'dict' when running chat-with-wikipedia flow #3834

sgahlot · 2024-10-25T15:14:31Z

Describe the bug
When running chat-with-wikipedia flow, I'm encountering the following error:

...
  File "/usr/local/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 145, in <dictcomp>
    key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0)
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

How To Reproduce the bug
Steps to reproduce the behavior, how frequent can you experience the bug:

I used Open in GitHub Codespaces option to use the pre-built environment
Run the examples/flows/chat/chat-with-wikipedia flow, using Run it with standard mode option
The command I see in the terminal is: /usr/local/bin/python3 -m promptflow._cli._pf.entry flow test --flow /workspaces/promptflow/examples/flows/chat/chat-with-wikipedia --user-agent "prompt-flow-extension/1.20.2 (linux; x64) VSCode/1.94.2"
The following error comes every single time this flow is run

2024-10-25 14:55:25 +0000   17101 execution.flow     INFO     Executing node augmented_chat. node run id: edeaf1be-c107-4555-851f-38d5706e7a30_augmented_chat_0
WARNING:opentelemetry.attributes:Invalid type dict for attribute '__computed__.cumulative_token_count.completion' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type dict for attribute '__computed__.cumulative_token_count.prompt' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type dict for attribute 'llm.usage.completion_tokens_details' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type dict for attribute 'llm.usage.prompt_tokens_details' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type dict for attribute '__computed__.cumulative_token_count.completion' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
WARNING:opentelemetry.attributes:Invalid type dict for attribute '__computed__.cumulative_token_count.prompt' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types
2024-10-25 14:55:26 +0000   17101 execution          ERROR    Node augmented_chat in line 0 failed. Exception: Execution failure in 'augmented_chat': (TypeError) unsupported operand type(s) for +: 'dict' and 'dict'.
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/promptflow/_core/flow_execution_context.py", line 182, in _invoke_tool_inner
    return f(**kwargs)
  File "/usr/local/lib/python3.9/site-packages/promptflow/executor/flow_executor.py", line 1344, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 562, in wrapped
    token_collector.collect_openai_tokens_for_parent_span(span)
  File "/usr/local/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 143, in collect_openai_tokens_for_parent_span
    merged_tokens = {
  File "/usr/local/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 145, in <dictcomp>
    key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0)
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

Expected behavior
I am expecting the flow to run without any errors

Screenshots
I'm attaching two screenshots to show the error

Running Information(please complete the following information):

Promptflow Package Version using pf -v: [e.g. 0.0.102309906]

{
 "promptflow": "1.16.1",
 "promptflow-azure": "1.16.1",
 "promptflow-core": "1.16.1",
 "promptflow-devkit": "1.16.1",
 "promptflow-tracing": "1.16.1"
}

Executable '/usr/local/bin/python3.9'
Python (Linux) 3.9.20 (main, Oct 19 2024, 01:00:05) 
[GCC 10.2.1 20210110]

Operating System: Linux (GitHub Codespaces) - Debian GNU/Linux 11 (bullseye)
Python Version using python --version: Python 3.9.20

Additional context
This is my first time using GitHub Codespaces as well as Prompt flow, so if there's some additional initialization/config that I had to run, please let me know

The text was updated successfully, but these errors were encountered:

sgahlot · 2024-10-25T15:45:46Z

I also tried setting the tracing to none thinking it won't go to _trace.py but that didn't make any difference. But when I set the
PF_DISABLE_TRACING environment variable to true (in the same terminal) and re-ran the flow command, I didn't get this error as the tracing code didn't run in that case.

yeamusic21 · 2024-11-13T01:36:00Z

Looks like I had the same, or at least a similar, issue. See #3850

redhatpeter · 2024-12-12T14:48:00Z

it seems that same issue.
error message: pf.flow.test failed with UserErrorException: ServiceResponseException: Execution failure in 'qna_multi_agent_step': (ServiceResponseException) ("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", TypeError("unsupported operand type(s) for +: 'int' and 'dict'"))

command : .venv\Scripts\python.exe -m promptflow._cli._pf.entry flow test --flow c:\Users\leepete\Documents\3.AI-Data\Development\1.2AOAIFeatures\jake-promptflow\flows\semantic_kernel_agent_qna --user-agent "prompt-flow-extension/1.20.2 (win32; x64) VSCode/1.96.0"
{
"promptflow": "1.16.2",
"promptflow-azure": "1.16.2",
"promptflow-core": "1.16.2",
"promptflow-devkit": "1.16.2",
"promptflow-tracing": "1.16.2"
}
Python 3.11.7

apthagowda97 · 2024-12-13T08:20:10Z

Earlier I used get this as warning, now it changed to error. Any fixes??

# Description In the latest version of OpenAI some keys have tokens that are not ints. This is causing TypeErrors like `TypeError: unsupported operand type(s) for +: 'int' and 'dict'` This aims to handle None types and Dicts better in tracing. Should resolve bug #3834 # All Promptflow Contribution checklist: - [X] **The pull request does not introduce [breaking changes].** - [X] **CHANGELOG is updated for new features, bug fixes or other significant changes.** - [X] **I have read the [contribution guidelines](https://github.com/microsoft/promptflow/blob/main/CONTRIBUTING.md).** - [X] **I confirm that all new dependencies are compatible with the MIT license.** - [X] **Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: [suggested workflow](../CONTRIBUTING.md#suggested-workflow).** ## General Guidelines and Best Practices - [X] Title of the pull request is clear and informative. - [X] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, [see this page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md). ### Testing Guidelines - [X] Pull request includes test coverage for the included changes.

steve122192 · 2024-12-18T17:14:45Z

{
"promptflow": "1.16.2",
"promptflow-azure": "1.16.2",
"promptflow-core": "1.16.2",
"promptflow-devkit": "1.16.2",
"promptflow-tracing": "1.16.2"
}

Python 3.10.15

Error:
in
key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0)
TypeError: unsupported operand type(s) for +: 'int' and 'dict'

johnnieskywalker · 2024-12-18T18:11:14Z

{ "promptflow": "1.16.2", "promptflow-azure": "1.16.2", "promptflow-core": "1.16.2", "promptflow-devkit": "1.16.2", "promptflow-tracing": "1.16.2" }

Python 3.10.15

Error: in key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0) TypeError: unsupported operand type(s) for +: 'int' and 'dict'

in flow.dag.yaml at the end try

environment_variables:
  PF_DISABLE_TRACING: true
environment:
  python_requirements_txt: requirements.txt

steve122192 · 2024-12-18T18:35:56Z

{ "promptflow": "1.16.2", "promptflow-azure": "1.16.2", "promptflow-core": "1.16.2", "promptflow-devkit": "1.16.2", "promptflow-tracing": "1.16.2" }
Python 3.10.15
Error: in key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0) TypeError: unsupported operand type(s) for +: 'int' and 'dict'

in flow.dag.yaml at the end try

environment_variables:
PF_DISABLE_TRACING: true
environment:
python_requirements_txt: requirements.txt

That gets it to run but I do need the tracing

johnnieskywalker · 2024-12-18T18:43:02Z

{ "promptflow": "1.16.2", "promptflow-azure": "1.16.2", "promptflow-core": "1.16.2", "promptflow-devkit": "1.16.2", "promptflow-tracing": "1.16.2" }
Python 3.10.15
Error: in key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0) TypeError: unsupported operand type(s) for +: 'int' and 'dict'

in flow.dag.yaml at the end try
environment_variables:
PF_DISABLE_TRACING: true
environment:
python_requirements_txt: requirements.txt

That gets it to run but I do need the tracing

I dunno any other fix and this bug exists for months

martinmoldrup · 2024-12-19T09:10:31Z

I get the same issue in our production environment using prompt flow. The issue started happing [UTC]: 12/12/2024, 3:12:59.908 PM
and stopped again 12/13/2024, 12:31:06.007 PM

Then yesterday it started again yesterday 12/18/2024, 2:25:49.819 PM and is still happening.

In the code I have two calls to gpt-4o-mini. Interesting the issue always happens on line 48, and not on 45 that also calls the same model. (see screenshot)

Runtime: mcr.microsoft.com/azureml/promptflow/promptflow-runtime-stable:20241126.v3

Traceback (most recent call last):
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/opentelemetry/trace/init.py", line 570, in use_span
yield span
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/opentelemetry/sdk/trace/init.py", line 1091, in start_as_current_span
yield span
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 98, in start_as_current_span
yield span
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 556, in wrapped
output = func(*args, **kwargs)
File "/var/azureml-app/azureml-models/productrag/12/flow_chat_product_companion/tool_rewrite_query.py", line 76, in tool_rewrite_query
rewrite = rewrite_query_InfoCQR(connection, question, chat_history)
File "/var/azureml-app/azureml-models/productrag/12/flow_chat_product_companion/tool_rewrite_query.py", line 45, in rewrite_query_InfoCQR
rewrite_edit = prompt2(question=question, chat_history=chat_history, rewrite=rewrite)
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 561, in wrapped
token_collector.collect_openai_tokens_for_parent_span(span)
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 143, in collect_openai_tokens_for_parent_span
merged_tokens = {
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 144, in
key: self._span_id_to_tokens[parent_span_id].get(key, 0) + tokens.get(key, 0)
TypeError: unsupported operand type(s) for +: 'int' and 'dict'

johnnieskywalker · 2024-12-19T09:35:25Z

I worked it around by not using tracing on deployed endpoints I added additional loggers in python files

mallikarjun-malli · 2024-12-19T10:32:41Z

https://microsoft.github.io/promptflow/reference/changelog/promptflow-tracing.html shows that this issue was fixed in

Where is this release available?
Looks like it is not yet published.

apthagowda97 · 2024-12-19T12:00:03Z

Use the below code in Managed Online Deployment Yaml to disable tracing in v1.16.2 Promptflow for quick fix.

environment_variables:
  PF_DISABLE_TRACING: "true"

steve122192 · 2024-12-19T15:14:05Z

I get the same issue in our production environment using prompt flow. The issue started happing [UTC]: 12/12/2024, 3:12:59.908 PM and stopped again 12/13/2024, 12:31:06.007 PM

Then yesterday it started again yesterday 12/18/2024, 2:25:49.819 PM and is still happening.

In the code I have two calls to gpt-4o-mini. Interesting the issue always happens on line 48, and not on 45 that also calls the same model. (see screenshot)

Runtime: mcr.microsoft.com/azureml/promptflow/promptflow-runtime-stable:20241126.v3

Traceback (most recent call last): File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/opentelemetry/trace/init.py", line 570, in use_span yield span File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/opentelemetry/sdk/trace/init.py", line 1091, in start_as_current_span yield span File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 98, in start_as_current_span yield span File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 556, in wrapped output = func(*args, **kwargs) File "/var/azureml-app/azureml-models/productrag/12/flow_chat_product_companion/tool_rewrite_query.py", line 76, in tool_rewrite_query rewrite = rewrite_query_InfoCQR(connection, question, chat_history) File "/var/azureml-app/azureml-models/productrag/12/flow_chat_product_companion/tool_rewrite_query.py", line 45, in rewrite_query_InfoCQR rewrite_edit = prompt2(question=question, chat_history=chat_history, rewrite=rewrite) File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 561, in wrapped token_collector.collect_openai_tokens_for_parent_span(span) File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 143, in collect_openai_tokens_for_parent_span merged_tokens = { File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 144, in key: self._span_id_to_tokens[parent_span_id].get(key, 0) + tokens.get(key, 0) TypeError: unsupported operand type(s) for +: 'int' and 'dict'

Interesting, I was having no problems until yesterday as well

Worth mentioning... I had begun using a mix of gpt-4o and 4o-mini in my flow. I just tried switching it back to only using 4o and I'm no longer getting the error and tracing is working.

eses-wk · 2024-12-20T02:54:16Z

I'm using gpt-4o, also sees this issue. It happens when I am running multiple "Python" Promptflow block each with AsyncAzureOpenai client calling the chat completion API.

Suspect it is some conflict between Promptflow Tracing and the internal thread pooling handling when gathering multiple OpenAI async calls at the same time.

Disabling Tracing in flow.dag.yaml per @johnnieskywalker mentioned works well for me

environment_variables:
  PF_DISABLE_TRACING: true

sgahlot added the bug Something isn't working label Oct 25, 2024

chanedwin mentioned this issue Nov 3, 2024

fix: improve merge token logic to account for nested dicts #3825

Closed

8 tasks

redhatpeter mentioned this issue Dec 12, 2024

[BUG] Running Prompt flow locally produces errors #3751

Closed

KronemeyerJoshua mentioned this issue Dec 15, 2024

Fixed TypeError in Tracing issue with tokens that are dicts #3885

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Getting unsupported operand type(s) for +: 'dict' and 'dict' when running chat-with-wikipedia flow #3834

[BUG] Getting unsupported operand type(s) for +: 'dict' and 'dict' when running chat-with-wikipedia flow #3834

sgahlot commented Oct 25, 2024

sgahlot commented Oct 25, 2024 •

edited

Loading

yeamusic21 commented Nov 13, 2024 •

edited

Loading

redhatpeter commented Dec 12, 2024

apthagowda97 commented Dec 13, 2024

steve122192 commented Dec 18, 2024

johnnieskywalker commented Dec 18, 2024

steve122192 commented Dec 18, 2024

johnnieskywalker commented Dec 18, 2024

martinmoldrup commented Dec 19, 2024 •

edited

Loading

johnnieskywalker commented Dec 19, 2024

mallikarjun-malli commented Dec 19, 2024

apthagowda97 commented Dec 19, 2024

steve122192 commented Dec 19, 2024 •

edited

Loading

eses-wk commented Dec 20, 2024 •

edited

Loading

[BUG] Getting unsupported operand type(s) for +: 'dict' and 'dict' when running chat-with-wikipedia flow #3834

[BUG] Getting unsupported operand type(s) for +: 'dict' and 'dict' when running chat-with-wikipedia flow #3834

Comments

sgahlot commented Oct 25, 2024

sgahlot commented Oct 25, 2024 • edited Loading

yeamusic21 commented Nov 13, 2024 • edited Loading

redhatpeter commented Dec 12, 2024

apthagowda97 commented Dec 13, 2024

steve122192 commented Dec 18, 2024

johnnieskywalker commented Dec 18, 2024

steve122192 commented Dec 18, 2024

johnnieskywalker commented Dec 18, 2024

martinmoldrup commented Dec 19, 2024 • edited Loading

johnnieskywalker commented Dec 19, 2024

mallikarjun-malli commented Dec 19, 2024

apthagowda97 commented Dec 19, 2024

steve122192 commented Dec 19, 2024 • edited Loading

eses-wk commented Dec 20, 2024 • edited Loading

sgahlot commented Oct 25, 2024 •

edited

Loading

yeamusic21 commented Nov 13, 2024 •

edited

Loading

martinmoldrup commented Dec 19, 2024 •

edited

Loading

steve122192 commented Dec 19, 2024 •

edited

Loading

eses-wk commented Dec 20, 2024 •

edited

Loading