[ISSUE] Serving Endpoint cannot handle if a list is returned from a query

[nsenno-dbr]

Description
The majority of the serving endpoint queries return something with a dictionary-like JSON structure. However, there are some models that return a list of dictionaries. Currently, the SDK fails when querying these endpoints. Interestingly, the same query does NOT fail when querying the endpoint using the API or the Databricks native UI.

Reproduction
See these built in chains here from Langchain which are part of their demonstration of how to build a RAG application. In an ideal world, this would work using Databricks serving endpoints.

Note that I plan on putting together a working repro to share in the near future.

https://python.langchain.com/docs/tutorials/rag/#built-in-chains

Expected behavior
Models that are successfully deployed to Databricks model serving endpoints with a valid input and output schema are able to be queried using the SDK w.serving_endpoint.query API, the same as making a request to the REST API.

Is it a regression?
Unsure. Probably not

Debug Logs
The SDK logs helpful debugging information when debug logging is enabled. Set the log level to debug by adding logging.basicConfig(level=logging.DEBUG) to your program, and include the logs here.

Unable to send logs because they are currently in a customers isolated environment. I am working on creating a repro using the Databricks RAG field demo.

Other Information
Servlerless DBR
databricks-sdk 0.34.0 (was explicitly pinned and installed on serverless)

Additional context
Note that I am able to mitigate this issue if I add a check that the object returned by the POST. After checking if it is a list I simply take the first element which is itself a dictionary. There could be some additional checking as part of this process.