You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Bug] [OssResourceManager] The root path of the OSS configuration must not start with "/", which conflicts with the configuration of the History Server.
#4088
Open
2 of 3 tasks
jiangwwwei opened this issue
Dec 24, 2024
· 0 comments
I had searched in the issues and found no similar issues.
What happened
Dinky's OssResourceManager relies on the aws-sdk implementation, and when constructing requests using its API, the file path is the Object Name, which should not start with a "/" (the aws SDK will automatically add a "/" at the beginning of the object name when building the HTTP request). If the root path in Dinky's Oss configuration starts with "/", it will cause an error indicating an invalid object name.
However, the HistoryServer's request to oss is implemented by flink-s3-fs-presto, and it requests the direct path. If a "/" is not added at the beginning, it will report an error stating that it is not an absolute path.
What you expected to happen
Both the resource manager and the history server can access OSS normally.
How to reproduce
Use OSS Resource
In the root path configuration, add and do not add "/"
Observe the file upload/download status of the Resource and the operation of the History Server.
Anything else
It is worth mentioning: In OssResourceManager, the "/" before the path in getFullDirectoryStructure() is specifically handled and removed, but other methods do not have this treatment.
Search before asking
What happened
Dinky's OssResourceManager relies on the aws-sdk implementation, and when constructing requests using its API, the file path is the Object Name, which should not start with a "/" (the aws SDK will automatically add a "/" at the beginning of the object name when building the HTTP request). If the root path in Dinky's Oss configuration starts with "/", it will cause an error indicating an invalid object name.
However, the HistoryServer's request to oss is implemented by flink-s3-fs-presto, and it requests the direct path. If a "/" is not added at the beginning, it will report an error stating that it is not an absolute path.
What you expected to happen
Both the resource manager and the history server can access OSS normally.
How to reproduce
Anything else
It is worth mentioning: In OssResourceManager, the "/" before the path in getFullDirectoryStructure() is specifically handled and removed, but other methods do not have this treatment.
Version
1.2.0
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: