[Bug]: "Message":"Missing metadata for chunked upload" and for file upload > 10 MB #38801
Comments
sjash
added
0. Needs triage
|
You can try for change default max_chunk_size :
good luck |
|
Please update to 26.0.3 and report back please 👍 |
|
@nadi-dx : increasing the max chunks size resolved the issue. |
I upgraded the version to 26.0.3, the issue persisit. But if we manually change the chunk size suggested by @nadi-dx , it resolves the issue. |
|
Changing the chunk size did not fix for me.
Opened separate issue nextcloud/desktop#6079 |
|
Nextcloud Hub 6 (27.1.1) still persist |
|
I can confirm, the problem still exists in NC 27.1.1. As soon as the file to be uploaded exceeds the value in max_chunk_size, the upload aborts without error message. Workaround: |
|
I confirm that the same thing happens to me, but changing the max_chunk_size parameters, the problem persists, I cannot upload 10MB files with the client. Edit:
Edit2: I have gone back to version 3.9.4 and I have no problem with uploading files larger than 10MB, now what do I do with my users that are more than 600? :D |
|
I've got the same problem on mac with client 3.10.0 . No problems on linux client 3.9.3 - fedora for the same files. |
|
This happens to me on 3.10.0, but downgrading to 3.8.2 works |
|
Hi, can I do something in the meantime, I love nextcloud, but this kind of bugs make me look bad to my users. Any manual or method I can use just to solve this problem? |
|
If related to this... nextcloud/desktop#6079 then it's a client issue so downgrading is the only solution for now. 3.9.4 works for me. |
|
Will I have to compile this with the patch resolved? I'm still having problems with this bug, thanks a lot guys. |
|
nextcloud ios still has this issue. |
|
I have this issue, but it also happens when I upload using the web client. I don't know if this discussion is around a fix in the desktop clients, or also the web client? Can someone verify? Or is my issue different? |
|
I think some of us are describing an issue that is caused by the desktop client, and some of us are describing an issue caused by the server. The confusing part is that it's the same error message but actually two separate issues. Although some of us initially suspected a server-side problem, the same error message actually points to a client-related issue for us. If you are having issues on the web client, then it's not a desktop client issue, but a server issue. Steps to determine the cause: |
This is definitely best described as 3. I first noticed it after moving my nextcloud instance (which runs in containers) to a new host. When I did so I updated it. It wasnt woefully out of date or anything. I don't recall what versions i went to or from. Currently on 27.1.2. I noticed the problem after attempting to sync a folder of photos and videos from my kids soccer game using the desktop client (windows 11, i believe the client is 3.10, but cant seem to figure out where to find the version in the client), none of the videos would upload, but the pictures did fine. Some trial and error, i found that around 10mb was the point where uploads started to fail, and i also couldn't upload via the web ui. The client and the server both give me very similar errors. The one about the metadata at the top, and then a not found error on whatever file i was uploading. The web client, while uploading, just seems to silently fail. I get a progress bar for the upload, then it just goes away, and the file doesnt upload. I have to set server log level 0 to get any error at all out of it in the logs. Let me know if I can provide any more info. Thanks! |
This commit fixes the problem with missing metadata for chunked uploads. Signed-off-by: Christoph Fiehe <c.fiehe@eurodata.de>
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
@solracsf Yes, you are right. I missed that |
|
@sjash Does the problem go away if you configure a memcache? You appear to lack any |
@joshtrichards Adding Memcache has earlier resulted in file locking in my environments. Additionally, I do not see a direct link where part of file which where not yet uploaded/ to-be uploaded in NC to have a relation with memcache. I recommend for thorough checking of client version 3.10.X because after setting the chunk size as suggested by @nadi-dx , I'm able to upload file. But when I try the same using client version >= 3.10.X it results the same error to client. |
|
The metadata is stored in the cache. The chunk size change works around the issue, but doesn't solve it. You'll just run into it at a different file size and/or be limited in what file sizes you can upload (by your web server/proxy configuration). The reason you're seeing it only in the newer client version(s) is because that's when support for v2 chunked upload streaming for s3 was added (nextcloud/desktop#5819). #41198 is a follow-up (not yet implemented) that will likely tackle this in environments where caching isn't functioning by falling back on the old behavior. But that also means lower performance and higher disk usage (lack of #27034). I'm not sure what the locking matter is you're referring to, but you can just disable locking if need be. |
I would like to add 4. - Mobile client.
Just ran into the following problem:
I can provide screenshots, client protocol and server protocol if needed. |
We just had the same issue, fresh install, activating Redis solved the s3 chunking issue immediately. |
Does this mean you got it working using Redis? I'm also running Redis nearly plain out of the box (just installed it and configured nextcloud to use it) and still got the "missing metadata for chunked upload" error. Had a fresh install of nextcloud (27.1.3) yesterday. I'm receiving the error by using the iOS App of nextcloud (Version 4.9.1.0) |
|
Yes, you need a memory Cache for the Chunks not to get lost. |
|
did you make changes to the /etc/redis/redis.conf ? |
|
can confirm, simply enabling redis did nothing for my issue. I: Issue is exactly the same as before. Is more than the basic configuration required here? |
|
It depends on your setup. You should only configure Redis as a distributed cache if it is actually shared between all possible Nextcloud nodes, which I assume could be an issue in a kubernetes setup (as you mention pods). However there was a bug that even without a proper distributed cache we attempted to make use if it which has been fixed with #41354 and will be part of this weeks maintenance releases. |
|
anyone using Garage as their s3 implementation might be hitting https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/669 setting a part number marker of 0 when listing parts of an upload would erroneously leave off the first part (since part numbers are only valid between 1 & 10000, garage assumed part number markers were only valid for the same range and clamped 0 to 1) this manifests as the call to CompleteMultipartUpload failing in cases where there was only 1 part, and uploading broken files in cases where there was more than one part |
TL;DRIf you are using the desktop client (I only tested it on Windows) and still have the problem after increasing Full messageHello, If like me, you stuble upon this issue with the "Missing metadata for chunked upload" using the desktop client (Windows for my case) and you also tried suggestions from this issue like this one or increasing the php.ini While looking for a solution to my problem, I stumbled upon this documentation page Which led me to this one and more importantly this section This page states that you can create a environment variable named Hope this helps. |
Bug description
My Nextcloud instance version is 26.0.2 with mariadb and s3 as primary storage.
After all configuration, I'm unable tyo upload large file of size 10MB and higher.
The same configuration is working perfectly for version 25.
Steps to reproduce
Expected behavior
The issue exist in version 26 and above where s3 primary storage is selected. The same configuration is working perfectly for version 25.
Installation method
Community Manual installation with Archive
Nextcloud Server version
26
Operating system
RHEL/CentOS
PHP engine version
PHP 8.2
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
Configuration report
{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "mccloud.example.in" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "26.0.2.1", "overwrite.cli.url": "https:\/\/mccloud.example.in", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory", "objectstore": { "class": "\\OC\\Files\\ObjectStore\\S3", "arguments": { "bucket": "mccloud1", "autocreate": true, "key": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "hostname": "s3-pdc.example.in", "port": 443, "use_ssl": true, "region": "optional", "use_path_style": true } }, "maintenance": false, "bulkupload.enabled": "true", "filelocking.enabled": false, "knowledgebaseenabled": false, "logtimezone": "Asia\/Kolkata", "log_rotate_size": 52428800, "loglevel": 0, "lost_password_link": "https:\/\/adauth.example.in\/forget-password", "app_install_overwrite": [] } }List of activated Apps
Nextcloud Signing status
Nextcloud Logs
{"reqId":"ZIlFLoMmYtUHgHCLRcxebgAAzQM","level":0,"time":"2023-06-14T10:12:22+05:30","remoteAddr":"10.20.4.25","user":"ecadmin","app":"webdav","method":"PUT","url":"/remote.php/dav/uploads/ecadmin/web-file-upload-0017df3152a12b2989c7740d0c8fe876-1686717741220/1","message":"Missing metadata for chunked upload","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36","version":"26.0.2.1","exception":{"Exception":"Sabre\\DAV\\Exception\\PreconditionFailed","Message":"Missing metadata for chunked upload","Code":0,"Trace":[{"file":"/web/service/apps/dav/lib/Upload/ChunkingV2Plugin.php","line":148,"function":"checkPrerequisites","class":"OCA\\DAV\\Upload\\ChunkingV2Plugin","type":"->"},{"file":"/web/service/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"beforePut","class":"OCA\\DAV\\Upload\\ChunkingV2Plugin","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":456,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/apps/dav/lib/Server.php","line":366,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/web/service/remote.php","line":172,"args":["/web/service/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/web/service/apps/dav/lib/Upload/ChunkingV2Plugin.php","Line":284,"message":"Missing metadata for chunked upload","exception":{"header":null},"CustomMessage":"Missing metadata for chunked upload"}} {"reqId":"ZIlFLoMmYtUHgHCLRcxebwAAzQM","level":0,"time":"2023-06-14T10:12:22+05:30","remoteAddr":"10.20.4.25","user":"ecadmin","app":"webdav","method":"PROPFIND","url":"/remote.php/dav/files/ecadmin/Test/AM-eval-6.5.1.zip","message":"File with name /Test/AM-eval-6.5.1.zip could not be located","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36","version":"26.0.2.1","exception":{"Exception":"Sabre\\DAV\\Exception\\NotFound","Message":"File with name /Test/AM-eval-6.5.1.zip could not be located","Code":0,"Trace":[{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Tree.php","line":78,"function":"getChild","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":971,"function":"getNodeForPath","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":1662,"function":"getPropertiesIteratorForPath","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":1647,"function":"writeMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":346,"function":"generateMultiStatus","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPropFind","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/apps/dav/lib/Server.php","line":366,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/web/service/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/web/service/remote.php","line":172,"args":["/web/service/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/web/service/apps/dav/lib/Connector/Sabre/Directory.php","Line":227,"message":"File with name /Test/AM-eval-6.5.1.zip could not be located","exception":{},"CustomMessage":"File with name /Test/AM-eval-6.5.1.zip could not be located"}}Additional info
No response
The text was updated successfully, but these errors were encountered: