Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple file uploads returns uncatchable B2 Exception too busy when fails #26

Open
PaddyLock opened this issue Feb 14, 2019 · 11 comments

Comments

@PaddyLock
Copy link

PaddyLock commented Feb 14, 2019

I am looping through multiple files and passing to a recursive function to to upload to a B2 bucket like so:

private function upload($client, $file, $fileInfo, $savePath)
    {
        try {
            $client->upload([
                'BucketName' => 'my-bucket-name',
                'FileName' => $file->getId() . '/' . $fileInfo->getFilename(),
                'Body' => fopen($savePath .'/' . $fileInfo->getFilename(), 'r')
            ]);
        } catch (Exception $e) {
            $this->upload($client, $file, $fileInfo, $savePath);
        }
    }

The problem is that with multiple file uploads it is expected behaviour for B2 to occasionally fail. See here for 'Uploading Large Files' and 'Code Structure' - https://www.backblaze.com/b2/docs/uploading.html

The error I get after several successful file uploads looks like this

In ErrorHandler.php line 36: Received error from B2: c002_v0001118_t0038 is too busy

The problem here is firstly that the real error and response code is hidden, and secondly even though I am in a try catch block, it doesn't catch the exception.

My question is how should I handle multiple large file uploads to handle these expected (and other) potential exceptions?

Thanks

@PaddyLock PaddyLock changed the title Multiple file uploads returns uncatchable Exception too busy when fails Multiple file uploads returns uncatchable B2 Exception too busy when fails Feb 14, 2019
@rodber
Copy link

rodber commented Feb 28, 2019

I also got stumbled by this one and I found an explanation on whats going on:
What To Do When You Get a B2 503 (or 500) Server Error
TL:DR Catch the error, do some sleep() then try again. It seems that the SDK doesn't handle this sorcery.

@rodber
Copy link

rodber commented Feb 28, 2019

Please note that the SDK handles the HTTP responses, it is way easier if you add the workaround in this repo.

@wouterbles
Copy link

wouterbles commented Apr 2, 2019

Any reason why this issue was closed? I am also facing these problems and I think adding a sleep() and retry in this package would be a great addition.

@mhetreramesh
Copy link
Member

@wouterbles I considered this has to handle on application level so I closed it previously. I'm currently not using backblaze extensively for any of my projects so never came across.

I saw your comments on the thread. Since you have the tested solution in hand, could you add it to the repo & raise a PR.

@wouterbles
Copy link

I'll see what I can do! I wasn't able to catch the error on application level so I started looking at this package. The change I posted in the other thread was a very crude implementation and should be updated somewhat. When I've done that I'll open a PR for sure.

@rodber
Copy link

rodber commented Apr 2, 2019

I considered this has to handle on application level so I closed it previously.

I believe that since your SDK handles the HTTP client that interacts with Backblaze, is your client which is wrong and not the applications which implement your SDK. Here: https://github.com/gliterd/backblaze-b2/blob/master/src/Http/Client.php#L27 you only handle HTTP 200 and you don't touch any other status code at all. In that method your SDK should handle 503 and whatnot.

I'm currently not using backblaze extensively for any of my projects so never came across.

Totally understood, but in that case I suggest you to drop this repo and suggest to use obregonco/backblaze-b2 which also works with application keys rather than master keys.

@wouterbles
Copy link

Does the repo you suggest also support the https://github.com/gliterd/flysystem-backblaze and https://github.com/gliterd/laravel-backblaze-b2 that depend on this SDK? After taking a quick look at the code I think it would not be a problem.

If this is the case, it might be nice to change the dependency of laravel-flysystem to a more actively maintained one.

@rodber
Copy link

rodber commented Apr 6, 2019

Does the repo you suggest also support the https://github.com/gliterd/flysystem-backblaze and https://github.com/gliterd/laravel-backblaze-b2 that depend on this SDK? After taking a quick look at the code I think it would not be a problem.

We are using it on Chevereto and the only breaking change that I spotted was that obregonco's constructor is slightly different as it supports both master and application keys.

@wouterbles
Copy link

@rodolfoberrios Dit you not have the issue I described in obregonco/backblaze-b2#2?

@rodber
Copy link

rodber commented Apr 6, 2019

@rodolfoberrios Dit you not have the issue I described in obregonco/backblaze-b2#2?

Yes, I did and I fixed that issue the problem is that obregonco hasn't tagged a new release.

@dezertdezine
Copy link

dezertdezine commented Jun 29, 2023

I know this is super old, but if anyone can show me a fix for this I'd be very grateful.

  • UPDATE

I ended using the S3-Compatible API:
https://www.backblaze.com/apidocs/introduction-to-the-s3-compatible-api

That's working perfectly for my use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants