Only half of the videos are actually downloading, the other half are failing to. #150
Comments
|
I'm not sure about the first error where it was unauthorised but the second one means UniFi protect returned a blank file when you requested to download an event. It has always done this occasionally but I've been seeing it more often with recent updates. I'm not really sure what else we can do except retry which the tool already does |
|
Might be related to other pull requests that were merged which fixed auths? |
|
Will it re-attempt the abandoned events eventually? |
|
It will retry in 10 minutes by default I believe |
UniFi OS has a really strict rate limit for auth. It returns a 401 for unauthorized and rate limit issues. You need to make sure cached sessions are being used so it is not trying to re-auth each time, especially if the API client connects/creates a new session more than once every 30 minutes. |
|
Regarding empty downloads, https://github.com/AngellusMortis/pyunifiprotect/pull/397 might help resolve it. Either way, it’s how it’s done on the new versions of UniFi Protect which could mean the existing method will be dropped at some point. |
@AngellusMortis Thanks for letting me know about this. Since I am only using a single instance of @cyberpower678 Your pr looks promising, please let me know when it is merged and I can add a different downloader as an experimental option that can be enabled to see if it improves this. How bad is the +/- a few seconds on the generated clips? It occassionally happens with the current method but when it does occur almost always its longer than requested. I think ive only seen a handful of short events get returned in my setup. |
|
I’m just waiting on him to approve or request changes. Hopefully soon. |
|
@ep1cman I don't suppose you'd be willing to fashion up a Docker tag that pulls my fork of the pyunifiprotect library. Aside from my changes, it should be up to date otherwise. Would like to really test the unifi-protect-backup with this patch. |
|
You could do docker exec to get a shell into the container and update the version of pyunifiprotect inside to be yours, then just restart the container |
|
Won't work on TrueNAS. It will just redeploy the image/tag. |
|
Ok I'll see what I can do |
|
Ok there should now be a 'experimental' docker tag thats added the new experimental downloader using this. You will need to set the I suspect you might hit the bug beign discussed in #151, let me know if you do. I will work on that next |
|
When trying to run the experimental tag and env variable I get the following upon spinning the container up:
I tried to force it to linux/amd64 and received:
|
|
Whoops, forgot to also build x86. I'll sort this out later today |
|
Let me know when this is fixed, I have the container ready to deploy. |
|
x86 build published |
|
experimental downloader does not appear to work on |
|
@AngellusMortis We seem to be hitting the 401 issue you mentioned. I've spent some time looking at how you've used Do you have any advice? |
|
I have run the Pyunifiprotect library on my local machine using a test script and it's communicating with my UNVR just fine. No 401 issues. prepare/download is also working as expected. So it's highly likely this is an issue with the unifi-protect-backup environment. |
|
Make sure you are defining cache_dir when instantiating ProtectApiClient, it doesn't seem like you are. |
|
@cyberpower678, I managed to track down the cause of the 401 issues, which you can see explained here: https://github.com/AngellusMortis/pyunifiprotect/issues/401 Still working on a work around |
|
It would stand to reason that once the ProtectAPIClient has successfully connected, the session will remain persistent while the container is active? If that is the case, maybe just running the protect_client.update() command repeatedly, while backing off might be the most effective workaround, until it gets a session going? |
|
That is what I had thought I initially but it turns out there is an is_authenticated method that checks if the token from the cookie has been set and if it hasn't (because UniFi now send out a cookie entry python counts as invalid currently) it considers it not authenticated and redoes the authentication on every request which quickly starts getting 401 responses as unfi rate limits the authentication requests |
|
If you you know how to fix and want to push a fix to my fork which should propagate to the experimental, presumably, I'll approve the pull and fix the build issue checks. |
|
Please try the latest |
|
Still getting a 401 2024-06-08 06:40:07 [ DEBUG ] unifi_protect_backup.missing_event_checker : Running check for missing events... |
|
Seems to me, it's using the site package and not your fork. |
|
my bad, forgot to point it to the correct brach. Hopefully now its fixed! |
|
Now we're ripping. No download errors either. |
|
Decided to nuke the sqlite DB so it can start from scratch. Not one single error right now. No blank downloads. Paradise. Now we just need both of our fixes to make their way to pyunifiprotect repo. |
|
I am still seeing a few errors when using the experimental downloader: I am not sure if this is from the previous que that was having trouble before (it stacked up pretty high). It looks like by in large however the vast majority of the downloads work (all but this one really). It may be a different issue all together seeing as it's a 500 error now? |
|
My experience on the matter is that the downloader is running so fast the UDM occasionally doesn’t keep up hence the 500 meaning an internal server error happened. Simply reattempting them by restarting the queue should clear those out and pull any still missing. My queue is sitting at 0 missing events. |
|
it killed it after 10 tries and just ignored it. For whatever reason it was just that one video with the problem. Its been smooth sailing the past 45min. no errors. |
|
It should retry immediately a few times, and then fall back and try again when the missing event checker fires. It will do this upto 5 or 10 times (I don't recall exactly) and if all of those attempts (25, or 50 in total) it will just give up on the event and ignore it. Some times I've noticed UniFi protect will just fail to record an event. You can try open them in the WebUI and they just don't play |
Description
Getting sporadic and random download failures. About 50% hit or miss when downloading videos.
What I Did
I restarted the UNVR and the docker and force pulled a new image.
The text was updated successfully, but these errors were encountered: