better (any) error handling

[fenio]

Kyoo's version

latest

What happened?

While meilisearch component is down and other components have no way to connect to it some other components they failed which is fine. But error handling should be a bit better probably.

This is how kyoo-back is trying to connect to meilisearch:

[☸ lab:media] [ main]
❯ homelab/cluster/apps/o11y/flux-webui k logs kyoo-back-7c769788b8-cpb76
Defaulted container "app" out of: app, 00-init-db (init), 01-migrations (init)
[08:55:33 FTL                           (4161753916)] Unhandled exception
Meilisearch.MeilisearchCommunicationError: CommunicationError
 ---> System.Net.Http.HttpRequestException: Permission denied (kyoo-meilisearch:7700)
 ---> System.Net.Sockets.SocketException (13): Permission denied
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Meilisearch.MeilisearchMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Meilisearch.MeilisearchMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Meilisearch.Extensions.HttpExtensions.PostJsonCustomAsync[T](HttpClient client, String uri, T body, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Meilisearch.MeilisearchClient.CreateIndexAsync(String uid, String primaryKey, CancellationToken cancellationToken)
   at Kyoo.Meiliseach.MeilisearchModule._CreateIndex(MeilisearchClient client, String index, Boolean hasKind) in /kyoo/src/Kyoo.Meilisearch/MeilisearchModule.cs:line 140
   at Kyoo.Meiliseach.MeilisearchModule.Initialize(IServiceProvider provider) in /kyoo/src/Kyoo.Meilisearch/MeilisearchModule.cs:line 126
   at Program.<Main>$(String[] args) in /kyoo/src/Kyoo.Core/Program.cs:line 104
   at Program.<Main>$(String[] args) in /kyoo/src/Kyoo.Core/Program.cs:line 104
   at Program.<Main>(String[] args)
Unhandled exception. Meilisearch.MeilisearchCommunicationError: CommunicationError
 ---> System.Net.Http.HttpRequestException: Permission denied (kyoo-meilisearch:7700)
 ---> System.Net.Sockets.SocketException (13): Permission denied
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Meilisearch.MeilisearchMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Meilisearch.MeilisearchMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Meilisearch.Extensions.HttpExtensions.PostJsonCustomAsync[T](HttpClient client, String uri, T body, JsonSerializerOptions options, CancellationToken cancellationToken)
   at Meilisearch.MeilisearchClient.CreateIndexAsync(String uid, String primaryKey, CancellationToken cancellationToken)
   at Kyoo.Meiliseach.MeilisearchModule._CreateIndex(MeilisearchClient client, String index, Boolean hasKind) in /kyoo/src/Kyoo.Meilisearch/MeilisearchModule.cs:line 140
   at Kyoo.Meiliseach.MeilisearchModule.Initialize(IServiceProvider provider) in /kyoo/src/Kyoo.Meilisearch/MeilisearchModule.cs:line 126
   at Program.<Main>$(String[] args) in /kyoo/src/Kyoo.Core/Program.cs:line 104
   at Program.<Main>$(String[] args) in /kyoo/src/Kyoo.Core/Program.cs:line 104
   at Program.<Main>(String[] args)

And since it fails then scanner can't connect to back:

[☸ lab:media] [ main]
❯ homelab/cluster/apps/o11y/flux-webui k logs kyoo-scanner-8bd8c9ccb-vgxbr
INFO:scanner.scanner:Starting the scan. It can take some times...
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1025, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1122, in create_connection
raise exceptions[0]
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1104, in create_connection
sock = await self._connect_sock(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1007, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 651, in sock_connect
return await fut
^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 659, in _sock_connect
sock.connect(address)
PermissionError: [Errno 1] Operation not permitted

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/app/scanner/main.py", line 6, in
asyncio.run(scanner.main())
File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/app/scanner/init.py", line 20, in main
await asyncio.gather(
File "/app/scanner/init.py", line 18, in scan_all
await scan(path, publisher, client, remove_deleted=True)
File "/app/scanner/scanner.py", line 32, in scan
registered = await client.get_registered_paths()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/providers/kyoo_client.py", line 39, in get_registered_paths
async with self.client.get(
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1197, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 581, in _request
conn = await self._connector.connect(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 544, in connect
proto = await self._create_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 944, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1257, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1226, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/connector.py", line 1033, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host kyoo-back:5000 ssl:default [Operation not permitted]