Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nodes with local cache fail if "Require Review" step exists in flow #1080

Open
quadcom opened this issue Sep 16, 2024 · 0 comments
Open

Nodes with local cache fail if "Require Review" step exists in flow #1080

quadcom opened this issue Sep 16, 2024 · 0 comments

Comments

@quadcom
Copy link

quadcom commented Sep 16, 2024

v2.25.01 Ubuntu NVENC Docker - Server with local node
v2.25.01 Win10 NVENC - Remote node

I followed the post regarding a request for local node cache (#945) and set up a Windows node fed from a Linux server.

The local node has path mapping set up and can access the same source files.

The path mapping also has a cache mapping but points the local side to a local folder.

Flow steps operate correctly on the node except for the last step in my flow. If a "Require Review" step precedes the "Replace Original File," the flow will be paused until the review is completed. Once completed, the task is assigned to the next available node to complete the remaining steps in the flow. The problem is if the task is assigned to a node other than the original node, the task will fail as the cache will not be found. Assigning the correct node cannot be guaranteed, so the potential for failure is high.

A local cache is crucial to increasing processing speed and reducing network traffic. However, a flag needs to be placed on the Staged Files listed indicating which node the task should be assigned to once the review process is completed. Either that or each node should have its own staged files listing tasks that only that node looks after.

I would also note that while a remote node (with a local cache) is processing a file, nothing is listed in the staged cache browser. As I understood the situation, I expected this as the server has no connection to the remote cache location. But the server could be aware of the remote cache contents if path mapping was used differently. If the path mapping simply connected both cache folders across the network and all of the systems have access to RW the files on that share, both the server and all of the the nodes would know where to go to find the temporary cache files of any node and any node could be tasked with moving files from cache to the original location or any other location as defined in the flow steps.

*Via a mounted volume - Mount the cache folder on the server to the remote node. The paths connect the server cache to the remote cache

    {
      "server": "/temp",
      "node": "/mnt/remote_temp"
    }
  ],

*Via UNC - Provide the server (and other nodes) a direct path to where the cache files are located on that specific node

    {
      "server": "/temp",
      "node": "\\\\192.x.x.x\\temp"
    }
  ],

I think the Tdarr_Node_Config.json file should have another VAR added for the local cache location that does not use UNC paths, as this can cause issues on Windows systems.

My brain is now fried after troubleshooting getting a remote node across different platforms working -with the addition of using a local cache, which is somewhat of a new feature.

If there's anything I can provide regarding log files, I can try to pull them together. But I hope that I've explained the problem/solution clearly enough to avoid that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant