Debugging filesystem operations

[rkhoury]

Hello,

I'd like to know what's the easiest way to debug filesystem operations? I'm not familiar with Rust but am happy to attempt to debug a problem I'm having, I just need a few pointers if possible.

I have a very unique situation where I'm running ripgrep in Windows over a custom filesystem layer that makes use of ntfs reparse points to dynamically instantiate files(download from external location) only as they're required - at 'open' time is the most likely case. There's an interesting scenario where I can successfully run ripgrep from the cmd prompt just looking at certain file extensions that I know are already instanced locally, but if I do the same thing via FZF by setting FZF_COMMAND_DEFAULT to my ripgrep command it appears to result in more files of different extensions being read(downloaded) somehow.
Assume I'm running the command rg --type=rust.
Running with --debug shows that these extra files that get pulled in to my filesystem result in the following log message:

path/to/file.bin: This file is currently not available for use on this computer. (os error 4350)

... which I plan use as the place to start debugging if possible. This error message has nothing to do with ripgrep, by the way, so don't worry about this being an error (that's an issue on my side and I'm attempting to exploit it for debugging this issue).
Also to note, this message is surrounded by other DEBUG messages that show other files being successfully ignored:

DEBUG|ignore::walk|crates\ignore\src\walk.rs:1730: ignoring path/to/otherfile.cpp: Ignore(IgnoreMatch(Types(Glob(UnmatchedIgnore))))

So I'm trying to figure out exactly why running rg through fzf is causing this FS layer to 'instance' extra files with extensions that aren't .rs, and I'm hoping there's a nice way to perhaps print out or breakpoint these failing FS operations to know exactly what operation was run and why running it under fzf changes things? Any help you can offer would be great!

Many thanks.

[rkhoury]

OK! I figured out the offending bit of code. It looks like the check for path_equals({d}ent, stdout) is actually creating an open handle for the directory entry, and this check is done before the check to see if a path should be ignored. So I rearranged this in two places in walk.rs and now ripgrep from within fzf appears to operate as expected. Is this a valid solution (should I create a pull request for this change)?

[BurntSushi]

Interesting. I do believe that is a valid re-ordering. It's probably just a band-aide over the problem, since I'm guessing the troublesome files aren't being troublesome purely because they are ignored first. This somewhat means that any code in the future should do an ignore check before actually touching or inspecting the file itself. Regardless, running the ignore check first is probably the smarter thing to do, since it's likely to be less expensive than the stdout handle check, which requires a stat call to the OS. (And especially so in your case, where the stat call seems to result in a network download.)

Your proposed PR sounds great. Could you also add a comment explaining why the ordering is important on Windows? Thanks!

[rkhoury]

I've added #1660
It's fairly trivial so hopefully it doesn't conflict with any other changes in-flight.