You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the issue
I expected that mounting gcsfuse on a directory with existing files would result in those files being synced to GCS. However, the files are not present in the dir when gcsfuse is started (my bucket is empty).
To Collect more Debug logs
Steps to reproduce the behavior:
Please make sure you have no other security, monitoring, background processes which can offend the FUSE process running. Possibly reproduce under a fresh/clean installation.
Please rerun with --debug_fuse --debug_fs --debug_gcs --debug_http --foreground as additional flags to enable debug logs.
Monitor the logs and please capture screenshots or copy the relevant logs to a file (can use --log-format and --log-file as well).
Attach the screenshot or the logs file to the bug report here.
If you're using gcsfuse with any other library/tool/process please list out the steps you took to reproduce the issue.
A few seconds later, the file is "deleted" (ls doesn't show it if gcsfuse is running, but it does if gcsfuse is stopped).
Logs:
{"name":"root","levelname":"INFO","severity":"INFO","message":"Start gcsfuse/1.0.1 (Go version go1.20.5) for app \"\" using mount point: /home/user/mount-folder/\n","timestampSeconds":1692055966,"timestampNanos":540149851}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Opening GCS connection...\n","timestampSeconds":1692055966,"timestampNanos":540225421}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a mount at \"/home/user/mount-folder/\"\n","timestampSeconds":1692055966,"timestampNanos":543620609}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a new server...\n","timestampSeconds":1692055966,"timestampNanos":543676400}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Set up root directory for bucket $BUCKET\n","timestampSeconds":1692055966,"timestampNanos":543692277}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x0: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055966,"timestampNanos":543706580}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x0: -\u003e ListObjects(\"\") (164.540755ms): OK\n","timestampSeconds":1692055966,"timestampNanos":708261265}
{"name":"root","levelname":"INFO","severity":"INFO","message":"Mounting file system \"$BUCKET\"...\n","timestampSeconds":1692055966,"timestampNanos":708410485}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Beginning the mounting kickoff process\n","timestampSeconds":1692055966,"timestampNanos":708494846}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Parsing fuse file descriptor\n","timestampSeconds":1692055966,"timestampNanos":708536589}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Preparing for direct mounting\n","timestampSeconds":1692055966,"timestampNanos":708543068}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully opened the /dev/fuse in blocking mode\n","timestampSeconds":1692055966,"timestampNanos":708572955}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Starting the unix mounting\n","timestampSeconds":1692055966,"timestampNanos":708609568}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Directmount failed. Trying fallback.\n","timestampSeconds":1692055966,"timestampNanos":708627188}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating a socket pair\n","timestampSeconds":1692055966,"timestampNanos":708695464}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating files to wrap the sockets\n","timestampSeconds":1692055966,"timestampNanos":708727252}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Starting fusermount/os mount\n","timestampSeconds":1692055966,"timestampNanos":708742879}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Wrapping socket pair in a connection\n","timestampSeconds":1692055966,"timestampNanos":710532113}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Checking that we have a unix domain socket\n","timestampSeconds":1692055966,"timestampNanos":710592272}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Read a message from socket\n","timestampSeconds":1692055966,"timestampNanos":710604488}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully read the socket message.\n","timestampSeconds":1692055966,"timestampNanos":710618163}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Converting FD into os.File\n","timestampSeconds":1692055966,"timestampNanos":710628138}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Completed the mounting kickoff process\n","timestampSeconds":1692055966,"timestampNanos":710660210}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating a connection object\n","timestampSeconds":1692055966,"timestampNanos":710673050}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000002 connection.go:416] \u003c- init\n","timestampSeconds":1692055966,"timestampNanos":710996856}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000002 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055966,"timestampNanos":711032812}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Successfully created the connection\n","timestampSeconds":1692055966,"timestampNanos":711051389}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Waiting for mounting process to complete\n","timestampSeconds":1692055966,"timestampNanos":711066841}
{"name":"root","levelname":"INFO","severity":"INFO","message":"File system has been successfully mounted.\n","timestampSeconds":1692055966,"timestampNanos":711083128}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000004 connection.go:416] \u003c- GetInodeAttributes (inode 1, PID 1762)\n","timestampSeconds":1692055973,"timestampNanos":500172064}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000004 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":500480409}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000006 connection.go:416] \u003c- GetXattr (inode 1, name \"security.selinux\", PID 27132, name security.selinux)\n","timestampSeconds":1692055973,"timestampNanos":782008471}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000006 connection.go:500] -\u003e Error: \"function not implemented\"\n","timestampSeconds":1692055973,"timestampNanos":782166805}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000008 connection.go:416] \u003c- OpenDir (inode 1, PID 27132)\n","timestampSeconds":1692055973,"timestampNanos":782590176}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000008 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":782714333}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000a connection.go:416] \u003c- ReadDir (inode 1, PID 27132)\n","timestampSeconds":1692055973,"timestampNanos":782837321}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x1: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055973,"timestampNanos":782914515}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x1: -\u003e ListObjects(\"\") (34.895007ms): OK\n","timestampSeconds":1692055973,"timestampNanos":817814151}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000a connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":817888715}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000c connection.go:416] \u003c- ReleaseDirHandle (PID 0)\n","timestampSeconds":1692055973,"timestampNanos":818017487}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000c connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055973,"timestampNanos":818134495}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000e connection.go:416] \u003c- OpenDir (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":822768898}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x0000000e connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":823035572}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000010 connection.go:416] \u003c- GetInodeAttributes (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":823130135}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000010 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":823274480}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000012 connection.go:416] \u003c- ReadDir (inode 1, PID 27135)\n","timestampSeconds":1692055974,"timestampNanos":823364910}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x2: \u003c- ListObjects(\"\")\n","timestampSeconds":1692055974,"timestampNanos":823424255}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req 0x2: -\u003e ListObjects(\"\") (43.3444ms): OK\n","timestampSeconds":1692055974,"timestampNanos":866772673}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000012 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":866882564}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000014 connection.go:416] \u003c- ReleaseDirHandle (PID 0)\n","timestampSeconds":1692055974,"timestampNanos":866983891}
{"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Op 0x00000014 connection.go:498] -\u003e OK ()\n","timestampSeconds":1692055974,"timestampNanos":867057195}
System (please complete the following information):
OS: 22.04.1-Ubuntu
Platform: GCE VM
Version: gcsfuse version 1.0.1
Additional context
Add any other context about the problem here.
SLO:
24 hrs to respond and 7 days to close the issue.
The text was updated successfully, but these errors were encountered:
By default, fuse does not allow mounting a bucket in a non-empty directory. This is because it can lead to data corruption if the files in the directory already exist and have the same names as the files in the bucket.
However, you can use the -o nonempty option to force fuse to mount the bucket in a non-empty directory. This option tells fuse to overwrite any existing files in the directory with the corresponding files from the bucket.
I think this is default behaviour of fuse only and not related to gcsfuse. Still I'll get back to you if we'll consider it as feature request.
Describe the issue
I expected that mounting gcsfuse on a directory with existing files would result in those files being synced to GCS. However, the files are not present in the dir when gcsfuse is started (my bucket is empty).
To Collect more Debug logs
Steps to reproduce the behavior:
A few seconds later, the file is "deleted" (
ls
doesn't show it if gcsfuse is running, but it does if gcsfuse is stopped).Logs:
System (please complete the following information):
Additional context
Add any other context about the problem here.
SLO:
24 hrs to respond and 7 days to close the issue.
The text was updated successfully, but these errors were encountered: