How to mount bucket in GCE and make it available to R Studio-Server

Question

I have setup a Google Compute Engine (GCE) instance and I want to mount a Google Cloud Bucket to it. Basically, I have uploaded my data to Google Cloud and I want to make it available for use in the R Studio-server I have installed in my instance. It seems my mounting was successful, but I cannot see the data on R (or in the shell).

I want the bucket to be mounted in /home/roberto/remote. I have run chmod 777 /home/roberto/remote and then gcsfuse my-project /home/roberto/remote. I got the following output:

2023/01/28 22:49:01.004683 Start gcsfuse/0.41.12 (Go version go1.18.4) for app "" using mount point: /home/roberto/remote
2023/01/28 22:49:01.022553 Opening GCS connection...
2023/01/28 22:49:01.172583 Mounting file system "my-project"...
2023/01/28 22:49:01.176837 File system has been successfully mounted.

However, I can't see anything inside /home/roberto/remote when I run ls or when I look inside of it from R Studio-server (see image below). What should I do?

UPDATE: I had uploaded my folders to google cloud, but when I uploaded an individual file, it suddenly showed up! This makes me think the issue has something to do with implicit directories. Supposedly, if I run the same command as before with the --implicit-dirs flag that would be enough (something like this: gcsfuse --implicit-dirs my-project /home/roberto/remote). However, this is returning an error message and I am not sure how to deal with it.

Error message:

2023/01/29 01:33:15.428752 Start gcsfuse/0.41.12 (Go version go1.18.4) for app "" using mount point: /home/roberto/remote
2023/01/29 01:33:15.446696 Opening GCS connection...
2023/01/29 01:33:15.548211 Mounting file system "my-project"...
daemonize.Run: readFromProcess: sub-process: mountWithArgs: mountWithConn: Mount: mount: running /usr/bin/fusermount3: exit status 1

Answer 1

Other answers here could be useful depending on your issue. In my case, what solved it was indeed running the command gcsfuse --implicit-dirs my-project /home/roberto/remote. The error I was getting in the edit for my question was due to the fact that I had previously mounted the bucket and was trying to mount it again without unmounting it first (here is the official documentation on how to unmount the bucket). For more details on the importance of the --implicit-dirs flag take a look at the official documentation here. There are very similar solutions using, for instance, the /etc/fstab file. For that, take a look at this discussion in the official github page of gcsfuse.

Answer 2

Try to edit the VM Cloud API access scopes of Storage to Full.

Follow the steps below:

1. Click/Select the VM instance
2. Stop the VM instance, then edit the VM instance.
3. Scroll down to Access scopes and select "Set access for each API"
4. Change the Storage from Read Only to Full.
5. Save and start your VM instance.

Then SSH to your VM instance and try to ls /home/roberto/remote

Answer 3

Try running gcsfuse mount command with debug flags which will help in knowing why the mount failed. Eg: gcsfuse --implicit-dirs --debug_fuse --debug_gcs --debug_http my-project /home/roberto/remote