Fetch Folder from drive for Google Colab

Question

I'm trying to run a deep learning model in jupyter notebook and its taking forever and also karnel dies during training . So i'm trying to run it on Google Colab . I've learned some basics that are available on the internet but its not helping me at all . The model gets it dataset from a module , this link https://github.com/awslabs/handwritten-text-recognition-for-apache-mxnet/blob/master/ocr/utils/iam_dataset.py has the module that extract and preprocess dataset for trining from local computer. I've uploaded the dataset in Gdrive now i want to change the path so that this module finds that 'dataset' folder . I've been stuck on it for 5 days and now i'm clueless .

Answer 1

I will suggest you not to load the dataset from GDrive to colab directly. It increases the dataset loading time.

Google Colab provides some local storage for your work(around 70 GB) that is shown on the upper-right corner below the RAM bar. Bring your dataset to that storage. This is how you can do it:-

import zipfile
from google.colab import drive

zip_ref = zipfile.ZipFile("/content/drive/My Drive/dataset.zip", 'r')
zip_ref.extractall("/content/")
zip_ref.close()

Please note that your entire dataset should be zipped.

It will be more than 20 times faster than the method you are trying...

Format of zipfile.ZipFile() function above:-

zip_ref = zipfile.ZipFile("/content/drive/Zip file location in GDrive", 'r')

If you click the folder icon in the left side in colab interface you should see your dataset there.

You can then access your dataset using the filepath='/content/dataset'