Tensorflow-Hub Modules not loading

Question

I am trying to use the universal sentence encoder 4, It works on my desktop at home but I am not home so I am having to re-set everything up. I am using this all in a virtual environment. I have changed nothing in the code. I am on hotel wifi I believe that could be an issue but I also tried it on my mobile hotspot. It typically just freezes as I start the program as if it is trying to initialize the module but nothing happens and it eventually spits out an error.

I have tried using a mobile hotspot to get around this problem and that did not work. I am okay with any kind of work around that doesn't envolve loading it from the internet provided, it would work on an aws instance eventually.

Here is the code I am trying to run:

import tensorflow_hub as hub
embed =  hub.load(
            'https://tfhub.dev/google/universal-sentence-encoder/4')
print('if this prints it actually loaded')

Here is the error: l/OneDrive/Desktop/pythontest/Testing environ/test.py" WARNING:absl:Deleting lock file C:\Users\reill\AppData\Local\Temp\tfhub_modules\063d866c06683311b44b4992fd46003be952409c.lock due to inactivity. Traceback (most recent call last): File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\resolver.py", line 192, in download_and_uncompress file_utils.extract_tarfile_to_destination( File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\file_utils.py", line 52, in extract_tarfile_to_destination
extract_file(tgz, tarinfo, abs_target_path, log_function=log_function) File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\file_utils.py", line 35, in extract_file buf = src.read(buffer_size) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\reill\AppData\Local\Programs\Python\Python311\Lib\tarfile.py", line 696, in readinto buf = self.read(len(b)) ^^^^^^^^^^^^^^^^^ File "C:\Users\reill\AppData\Local\Programs\Python\Python311\Lib\tarfile.py", line 687, in read raise ReadError("unexpected end of data") tarfile.ReadError: unexpected end of data

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\test.py", line 2, in embed = hub.load( ^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\module_v2.py", line 93, in load module_path = resolve(handle) ^^^^^^^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\module_v2.py", line 48, in resolve return registry.resolver(handle) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\registry.py", line 49, in call return impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\compressed_module_resolver.py", line 67, in call return resolver.atomic_download(handle, download, module_dir, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\resolver.py", line 421, in atomic_download download_fn(handle, tmp_dir) File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\compressed_module_resolver.py", line 64, in download return resolver.DownloadManager(handle).download_and_uncompress( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\reill\OneDrive\Desktop\pythontest\Testing environ\env\Lib\site-packages\tensorflow_hub\resolver.py", line 200, in download_and_uncompress raise IOError("%s does not appear to be a valid module." % self._url) OSError: https://tfhub.dev/google/universal-sentence-encoder/4 does not appear to be a valid module.

Answer 1

I am able to download the TF Hub universal-sentence-encoder/4 module in Google Colab using Python 3.10 and in conda environment using Python 3.11.
It seems this OSError could be due to network issue as you mentioned. There are some common issues in TF Hub as below -
1. "EOF occurred in violation of protocol" - when the installed python version does not support the TLS requirements of the server hosting the module
2. "cannot verify tfhub.dev's certificate" - which is likely to be generated if something on the network is trying to act as the dev gTLD.
3. Failures to write to the cache directory /tmp/tfhub_modules

You can try to manually download this module by simulating the protocol of attaching ?tf-hub-format=compressed to the URL to download a tar compressed file that has to be manually decompressed into a local file

# Create a folder for the TF hub module.
!mkdir /tmp/moduleA

#Download the module, and uncompress it to the destination folder. You might want to do this manually.
!curl -L "https://tfhub.dev/google/universal-sentence-encoder/4?tf-hub-format=compressed" | tar -zxvC /tmp/moduleA

#Test to make sure it works.
import tensorflow_hub as hub
hub.Module("/tmp/moduleA")

or

import tensorflow_hub as hub
embed =  hub.load(
            'https://tfhub.dev/google/universal-sentence-encoder/4?tf-hub-format=compressed')
print('if this prints it actually loaded')

Please have a look at this similar issue for your reference.