os.open with non-utf8 characters in file name

46 Views Asked by At

I am trying to copy a file from a source NFS volume to destination NFS volume.

The file name has non-utf8 character and I am using bytes to open/read/write.

Using os.open, the path opens fine on the source , but gives invalid argument error on destination.

Below is the minimal problem example

    >>> import os
    >>> x = b'/x/en/local/noarch/agnostic/docs/FSques\x8awithrepl.doc'
    >>> os.open(x, os.O_RDONLY)
    3
    >>> fd = os.open(x, os.O_RDONLY)
    >>> os.path.getsize(fd)
    37888
    >>>
    >>> y=b'/mnt/x/dest/WAFSquestionnai\x8awithreplies.doc'
    >>> os.open(y, os.O_RDONLY)
    Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
    OSError: [Errno 22] Invalid argument: 
        b'/mnt/x/dest/WAFSquestionnai\x8awithreplies.doc'
    
    >>> import cchardet as ct
    >>> ct.detect(x)
    {'encoding': 'ISO-8859-3', 'confidence': 0.7991858124732971}
    >>>
    >>> ct.detect(y)
    {'encoding': 'ISO-8859-3', 'confidence': 0.8912176489830017}
    >>>


    >>>
    >>> import sys
    >>> sys.getdefaultencoding()
    'utf-8'
    >>>

Why does os.open pass on one and fail on the other? Shouldn't I at least get a FileNotFound error on the destination path?

0

There are 0 best solutions below