Can't seem to pass the contents of a previous read into a subsequent subprocess.Popen cmd input

Question

This is in Windows 10 Linux (wsl) running Python 3.8.5 and 3.90rc1. Not using Windows native.

I'm os.walking through a directory hierarchy consisting of a number of recorded Albums listed by Album Title first, and within that title there is a song list in track order. I'm trying to use the full name of the song and output it to the Popen process which opens an external program /usr/bin/flac -ts which scans the single file and reports (usually through stderr) if it is in any way defective. The tracks are all .FLAC files. I've tried a variety of ways to accomplish this using sample techniques from here and other sites. Not matter what, though, there is always some problem, like the flac program never gets the filename, or causes Python to fail from (seemingly) some strange error. Here's an excerpt from my current version:

    for path, dirs, files in os.walk(StartScanDir):
        i=0
        runout=[ ]
#       runout[i]=path
        runout.append(path)
        print (runout[i])
#       print (path)
        for f in files:
            if not f.endswith('.flac'):
                continue
            i=i+1
#           print(i)
            runout.append(f)
            print (runout[i]) # > FLAClistOut
#           print ("f")
            pipe=Popen("/usr/bin/flac","-ts", stdout=PIPE, stderr=PIPE)
            text = pipe.communicate()[0]
#           text = pipe.communicate(b"f")
            print(text)
            print (stdout)
            print (stderr)

This one raises the exception from /usr/lib/python3.9/subprocess.py 'bufsize must be an integer'. I sometimes get that, but in other variations of my coding, I get a complaint about the input wanting binary and getting a str. It's the same behavior on either version of Python.

Can anyone spot what I might be doing wrong and/or suggest a better way? There are some lines commented out that were just for testing.

Thanks.

I've updated and commented the code so what's going on is clearer.

    os.chdir(StartScanDir)  #start at the top of the directory hierarchy
#   print(os.getcwd())
    for path, dirs, files in os.walk(StartScanDir):
        i=0
        runout=[ ]  #clear logging string array
        runout.append(path) #add album title
        print (runout[i])
        for f in files:
            if not f.endswith('.flac'):
                continue
            i=i+1
            runout.append(f)    #add individual song titles
            cmd = ['/usr/bin/flac','-t','-s']
            print (runout[i]) # > FLAClistOut
            p=subprocess.Popen(cmd, stdout = subprocess.PIPE, stdin = subprocess.PIPE)
            out,err=p.communicate('f' ) 
            print (out)
            print (err)

This produces the error:
Traceback (most recent call last):
  File "/mnt/x/incoming/py-work/fixflac.py", line 35, in <module>
    out,err=p.communicate('f' )
  File "/usr/lib/python3.9/subprocess.py", line 1130, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.9/subprocess.py", line 1957, in _communicate
    input_view = memoryview(self._input)
TypeError: memoryview: a bytes-like object is required, not 'str'

Answer 1

    os.chdir(StartScanDir)  #start at the top of the directory hierarchy
#   print(os.getcwd())  #1
    for path, dirs, files in os.walk(StartScanDir):
        i=0
        runout=[ ]  #clear logging string array
        runout.append(path) #add album title
        print (runout[i])   #2 album title
        os.chdir(runout[i])  #cd to album
        for f in files:
            if not f.endswith('.flac'):
                continue
            i=i+1
            runout.append(f)    #add individual song titles
            print (runout[i])   #3 > FLAClistOut
            print(os.getcwd())  #4  confirm we're in the right dir
            
            p=subprocess.Popen(["flac","-ts",'f'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
#           p.stdout.write('f')     #produces error io.Unsupported operation
#           p.stdout.close()        #closes io too soon
            exit_code = p.wait()    #doesn't make any diff
            print(exit_code)        # always 1 since flac reports error reading everytime
            out_bytes =p.stdout.read()
            out_str = out_bytes.decode('utf-8')
            err_bytes =p.stderr.read()
            err_str = err_bytes.decode('utf-8')
#           print (out_str)     #never seen anything here
            print (err_str)     #error info comes out here

This is what I have now and extra comments added. Still cannot get the actual filename sent to flac. It errors saying it can't read the filename or the file doesn't exist. This is a typical directory name that I cd into: /mnt/y/FLAC/flac-oldies-albums/Various Artists/#1 Radio Hits of the 60's And the format of the filename in that directory: 01 - Righteous Brothers - (You're My) Soul and Inspiration.flac

I think I've tried all the suggestions and what you see above is the result. I'm stuck.

I can copy a song name and on the bash command line manually enter or paste the name of a file as 'flac -ts 4 Seasons - Hurt Yourself.flac' and it works fine, as well as from a bash script I wrote with find and -exec. Just can't get the name right in python...

Answer 2

out, err = p.communicate('f')

can be replaced with

out, err = p.communicate(b'f')

as long as you only want b'f' to be the only input. The input must be a bytes array, since a str could be encoded as ISO-8859-1, UTF8, UCS16, etc. Python doesn't know which encoding flac will be expecting.

However, p.communicate() does several different steps, and it may be clearer to handle each step separately. For example, the err variable will always be None because your call to Popen() does not include stderr=subprocess.PIPE. Instead of p.communicate(), you could write:

p.stdout.write(b'f')
p.stdout.close()
out = p.stdin.read()
exit_code = p.wait()

The error that you encountered in the first example is because all the flac arguments must be in the first argument to subprocess.Popen(). If you use a str as the first argument, you can't pass any other arguments to flac. Popen("/usr/bin/flac", "-ts", stdout=PIPE, stderr=PIPE) passes "-ts" as the second argument, which is the bufsize argument, and it expects an integer.

I haven't used flac before, but I believe you will need to pass the filename as an argument. For example:

subprocess.Popen(["flac", "-ts", f], stdout=subprocess.PIPE, stdin=subprocess.PIPE)

I don't think you will need to send data to flac, so you can just call out_bytes = p.stdout.read(). It will return a byte array. If you want to convert it to a str, you can call out_str = out_bytes.decode('utf-8'). flac is probably just outputing ascii, but UTF-8 is a superset of ascii, and it is the most common encoding.