pexpect - capturing stdout of a process started by a process

Question

In order to run some functions, I have a series of consoles I must recursively connect to.

• localhost: aws ssm start-session ... to shell into a bastion instance.
• bastion: call a shell script that starts a ruby on rails container and connects to it
• rails container: here is where I call some functions and then exit

I want to programmatically manage this, and as far as I can tell, pexpect is the best answer.

Expected behavior: pexpect.spawn a process, trigger a series of sendline() commands, capture all stdout from the rails container, exit.

In the aforementioned layout, stdout is not captured from the rails container. I am running a form of the following:

aws_command = "aws ssm start-session ..."
p = pexpect.spawn(aws_command, encoding='utf-8')
logfile = open("logs.txt", 'w')
p.logfile = logfile
p.expect('$')
p.sendline('ls')

# ... other setup commands/env vars ...

p.sendline('this is where the command for a shell script that starts a ruby on rails container is sent')
p.expect('Loading') # this is the final line catpured from the ssm process before the rails console begins
time.sleep(30) # extra wait for good measure
p.sendline('2+2') # should see an output of 4 in the logs
time.sleep(5)
p.sendline('exit')
p.kill(0)

This yields the following in logs.txt (parts redacted):

Starting session with SessionId: ... # initial SSM output

sh-4.2$ # SSM shell

# the ls input and output is successfully captured here.

sh-4.2$ 'ruby on rails container shell script invoked here'

Starting console ... # output from the rails container shell script

Loading ... (Rails ...) # final output captured from the rails container shell script

2+2 # 4 not captured
exit

While it is clear that stdout from the rails container is not being captured, I'm not even sure if the 2+2 input from the sendline() is even reaching the rails container. I've done a test where I've added p.interact() immediately after sending 2+2 (p.interact() gives the user immediate control over the current process). This successfully brings me to the rails console, where 2+2 is entered and 4 is returned. However, I cannot tell what is happening when I don't call p.interact() (which, in final usage, I do not want to ever call p.interact()).

I have also tried piping the rails container shell script command into tee and then catting the file after the exit command is sent. However, this didn't yield any inputs or outputs (which might be a smoking gun).

Main questions:

• is pexpect capable of managing a process-within-a-process like above?
• is output not being captured, or is input not being sent to the rails container?

Curious to see if anyone else has had this issue before. I suspect I will have to inject an intermediate python script to the ssm instance from which I could manage the rails container command, and then fully pipe input and output through the intermediate process to the localhost process.