Use python to execute shell command and redirect the output to a variable

Question

I am trying to do 2 things with Python-

1. Execute a command at the terminal
2. Save the output of the above command to a variable.

This is my code till now -

import subprocess
ip_address = "http://ipwho.is/104.123.204.11"
query_reply = subprocess.run(["curl" , ip_address])
print("The exit code was: %d" % query_reply.returncode)

The query_reply only saves the return code as I am using query_reply.returncode. Whereas I want to save the general output of the command curl http://ipwho.is/104.123.204.11 to a variable.

This output is a dict-like structure -

{"ip":"104.123.204.11","success":true,"type":"IPv4","continent":"North America","continent_code":"NA","country":"United States","country_code":"US","region":"California","region_code":"CA","city":"San Jose","latitude":37.3382082,"longitude":-121.8863286,"is_eu":false,"postal":"95113","calling_code":"1","capital":"Washington D.C.","borders":"CA,MX","flag":{"img":"https:\/\/cdn.ipwhois.io\/flags\/us.svg","emoji":"\ud83c\uddfa\ud83c\uddf8","emoji_unicode":"U+1F1FA U+1F1F8"},"connection":{"asn":16625,"org":"Akamai Technologies, Inc.","isp":"Akamai Technologies, Inc.","domain":"gwu.edu"},"timezone":{"id":"America\/Los_Angeles","abbr":"PDT","is_dst":true,"offset":-25200,"utc":"-07:00","current_time":"2022-07-25T11:26:47-07:00"}}

The final goal is to access the fields like the region, city etc. inside the above structure. What is the general process to approach this sort of problem?

Answer 1

there is an arg to get output

import subprocess
import json

r = subprocess.run(["curl" , "http://ipwho.is/104.123.204.11"], capture_output=True)
djson = json.loads(r.stdout.decode())
djson["region"], djson["city"]

or better just querying it

import requests

with requests.get("http://ipwho.is/104.123.204.11") as response:
    djson = response.json()
djson["region"], djson["city"]

Answer 2

You can use subprocess.check_output, which returns the subprocess output as a string:

import subprocess
import json

raw = subprocess.check_output(["curl", "http://ipwho.is/104.123.204.11"])
data = json.loads(raw)
print(data)

Although this works, it's rarely if ever a good idea to shell out to curl to retrieve a URL in Python. Instead, use the excellent requests library to get the URL directly:

import requests

req = requests.get("http://ipwho.is/104.123.204.11")
req.raise_for_status()
data = req.json()
print(data)

This is simpler, handles errors better (raise_for_status will raise an easy-to-understand exception if the server returns an error code), faster, and does not depend on the curl program being present.

(Note: although similar to the other answer, the code snippets in this answer should work directly without modification).