How to get query id for impala queries executed via sqlalchemy

1.4k Views Asked by At

I am querying impala using sqlalchemy which internally uses impyla.

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

engine = create_engine("impala://{host}:{port}/{database}".format(
    host=host,
    port=21051,
    database=schema
))

Session = sessionmaker(bind=engine, autocommit=False)
_session = Session()
result = _session.execute("""SELECT * FROM TABLE1 LIMIT 2""")
data = result.fetchall()
print(data)
_session.close()
Session.close_all()

Now I want to get the query_id than can be used to monitor the progress of this query in Cloudera manager UI.

For example, if I execute the same query using imapal-shell in console.

I get this:

[MYMACHINE:21001] > select * from table1 limit 2;
Query: select * from table1 limit 2
Query submitted at: 2020-08-24 10:28:15 (Coordinator: http://MYMACHINE:25000)
Query progress can be monitored at: http://MYMACHINE:25000/query_plan?query_id=8646e69bf8aa9707:df03d6ff00000000

I need to log this query-id 8646e69bf8aa9707:df03d6ff00000000.

Is there any way I can fetch this using sqlalchemy or any other library?

0

There are 0 best solutions below