impala.error.HiveServer2Error: Failed after retrying 3 times

Question

I use impyla and ibis to connect hive server, but I got the error. I tried the following code:

from impala.dbapi import connect
impcur = connect(host="kudu3", port=10000, database="yingda_test", password=None, user='admin', kerberos_service_name='None').cursor()

The new error came out:

Traceback (most recent call last):
  File "/Users/edy/src/PythonProjects/dt-center-algorithm/test/1.py", line 4, in <module>
    impcur = connect(host="kudu3", port=10000, database="yingda_test", password=None, user='admin', kerberos_service_name='None').cursor()
  File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 129, in cursor
    session = self.service.open_session(user, configuration)
  File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1187, in open_session
    resp = self._rpc('OpenSession', req, True)
  File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1080, in _rpc
    response = self._execute(func_name, request, retry_on_http_error)
  File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1142, in _execute
    .format(self.retries))
impala.error.HiveServer2Error: Failed after retrying 3 times

thrift 0.15.0 thrift-sasl 0.4.3 thriftpy2 0.4.14 pure-sasl 0.6.2 sasl 0.2.1 thrift-sasl 0.4.3 ibis-framework 2.0.0 impyla 0.17.0 python version: 3.7.12 with anaconda

And I have tried ibis-1.3.0 and 2.0 version. Can u guys give some advices? tks a lot

Answer 1

I also met this problem.

My code is:

from impala.dbapi import connect
import psycopg2

conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin', 
                    password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data...
hive_cursor.close()
conn_hive.close()

After my colleagues and I tried, we find it will be successful when we reconnect hive manually.

It means if you want to get different data from the same hive database, you had better close the connection and reconnect hive manually by the following codes:

conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin', 
                    password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data1...
hive_cursor.close()
conn_hive.close()

conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin', 
                    password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data2...
hive_cursor.close()
conn_hive.close()

Finally, my colleagues told me that there was a problem with impala recently.