ibis ImpalaTable to pyspark dataframe

Question

In my case, I need to load impala data to spark(pyspark). Because I want to use FPGrowth of spark mllib.

Data is in kudu and it was made by impala. Connecting to directly kudu on spark was rejected by a relevant department. And I also failed connecting with impala jdbc made by cloudera.
So my last choice is

1. Load data with ibis (https://github.com/ibis-project/ibis)
2. Convert ImpalaTable to spark's Dataframe

But I couldn't find a way.
Do I think wrong?

Answer 1

Previously, this way was not worked for me.
I could get schema of tables, but I couldn't query because of timeout.

And I finally found a problem. My problem caused by firewall.
I opened ports of only master nodes, but I needed to open ports of data nodes.
And now everything is fine.