Flink didn't write to hbase

Question

I'm encountering an issue while writing to an HBase table using Apache Flink. I've successfully configured reading and writing from Kafka as well as reading RowKey from HBase. However, when attempting to write to HBase, only the RowKey is being written, while the columns contain null values. Below is the table creation and insertion code I'm tried two ways one is sql and 2nd is TableApi:

tEnv.executeSql("CREATE TABLE hTable (\n" +
    " rowkey INT,\n" +
    " persona_data ROW<age INT, name STRING>,\n" +
    " PRIMARY KEY (rowkey) NOT ENFORCED\n" +
    ") WITH (\n" +
    " 'connector' = 'hbase-2.2',\n" +
    " 'table-name' = 'test1',\n" +
    " 'zookeeper.quorum' = 'This line is deleted due to privacy'\n" +
    ");");

tEnv.executeSql("insert into hTable select 2 as rowKey, ROW(age, name) as persona_data from kafka_source");

TableResult result = tEnv.from("kafka_source")
    .select(lit(2).as("rowKey"), row($("age"), $("name")).as("persona_data"))
    .insertInto("hTable")
    .execute();

I expect the persona_data columns (age and name) to be populated with values fetched from Kafka. However, when I check the HBase table after executing this code, only the row key is present, while the columns families remain null. Any insights into why the column values aren't being written to HBase would be greatly appreciated.

Answer 1

Often you have null values in the situation when you have a problem between your data schema and table schema (expected). It could help to print schema before inserting:

TableResult result = tEnv.from("kafka_source")
    .select(lit(2).as("rowKey"), row($("age"), $("name")).as("persona_data"))

result.printSchema()

I think when you constructing your row you have something like

`persona_data` ROW<`f0` INT NOT NULL, `f1` STRING> NOT NULL

but your table have fields <age INT, name STRING>