How to get attribute value with correct datatype from Row in java spark sql

Question

I am using Spark Java (not scala or python).

I have a dataframe (Dataset<Row>) and I want to access specific fields from the Row. I have been able to get the field value if it is a String, but I don't know what to do if it is a Long, Int...etc. Here is how I access it if its a String attribute:

final String id = row.getAs("id").toString();

Can the getAs() Row method be used for other datatypes as well? I don't know if it can properly type the retrieved value. If in the database I queried, quantity is type Long then I want to retrieve it from Row as type Long too. Such as:

final Long quantity = row.getAs("quantity");

I saw that there are Row methods like getLong(), getInt() but I don't know how to use them because the methods take in an int. I only have the field name of the value I want to access so I can only pass in a field name string.

Answer 1

getAs(column) method already gives you the type of the column's value, then you can just cast it if you know it in order to handle it.

Long myId = (Long)row.getAs("id")

Other methods rely on the index of the value, so if you have a dataset having those columns:

longId | stringValue

then you could use:

row.getLong(0)
row.getString(1)

see more: https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/sql/Row.html#getAs(java.lang.String)