Is there any way to make mathematical operations for some values in files with apache nifi?

Question

I am getting some numerical data with API from URL and I am looking for a way to make some mathematical operations in apache nifi before putting data to file directory. Thanks already now.

By the way, I am using InvokeHTTP processor to get data and to put file in somewhere I am using PutFile processor. I searched some related websites but I could not find out a working way.

Answer 1

Try using QueryRecord processor and Define Record Reader/Writer controller services to read/write the flowfile.

• Add new property to the QueryRecord processor by using Apache calcite SQL query with your mathematical operations on flowfile.

• Results of the SQL query will be added to the outgoing flowfile in your desired format.

Answer 2

Ultimately the answer depends on whether the data you're working with is in the content of the FlowFile or in the attributes. If the data is small enough and it's only a couple operations, the suggested approach would be to work with the data as attributes and use NiFi's expression language to do the transformations.

There is a section of mathematical operations[1] in the Apache documentation[2]. The operations range from simple operand like plus/minus to exposing the java.lang.Math static methods.

[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#numbers [2] https://nifi.apache.org/docs.html

Answer 3

You can try ExecuteStreamCommand if you want to intake the whole file and then run operations. Alternatively, you can fiddle around with the variables on the flowfile - depending on how large your operation is.

For example if you have some initial variables you can include them in the name of your file and then extract them, run the operations within the variables of the flowfile, then add to the bottom of the original file