Apache NiFi interprocess communication

Question

To separate the logic, simplify the main flow and reuse processors groups I'm looking for something to implement interprocess communication logic in Apache NiFi.

So far the only thing I was able to find is to use InvokeHTTP in the main flow and HandleHTTPRequest/Response pair in the group. However, having too many open HTTP ports may lead to security issues. Is there a better way to do the same, i.e., routing the flow file and attributes to a needed processor and then return it back? If not, I think it would be quite helpful to have route / intercept flow processors built-in in NiFi.

PS. I know that I can turn a group into a template and use it in the flow, I do not want to do that in this case. I want to be able to use the same logic in multiple flows.

Answer 1

You can use outpout and input port.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html

Port: Dataflows that are constructed using one or more Process Groups need a way to connect a Process Group to other dataflow components. This is achieved by using Ports. A DFM can add any number of Input Ports and Output Ports to a Process Group and name these ports appropriately.

Answer 2

Inner Remote port can be used to communication between not connected processors in NiFi 1.11.x and above: Configure Site-to-Site Server NiFi Instance

Send FlowFile to not directly connected process goup:

1- Add remote process group to NiFi and connect it to current instance.

2- Add remote port to the process group, which you want to receive flowFiles.

3- Enable remote port which has been added to process group.

Receive FlowFile from not directly connected process group:

1- Add remote output port to source process group:

2- Add remote process group to NiFi and connect it to current instance.

3- Enable remote process group and active output port: