Camel SFTP started failing after camel 2.22.1 - camel 3.9.0 upgrade

Question

In camel 2.22.1 I used the following camel route to perform a file operation

from(sftp://" + sourceUrl + "&preferredAuthentications=password&includeExt=xml&delete=true&disconnect=true&maxMessagesPerPoll=50&preMove=${file:name.noext}.process")

this will rename the files with .xml extension to .process and perform further route operations and finally delete the .process file from input folder. However camel 3.9.0 this route started failing. And I got the following error from the Camel file component:

org.apache.camel.component.file.GenericFileOperationFailedException: Cannot delete file: source/do-sfdc-case-import-0/2451165.process
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.jcraft.jsch.SftpException: No such file
    at org.apache.camel.component.file.remote.SftpOperations.deleteFile(SftpOperations.java:488)
    ... 22 common frames omitted

org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory to: ..
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.jcraft.jsch.SftpException:   
    at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:682)
    ... 11 common frames omitted
Caused by: java.io.IOException: Pipe closed 
    at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:337)
    ... 12 common frames omitted

To resolve the issue I tried to set stepwise=false flag however the application became dead slow, even though when the file mentioned in this stacktrace is available in the folder, but camel reports it to be not found or is not able to change to the corresponding directory.

Any idea what am I doing wrong here? Appreciate any help/tips on the same.

Answer 1

After several tests and debugging sessions, I understood from my application that Camel/jsch is not handling multi-threading quite efficiently and after a few google searches, i found this mail thread https://www.mail-archive.com/[email protected]&q=subject:%22GenericFileOperationFailedException%22&o=newest&f=1 that supported my theory with camel/jsch and multi-threading.

There is always a timing issue as when then route is trying to delete the file in the specific folder, the file is already deleted and this caused the above mentioned error in the route. And, after reading the camel-sftp documentation, I used the synchronous=true flag which ensures that camel is strictly processing the routes synchronously, the problem was solved for me.