in Talend, catch an error message without the job dying

Question

in Talend, I am loading data from CSVs to SQL (and Redshift) tables

if one errors, I want to log and continue

my tMSSQLOutput component does not have Die on Error because I want the job to continue, but then I want to note the failure (in a log table)

the job writes an error in red in the log window when I run, but I don't seem to have access to that error in my subsequent tJava component

I've used onComponentError, onComponentOK, RunIf, onSubjobError; nothing seems to work

When I check the following: globalMap.get("tMSSqlOutput_2_ERROR_MESSAGE") it's always null--unless I check Die on Error which defeats the purpose

I suppose, my approach is wrong; but I want what I want (actually, what my manager wants); any recommendations?

[if there are issues with the way the question is posed, it's not malice/trolling/etc; I just need some help]

p.s.: die-on-error should be an available tag; it's very common in Talend
same for tMSSQLOutput

Answer 1

In Talend id any job has an error or failure, it will not continue the process. Jobs get stopped with the error. You have to resolve the error and load the delta data.

In order to log the error message you have to use tLogCatcher in your job and direct it to a file. Below is the design. Use this component in your job

And you cannot log the error and continue. You have to resolve the error, talend will not allow you to continue with error

Answer 2

If you want your job to continue even if there is an error on your tMSSQLOutput and be able to log each error, you will have to uncheck the "use batch" in advanced options of tMSSQLOutput. Then you will be able to use the Reject links in the component output (Right-Click>Main>Rejects). Then you can trace which line is in error, and log the error message. Unfortunately the job will be slower , as you won't be able to push data in DB using batch option. The other option is to redesign the job to intercept any possible error before using tMSSQLOutput (using tMap, tFilter, etc). This way you can still use batch option.