How to enable custom stage library for streamsets data collector?

Question

I have custom stage library which I want to use in Streamsets Data collector pipeline, I have followed all the steps given in below link to install custom stage lib, but still I am not able search stage library data collector.

Could you please help ?

SS document link :
docs.streamsets.com/datacollector/latest/help/datacollector/UserGuide/Configuration/CustomStageLibraries.html#concept_pmc_jk1_1x

Answer 1

There are only a few things to check:

First, ensure you've exported the USER_LIBRARIES_DIR environment variable so SDC can find the custom stage libraries:

export USER_LIBRARIES_DIR="/opt/sdc-user-libs/"

Then give the custom stage library the appropriate permissions in the JVM's security manager. (Verify that you've added a section like this example to the $SDC_CONF/sdc-security.policy file):

// custom stage library directory
grant codebase "file:///opt/sdc-user-libs/-" {
   permission java.security.AllPermission;
};

Verify that you've restarted SDC to pick up the previously-mentioned changes.

After restarting SDC, ensure the custom stage's jar files have been opened by the JVM:

lsof -p (sdc pid) | grep (jar file name)

The documentation can be found here