Trying to dump a small csv data into Kafka using Flume, I constantly got the error below no matter what input or configuration I tried when running "flume-ng avro-client -H localhost -p 4141 -F test_all_num.csv":
The flume agent seemed to start successfully when I ran "flume-ng agent -f kafkaSink.flm -n a1 -c /usr/local/Cellar/flume/1.9.0_1/libexec/conf".
test_all_num.csv is a small csv file consisting of 46 lines of numbers separated by comma.
Below is the screenshot of my configuration file for the flume agent:
I tried adding the limits of capacity for the channels (a1.channels.c1.capacity = 1000, a1.channels.c1.transactionCapacity = 100) but that did not work.
Editing JAVA_OPTS in flume-env.sh was also tried but did not seem to work. Below is the screenshot of the flume-env.sh file:
Currently I am a bit stuck. Does anyone have any suggestion for this issue? Any help or pointer will be appreciated, thanks!