I am learning Apache Flume and I am trying to create an agent that writes records to a Hive table using a Hive sink. When I try to start my agent, I get the following error:
2023-11-12T22:16:52,317 INFO [main] sink.DefaultSinkFactory: Creating instance of sink: databaseSink, type: hive
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hive/hcatalog/streaming/RecordWriter
at org.apache.flume.sink.hive.HiveSink.createSerializer(HiveSink.java:223)
I have already tried adding the following variations to my '''flume-env.sh''' script with no success:
FLUME_CLASSPATH="/usr/local/hive/lib/"
FLUME_CLASSPATH="/usr/local/hive/lib/*"
FLUME_CLASSPATH="/usr/local/hive/lib/hive-hcatalog-core-3.1.3.jar"
I have tried starting the agent with the following commands without any success
flume-ng agent -n agent -c ${FLUME_HOME}/conf -f ${FLUME_HOME}/conf/hivetest.conf
flume-ng agent -n agent -c ${FLUME_HOME}/conf -f ${FLUME_HOME}/conf/hivetest.conf --classpath /usr/local/hive/lib