java - Flume 1.2.0 自定义源码

标签 java hadoop flume

我有一个自行开发的事件收集和处理框架,我希望用 Apache Flume 1.2.0 来替换它,因为我们将开始将数据加载到 Hadoop 中。由于 Flume 没有 HTTP 源,因此我正在编写带有嵌入式 Tomcat 服务器的自定义源。我正在关注 http://flume.apache.org/FlumeUserGuide.html#custom-source 上的文档但是,当我开始时,我收到以下异常:

java.lang.IllegalArgumentException: No known runner type for source com.xxx.xxx.flume.TestSource@5219f360 at org.apache.flume.SourceRunner.forSource(SourceRunner.java:54) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSources(PropertiesFileConfigurationProvider.java:340) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:222) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722)

最佳答案

我想我已经明白了。 Flume 网站上的文档没有提到自定义实现不应实现 Source 接口(interface),而应实现 EventDrivenSource 或 PollableSource 接口(interface)。

关于java - Flume 1.2.0 自定义源码,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/13201893/

相关文章:

java - Java中如何优化并发操作?

hadoop - Spark 错误 : Failed to Send RPC to Datanode

hadoop - Flume NG FileChannel 很慢

hadoop - 在 HDFS 中创建文件但不附加任何内容

hadoop - Flume 创建小文件

java - 按值的降序对 Map 进行排序

java - 处理自定义菜单栏不显示

java - 找到损坏开始的数组索引

hadoop - Parquet vs ORC vs ORC with Snappy

java - Apache Spark : TaskResultLost (result lost from block manager) Error On cluster