Spark 的配置文件(用于检索指标),即 /conf/metrics.properties
,说明以下内容:
Within an instance, a "source" specifies a particular set of grouped metrics. there are two kinds of sources:
Spark internal sources, like MasterSource, WorkerSource, etc, which will collect a Spark component's internal state. Each instance is paired with a Spark source that is added automatically.
Common sources, like JvmSource, which will collect low level state. These can be added through configuration options and are then loaded using reflection.
下面是一些示例,如
master.source.jvm.class=org.apache.spark.metrics.source.JvmSource
但是,没有示例解释如何从 MasterSource
或 WorkerSource
等其他来源获取指标,并且仅替换 JvmSource
并不能解决问题。
建议?
最佳答案
MasterSource 或 WorkerSource(均为内部源)会通过指标自动跟踪,因此无需明确指定。
例如,
master.sink.cvs.class=org.apache.spark.metrics.sink.CsvSink
每 10 秒将主实例指标输出到 /tmp
目录。
关于java - Spark : configuration file 'metrics.properties' ,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/31614169/