apache-nifi - 自定义处理器的单独日志文件

Question

NiFi 1.2.0

除了NiFi创建的app.log文件之外，还需要创建一个单独的日志文件，例如customprocessor.log。

我经历了一些有趣的现有线程，但是，我无法弄清楚如何使其在代码中工作。

以下是现有的 logback.xml :

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="30 seconds">
    <contextListener>
        <resetJUL>true</resetJUL>
    </contextListener>
    <appender name="APP_FILE">
        <file>/var/log/nifi/nifi-app.log</file>
        <rollingPolicy>
            <!--
                               For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>/var/log/nifi/archive/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy>
                <maxFileSize>100MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <!-- keep 30 log files worth of history -->
            <maxHistory>3</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
     <immediateFlush>true</immediateFlush>
    </appender>
    <appender name="USER_FILE">
        <file>/var/log/nifi/nifi-user.log</file>
        <rollingPolicy>
            <!--
                               For daily rollover, use 'user_%d.log'.
              For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>/var/log/nifi/archive/nifi-user_%d.log</fileNamePattern>
            <!-- keep 30 log files worth of history -->
            <maxHistory>3</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>
    <appender name="BOOTSTRAP_FILE">
        <file>/var/log/nifi/nifi-bootstrap.log</file>
        <rollingPolicy>
            <!--
              For daily rollover, use 'user_%d.log'.
              For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>/var/log/nifi/archive/nifi-bootstrap_%d.log</fileNamePattern>
            <!-- keep 5 log files worth of history -->
            <maxHistory>5</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>
    <appender name="CONSOLE">
        <encoder>
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
    </appender>
    <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR -->
    <logger name="org.apache.nifi" level="INFO"/>
    <logger name="org.apache.nifi.processors" level="WARN"/>
    <logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO"/>
    <logger name="org.apache.nifi.controller.repository.StandardProcessSession" level="WARN" />
    <logger name="org.apache.zookeeper.ClientCnxn" level="ERROR" />
    <logger name="org.apache.zookeeper.server.NIOServerCnxn" level="ERROR" />
    <logger name="org.apache.zookeeper.server.NIOServerCnxnFactory" level="ERROR" />
    <logger name="org.apache.zookeeper.server.quorum" level="ERROR" />
    <logger name="org.apache.zookeeper.ZooKeeper" level="ERROR" />
    <logger name="org.apache.zookeeper.server.PrepRequestProcessor" level="ERROR" />
    <logger name="org.apache.calcite.runtime.CalciteException" level="OFF" />
    <logger name="org.apache.curator.framework.recipes.leader.LeaderSelector" level="OFF" />
    <logger name="org.apache.curator.ConnectionState" level="OFF" />
    <!-- Logger for managing logging statements for nifi clusters. -->
    <logger name="org.apache.nifi.cluster" level="INFO"/>
    <!-- Logger for logging HTTP requests received by the web server. -->
    <logger name="org.apache.nifi.server.JettyServer" level="INFO"/>
    <!-- Logger for managing logging statements for jetty -->
    <logger name="org.eclipse.jetty" level="INFO"/>
    <!-- Suppress non-error messages due to excessive logging by class or library -->
    <logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/>
    <logger name="com.sun.jersey.spi.spring" level="ERROR"/>
    <logger name="org.springframework" level="ERROR"/>
    <!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -->
    <logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/>
    <!--
        Logger for capturing user events. We do not want to propagate these
        log events to the root logger. These messages are only sent to the
        user-log appender.
    -->
    <logger name="org.apache.nifi.web.security" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.web.api.config" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.authorization" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.cluster.authorization" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <logger name="org.apache.nifi.web.filter.RequestLogger" level="INFO" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>
    <!--
        Logger for capturing Bootstrap logs and NiFi's standard error and standard out.
    -->
    <logger name="org.apache.nifi.bootstrap" level="INFO" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>
    <logger name="org.apache.nifi.bootstrap.Command" level="INFO" additivity="false">
        <appender-ref ref="CONSOLE" />
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>
    <!-- Everything written to NiFi's Standard Out will be logged with the logger org.apache.nifi.StdOut at INFO level -->
    <logger name="org.apache.nifi.StdOut" level="INFO" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>
    <!-- Everything written to NiFi's Standard Error will be logged with the logger org.apache.nifi.StdErr at ERROR level -->
    <logger name="org.apache.nifi.StdErr" level="ERROR" additivity="false">
        <appender-ref ref="BOOTSTRAP_FILE" />
    </logger>
    <root level="DEBUG">
        <appender-ref ref="APP_FILE"/>
    </root>
</configuration>

现在，我可以为自定义日志文件添加一个新的附加程序:

<!-- Start : Separate log file for custom processor -->
<appender name="CUSTOM_FILE">
        <file>/var/log/nifi/custom-processor.log</file>
        <rollingPolicy>
            <!--
                               For daily rollover, use 'app_%d.log'.
              For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'.
              To GZIP rolled files, replace '.log' with '.log.gz'.
              To ZIP rolled files, replace '.log' with '.log.zip'.
            -->
            <fileNamePattern>/var/log/nifi/archive/custom-processor_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern>
            <timeBasedFileNamingAndTriggeringPolicy>
                <maxFileSize>100MB</maxFileSize>
            </timeBasedFileNamingAndTriggeringPolicy>
            <!-- keep 30 log files worth of history -->
            <maxHistory>3</maxHistory>
        </rollingPolicy>
        <encoder>
            <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
        </encoder>
     <immediateFlush>true</immediateFlush>
    </appender>
<!-- End : Separate log file for custom processor -->



<!-- Start : Separate log file for custom processor -->
    <logger name="com.nifi.CustomLog" level="DEBUG" additivity="false">
        <appender-ref ref="CUSTOM_FILE" />
    </logger>
    <!-- End : Separate log file for custom processor -->

我有以下问题:

1. 我添加的条目是否正确

2. 在代码中，我使用以下代码片段来获取根记录器， 但是，我没有找到方法/构造函数来获取我的自定义记录器 在代码中，我该怎么做？

   导入 org.apache.nifi.logging.ComponentLog;..final ComponentLog logger = getLogger();logger.debug("...");

Answer 1

我有类名称为:org.apache.nifi.processors.groovyx.ExecuteGroovyScript

的自定义处理器

为了获取记录器，我正在使用 this.getLogger()

在这种情况下，将以下配置添加到 conf/logback.xml 将创建一个单独的文件 logs/x.log，其中仅显示来 self 的处理器的日志。

<appender name="X_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${org.apache.nifi.bootstrap.config.log.dir}/x.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/x_%d.log</fileNamePattern>
        <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
        <pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
    </encoder>
</appender>
<logger name="org.apache.nifi.processors.groovyx.ExecuteGroovyScript" level="INFO" >
    <appender-ref ref="X_FILE"/>
</logger>

请注意，在记录器定义中应该有您的处理器的类名

您还可以将包名称(或父包名称)定义为记录器名称，以覆盖位于该包中的多个处理器:

<logger name="org.apache.nifi.processors.groovyx" level="INFO" >
    <appender-ref ref="X_FILE"/>
</logger>

在本例中，org.apache.nifi.processors.groovyx 包的所有类 所有子包都会被记录到X_FILE记录器中。