java - kafka.admin.AdminClient.listAllConsumerGroups() 有时会失败

标签 java apache-kafka

我一直在尝试使用 AdminClient 类获取所有消费者组,但有时我会遇到运行时异常。我正在使用带有 Angular 前端的 Spring Boot 应用程序,并发出 http 请求来调用此方法:

@RequestMapping("groupIds")
    public List<String> getGroupIds() {

        /*get all of the consumer groups and convert the scala map to a java map*/
        Map<Node, scala.collection.immutable.List<GroupOverview>> consumerGroups = scala.collection.JavaConverters
                .mapAsJavaMapConverter(adminClient.listAllConsumerGroups()).asJava();

        /*Remove the GroupOverviews from the Map*/
        List<GroupOverview> consumerGroupList = getConsumerGroupOverviewList(consumerGroups);

        /*Remove the groupIds from the GroupOverviews*/
        return getGroupIdsFromGroupOverview(consumerGroupList);
    }

大多数时候它工作正常,但有时会失败。我似乎无法弄清楚是什么原因造成的,但它可以通过重新加载页面来修复。这是堆栈跟踪:

org.springframework.web.util.NestedServletException: Request processing failed;
nested exception is java.lang.RuntimeException: Request METADATA failed on broke
rs List(localhost:9092 (id: -1 rack: null))
        at org.springframework.web.servlet.FrameworkServlet.processRequest(Frame
workServlet.java:982) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServl
et.java:861) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) ~[javax.
servlet-api-3.1.0.jar:3.1.0]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkSer
vlet.java:846) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.
servlet-api-3.1.0.jar:3.1.0]
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841
) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1634) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(We
bSocketUpgradeFilter.java:206) ~[websocket-server-9.4.2.v20170220.jar:9.4.2.v201
70220]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.do
FilterInternal(ApplicationContextHeaderFilter.java:55) ~[spring-boot-1.5.2.RELEA
SE.jar:1.5.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilter
Internal(WebRequestTraceFilter.java:108) ~[spring-boot-actuator-1.5.2.RELEASE.ja
r:1.5.2.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(
RequestContextFilter.java:99) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInter
nal(HttpPutFormContentFilter.java:105) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELE
ASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInterna
l(HiddenHttpMethodFilter.java:81) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterIntern
al(CharacterEncodingFilter.java:197) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEAS
E]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilter
Internal(MetricsFilter.java:106) ~[spring-boot-actuator-1.5.2.RELEASE.jar:1.5.2.
RELEASE]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerR
equestFilter.java:107) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1621) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java
:541) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:143) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:548) ~[jetty-security-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:132) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandl
er.java:190) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandl
er.java:1592) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandl
er.java:188) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandl
er.java:1239) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandle
r.java:168) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:
481) ~[jetty-servlet-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandle
r.java:1561) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandle
r.java:166) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandle
r.java:1141) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.j
ava:141) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper
.java:132) ~[jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.Server.handle(Server.java:564) ~[jetty-serve
r-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) ~[j
etty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.jav
a:251) [jetty-server-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstra
ctConnection.java:279) [jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) [je
tty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[jetty-io-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.jav
a:122) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.inv
oke(ExecutingExecutionStrategy.java:58) [jetty-util-9.4.2.v20170220.jar:9.4.2.v2
0170220]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceC
onsume(ExecuteProduceConsume.java:201) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20
170220]
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Exec
uteProduceConsume.java:133) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPoo
l.java:672) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool
.java:590) [jetty-util-9.4.2.v20170220.jar:9.4.2.v20170220]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.lang.RuntimeException: Request METADATA failed on brokers List(l
ocalhost:9092 (id: -1 rack: null))
        at kafka.admin.AdminClient.sendAnyNode(AdminClient.scala:66) ~[kafka_2.1
2-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.findAllBrokers(AdminClient.scala:90) ~[kafka_
2.12-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.listAllGroups(AdminClient.scala:98) ~[kafka_2
.12-0.10.2.1.jar:na]
        at kafka.admin.AdminClient.listAllConsumerGroups(AdminClient.scala:112)
~[kafka_2.12-0.10.2.1.jar:na]
        at consumer.KafkaController.getGroupIds(KafkaController.java:154) ~[main
/:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.
0_131]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:62) ~[na:1.8.0_131]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43) ~[na:1.8.0_131]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_131]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvok
e(InvocableHandlerMethod.java:205) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.RELEASE]

        at org.springframework.web.method.support.InvocableHandlerMethod.invokeF
orRequest(InvocableHandlerMethod.java:133) ~[spring-web-4.3.7.RELEASE.jar:4.3.7.
RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocabl
eHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-
webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingH
andlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[sprin
g-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingH
andlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-web
mvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapt
er.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.7.RELEASE.ja
r:4.3.7.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(Dispatch
erServlet.java:963) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.DispatcherServlet.doService(Dispatche
rServlet.java:897) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(Frame
workServlet.java:970) ~[spring-webmvc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 57 common frames omitted

最佳答案

我不知道是什么导致了这个问题,但每次在调用它的方法之前创建 AdminClient 似乎已经解决了我的问题。从那以后我再也没有遇到过这个错误。所以,如果你每次调用之前都创建adminClient

Map<Node, scala.collection.immutable.List<GroupOverview>> consumerGroups = scala.collection.JavaConverters
                .mapAsJavaMapConverter(adminClient.listAllConsumerGroups()).asJava();

您永远不应该看到这个问题。

关于java - kafka.admin.AdminClient.listAllConsumerGroups() 有时会失败,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/44850431/

相关文章:

java - 查看寻呼机设置当前项目不能正常工作

java - 访问远程服务器时 MySQL 连接出错

azure - 将现有 VNET 与 Azure Function APP 结合使用

java - 使用自定义 TimestampExtractor 的 Kafka Streams 窗口

apache-kafka - Kafka : Continuously getting FETCH_SESSION_ID_NOT_FOUND

java - kafka_2.11-0.8.2.1 消费者再平衡

android - 找不到从方法 Y 引用的类 X

java - 时区 java : San Antonio, 得克萨斯州

java - 更改 servlet header 并重定向到过滤器中的其他 url

java - 反序列化 avro 对象时出现 spring MessageConversionException