Monday, August 19, 2013

Active 5.8.0 Master Slave Web Console Fix

Recently I ran into an issue when I attempted to use ActiveMQ 5.8.0 Web Console to monitor a two node Master/Slave ActiveMQ cluster.

The ActiveMQ 5.8.0 Web Console (hosted in a Tomcat container) connects to remote ActiveMQ instance over JMX. In ActiveMQ 5.8.0, the JMX connector is enabled for both Master and Slave nodes. Due to which the Web Console is unable to determine which node is the master node. Hence the issue. 

In order to get around this problem, I created a class which extends the RemoteJMXBrokerFacade class by adding intelligence to an overridden findBrokers(...) method such that the Web Console considers an ActiveMQ node as a master node if and only if it is able to fetch a reference to the BrokerViewMBean. 

The above operation throws an IllegalStateException if the broker was to be slave node. 

The Web Console gets the list of JMX Connectors for all the ActiveMQ Brokers in a cluster via the following property


-Dwebconsole.jmx.url=service:jmx:rmi:///jndi/rmi://amq-node-1-machine:12099/jmxrmi,service:jmx:rmi:///jndi/rmi://amq-node-2-machine:12099/jmxrmi

The following is the source code:

package org.apache.activemq.web;

import java.io.IOException;
import java.util.Set;

import javax.management.MBeanServerConnection;
import javax.management.MBeanServerInvocationHandler;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;

import org.apache.activemq.broker.jmx.BrokerViewMBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * This class helps ActiveMQ Web Console determine if the remote broker instance is master or slave
 * 
 * In the webconsole-properties.xml, replace the <code>org.apache.activemq.web.RemoteJMXBrokerFacade</code> class 
 * with <code>org.apache.activemq.web.MasterSlaveAwareJMXBrokerFacade</code>
 * <pre>
 * {@code
 *  <bean id="brokerQuery" class="org.apache.activemq.web.MasterSlaveAwareJMXBrokerFacade" autowire="constructor" destroy-method="shutdown">
    <property name="configuration" ref="configuration"/>
    <property name="brokerName"><null/></property>
   </bean>
 * }
 * </pre>
 * @author AKUNTAMUKKALA
 *
 */
public class MasterSlaveAwareJMXBrokerFacade extends RemoteJMXBrokerFacade {

 private static final transient Logger LOG = LoggerFactory
   .getLogger(MasterSlaveAwareJMXBrokerFacade.class);

 @Override
 protected Set<ObjectName> findBrokers(MBeanServerConnection connection)
   throws IOException, MalformedObjectNameException {

  LOG.debug("Finding brokers");
  ObjectName name;

  name = new ObjectName("org.apache.activemq:type=Broker,brokerName=*");

  Set<ObjectName> brokers = connection.queryNames(name, null);
  

  try {
   ObjectName brokerObjectName = (ObjectName) brokers.iterator().next();
   BrokerViewMBean mbean = (BrokerViewMBean) MBeanServerInvocationHandler
     .newProxyInstance(connection, brokerObjectName,
       BrokerViewMBean.class, true);

   LOG.debug("Broker Id : " + mbean.getBrokerId());

  } catch (Throwable t) {
   // TODO Auto-generated catch block
   LOG.error("Exception occurred while trying to retrieve BrokerViewMBean", t);
   throw new RuntimeException(t);
  }
  return brokers;
 }

}

If the Web Console attempts to connect to slave broker, the following exception gets thrown.
SEVERE: Servlet.service() for servlet [jsp] in context with path [/activemq-web-console-5.8.0] threw exception [javax.el.ELException: Error reading 'brokerName' on type org.apache.activemq.web.RemoteJMXBrokerFacade] with root cause
java.lang.IllegalStateException: Broker is not yet started.
 at org.apache.activemq.broker.jmx.BrokerView.safeGetBroker(BrokerView.java:459)
 at org.apache.activemq.broker.jmx.BrokerView.getBrokerName(BrokerView.java:75)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
 at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
 at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
 at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65)
 at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216)
 at javax.management.StandardMBean.getAttribute(StandardMBean.java:358)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
 at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638)
 at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404)
 at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
 at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
 at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
 at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:600)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
 at sun.rmi.transport.Transport$1.run(Transport.java:159)
 at java.security.AccessController.doPrivileged(Native Method)
 at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
 at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
 at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
 at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)


This solution has worked for me. I would like to know if you have a better way to fix this.

4 comments:

  1. Do you know if this bug is fixed in ActiveMQ 5.9.0?

    ReplyDelete
    Replies
    1. @johnlindwall I haven't checked this on 5.9.0 version yet. It is on my todo list :)

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hi Ashwini , Do you know hoe to get the broker or broker service instance of the existing broker ? i was just looking for the snippets to get the broker service instance of the remote broker which i should not create one instead i have to get the existing broker service or broker instance from the remote broker ? any suggestions.

    ReplyDelete