Getting data out of JMX and into Cacti

JMX, if you don't know, is one of the things that makes Java really great. It allows you to write your Java app and instrument it as if instrumentation was free. It's not free, of course, but it's so low cost, that in Cassandra, there's a JMX "bean" that counts every single read and write to a column family. And you can have this - in production. However, it's generally somewhat difficult to get data from JMX to another programming language. Depending on how the remote side (the JMX agent) is configured, it could be expecting to exchange raw Java objects. Not great for interoperability. In our particular case, we wanted to get data from Cassandra and into Cacti. We tried several different generic JMX clients with varying levels of success, however, none of them were fast.

Most of them could only list the attributes for one bean at a time, and required spawning an entire JVM for each request. Because we knew we were going to be collecting nearly every JMX statistic in a short amount of time, this wouldn't cut it.

Our solution was to create a new client application specifically for doing this kind of monitoring. Unimaginatively, we called it 'JMXClient', and it has a few distinguishing features:

  1. It's fast. 993 attributes can be retrieved in 1 second. Other solutions we saw did 10 attributes in 1.5 or even 2 seconds.
  2. It has no dependencies other than a Java 1.5 or 1.6 install.
    Some solutions required external Jar files, or even ant!
  3. It outputs results in PHP's native serialization format.
    All other solutions assumed a human would be consuming the data. 

For us, all three of these items were a big win. Future plans include adding or replacing the PHP serialization output with JSON or some other cross-platform/langauge format.

Comments

And another option: a JMX to JSON bridge: http://www.jolokia.org/ 

Wed, 03/16/2011 - 10:43

Jerry just sent me the link to this posting.

I wanted to let you know that I've done a somewhat similar project after recently finding a big need for a solid tool to query jmx and write the data out efficiently. My tool allows you to create Queries based on easy json configuration which hit jmx for the properties that you need and OutputWriter's that can write the data however you want. One of the cool writers I created is one that shells out directly to rrdtool. So, now we have a direct JMX->rrd query transformer. Its fully multithreaded at multiple levels so it should easily support a massive server farm.

Take a look... http://jmxtrans.googlecode.com/

Tue, 12/14/2010 - 22:38

Reply

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.
Website by Digital Loom