Collectd – Exploring Available Sensors

In this blog post we are going to take a look at some of the sensors that come with Collectd and how  to chart the information. Here are some useful links on searching in Logscape.

Before we get straight into the searches lets take some time and understand how Collectd outputs data.

Collectd Metric Data

The collectd daemon stores data from other linux host on disk by the server name and the plugin. The load plugin is set by default on most Linux distributions and is stored like this.

/var/lib/collectd/central_server/svr001/load/ ….

In this set up many collectd daemons are forwarding their metric data to one central host.

/var/lib/collectd/central_server/svr0002/tcpconns-80-local/ ….

Using the Collectd DataType

To chart the data in Logscape you need a passing familiarity with the how to search using a  data type.

metrics_diagram

To execute a search I would need to know which Collectd plugin I am interested in and what metrics it outputs. The table of all collectd plugins can be found here. Here’s an example which charts the load of a host svr0001

 | _type.equals(collectd) plugin.equals(load)  value.max(id)  _host.equals(svr0001)

One thing to note from the search is the use of the id field. This field is a unique key which identifies each unique metric value by  host, plugin  and instance.

Lets take a look at a few other search examples

Chart all Established TCP Connections

The tcpconns plugin provides connection counters for different TCP connection states. We are interested in the established connections in our example.

| _type.equals(collectd)  plugin.equals(tcpconns) value.max(id,) chart(line)  _filename.contains(ESTABLISHED)

Each connection state is stored in a separate file.

Selection_024

In the diagram above I have charted active connections in a Logscaep deployment. One of the agents in the QA environment shows a notable spike in the number of established connections for port 11090 compared to all the other Logscape agents.

System Load and CPU  (Line Chart )

The load plugin is easy to work with.  To build this search select the load plugin and then use the max analytic on the value field to display the load.  The search syntax is just a matter of selecting the plugin and using the max analytic on the value field.

| _type.equals(collectd) value.max(id,) plugin.equals(load)

 

Selection_019

CPU Temperatures

The temperature is slightly more trickysince collectd gets this value from the sensor plugin. The sensor plugin outputs values for power usage, temperatures, fans and so on. In this search we have chosen just the cpu temperature. Here is where it is stored

/var/lib/collectd/remote_SERVER_/indexstore-qa/sensors-coretemp-isa-0000/ ..

Inside this directory is a file for each core. Since this plugin includes many other sensors we use the contains filter instead of the equals. 

| _type.equals(collectd) plugin.contains(sensors) _filename.contains(temp) value.max(id,)  chart(line)

Selection_482_blanched

Workspace

The CPU temperatures, System load and active TCP connections can all used on a single dashboard to provide a particular view of a server environment.

Selection_483

Show all collectd plugins

We’ll end this post with a search that tells us a bit about our collectd environment. This search will show all the metrics being collected, grouped by host in a table. It is a useful search because it shows us which plugins have been configured for each host. Using this table we can easily see which  plugins have not yet been configured on certain hosts. Seeing no values may also indicate a misconfigured plugin or the sensor not supported on that server’s hardware.

Selection_025

You can download Logscape from here

 

You can  download the collectd data type from this location  and upload it through the backup page in the Settings section of Logscape.