In this blog post we are going to take a look at some of the sensors that come with Collectd and how to chart the information. Here are some useful links on searching in Logscape.
Before we get straight into the searches lets take some time and understand how Collectd outputs data.
Collectd Metric Data
The collectd daemon stores data from other linux host on disk by the server name and the plugin. The load plugin is set by default on most Linux distributions and is stored like this.
In this set up many collectd daemons are forwarding their metric data to one central host.
Using the Collectd DataType
To chart the data in Logscape you need a passing familiarity with the how to search using a data type.
To execute a search I would need to know which Collectd plugin I am interested in and what metrics it outputs. The table of all collectd plugins can be found here. Here’s an example which charts the load of a host svr0001
| _type.equals(collectd) plugin.equals(load) value.max(id) _host.equals(svr0001)
One thing to note from the search is the use of the id field. This field is a unique key which identifies each unique metric value by host, plugin and instance.
Lets take a look at a few other search examples
Chart all Established TCP Connections
The tcpconns plugin provides connection counters for different TCP connection states. We are interested in the established connections in our example.
| _type.equals(collectd) plugin.equals(tcpconns) value.max(id,) chart(line) _filename.contains(ESTABLISHED)
Each connection state is stored in a separate file.
In the diagram above I have charted active connections in a Logscaep deployment. One of the agents in the QA environment shows a notable spike in the number of established connections for port 11090 compared to all the other Logscape agents.
System Load and CPU (Line Chart )
The load plugin is easy to work with. To build this search select the load plugin and then use the max analytic on the value field to display the load. The search syntax is just a matter of selecting the plugin and using the max analytic on the value field.
| _type.equals(collectd) value.max(id,) plugin.equals(load)
The temperature is slightly more trickysince collectd gets this value from the sensor plugin. The sensor plugin outputs values for power usage, temperatures, fans and so on. In this search we have chosen just the cpu temperature. Here is where it is stored
Inside this directory is a file for each core. Since this plugin includes many other sensors we use the contains filter instead of the equals.
| _type.equals(collectd) plugin.contains(sensors) _filename.contains(temp) value.max(id,) chart(line)
The CPU temperatures, System load and active TCP connections can all used on a single dashboard to provide a particular view of a server environment.
Show all collectd plugins
We’ll end this post with a search that tells us a bit about our collectd environment. This search will show all the metrics being collected, grouped by host in a table. It is a useful search because it shows us which plugins have been configured for each host. Using this table we can easily see which plugins have not yet been configured on certain hosts. Seeing no values may also indicate a misconfigured plugin or the sensor not supported on that server’s hardware.
You can download Logscape from here
You can download the collectd data type from this location and upload it through the backup page in the Settings section of Logscape.