Analysing and Visualising NYC 311 complaints

In this blog post we’re going to be looking at what some people might call “big” data. No that doesn’t mean big in the conventional sense, it means big in the sense that the single file dataset is 10 Gb in size, and I wanted to make a “big data” pun.

The data in question is a record of NYC’s 311 complaints since 2010, the 6th most popular dataset on the opendata website. “311” is a complaints hotline in NYC, for those interested in following along or investigating the data themselves, it is freely available from the open data website.

Today we’re going to cover

  • Creating a data source and importing the data
  • First look at the data to determine interesting fields
  • Some basic visualisations of the data.

Continue reading

30 Seconds of reading, hours of watching – 10 monitoring talks everyone should see

Here at Logscape it should go without saying that monitoring is sort of a big deal. Some would even go as far as to say it’s even our “thing”. To go with that we’ve collated a collection, of what we think might be the best 10 monitoring talks people should watch. Regardless of whether you’re looking to implement a logging tool, build your own or are just a developer, these talks are worth the time.

Continue reading

Logscape and Docker – Get monitoring in 60 seconds

Monitoring Magic

It’s finally that day, Logscape is now on docker hub. As such I’m going to be walking you through the process of getting Logscape running, and once you’ve got the hang of it, you’ll be able to download, run and start using Logscape all within 60 seconds. Monitoring in a heart beat. Continue reading

Native JSON Support

json_bumper.sh-600x600Working with JSON in Logscape 3.2

Logscape 3.2 introduced native JSON support, meaning that when working with JSON data there’s no need for datatypes, instead Logscape automatically pulls the keys from your structure.

This removes the sometimes daunting configuration step, and instead lets you get straight down to business with visualising your data. With that in mind, today we’re going to be embracing our inner geek, and get to work visualising some JSON from the game EvE Online™.


 

Continue reading

Logscape 3.2 Touches Down

ssksLVBLogscape version 3.2 is now available for public download, you can get it now from the Logscape Website.

A brief rundown of Logscape 3.2 brings with it, and what we’re going to cover today…

  • File Explorer
  • JSON Support (Including JSON Arrays)
  • Failover Overhaul
  • Performance and Stability Changes

 


 

Continue reading

Advanced data analytics and use-cases in Logscape

Introduction
self_descriptionLogscape Analytics’ are incredibly powerful, however, are you using them to their full potential? In this blog post we’re going to go over some of the less used analytics, show you how to use them, and hopefully inspire you to use your Logscape instance in new and exciting ways. So, without further ado let’s get into some searches. Continue reading

Logscape Tutorials – Logscape in 10 minutes

Recently we’ve been working on creating new learning materials for the release of Logscape 3.0.Materials appropriate for both the Logscape expert and an individual just picking Logscape up for the first time. The first person to be addressed by this was of cof course the beginner, as such here’s a 10 minute introduction to the basics of Logscape 3.0.

 

 

Hopefully this help some of our newer users, and keep an eye out for more advanced tutorials!

Using Logscape with HPC, 1 of 3

fQf3scmToday we have a blog article by Guest writer Ben Newton, Ben manages a HPC Grid where Logscape is used in anger for all their management needs. This is the first of a three part series by Ben on how they use Logscape, and built their monitoring Solution. You can find more of Ben’s work on his github page, or his LinkedIn.

Microsoft HPC Server 2012 – More Compute, More Monitoring 

640K ought to be enough for anybody.

    – Unknown… but not Bill Gates! Continue reading

10 Ways to Improve Your Output File

cloudcomputingjoke-300x259So you have written an app or log – it’s brilliant, it grabs all the data you need and runs like greased lightning. All you need to do now is ensure your output file has a nice clean format – preferably one that means Logscape does all the work for you! So here are some of my top tips.

1) Add a full time stamp to every line. You wouldn’t believe how much trouble can be caused by people using just times or dates. At the best, you have to struggle to get your data properly organised. At worst, you end up with a mess and data appears in the wrong place on the graph. Do it right, set the date and time!

2) Add a time zone to that stamp. My computer will never move time-zone, surely it’ll be fine? Don’t count on it. British Summer Time changing the system time on half your servers, servers being reset to US time, data centres moving locations… All these things can and will happen. Adding the time zone to the stamp gives you a cast iron assurance that the data will always be correct. That peace of mind is worth a few bytes.

Continue reading

Monitoring Platform Symphony.

Platform
Sitting at my desk, I was approached and asked this question.

Do you guys support Platform Symphony?

An interesting question, really we “support” most anything, Logscape has the base functionality for most tasks, and product “apps” just build upon that functionality, the app provides the workspaces, data types and sources, as well as the tools that Logscape will use to find the data you want. But at the end of the day it’s still Logscape, just it’s running a groovy script to fetch your metrics.

So what is the process that goes into making a Logscape App?
Continue reading