Visualizing UK accident data with Logscape

In my ever onward quest to show to the world how easy it is to get up and started with Logscape, today I’m going to use a Logscape docker container in order to build visualisations based off some publicly available CSV files in no time at all. If you’ve never used the Logscape docker image, then check out my previous blog.

Today we’re going to be analysing data made available via the gov.uk website, which offers statistics for crashes in the UK for the year of 2015. The specific dataset is available for download here.

Continue reading

30 Seconds of reading, hours of watching – 10 monitoring talks everyone should see

Here at Logscape it should go without saying that monitoring is sort of a big deal. Some would even go as far as to say it’s even our “thing”. To go with that we’ve collated a collection, of what we think might be the best 10 monitoring talks people should watch. Regardless of whether you’re looking to implement a logging tool, build your own or are just a developer, these talks are worth the time.

Monitoring at Spotify – When things go ping in the night. – by Martin Parm

This non-tech talk covers how logging at Spotify has adapted over the years. From the days where monitoring what just a gleam in the developer’s eyes, to the months where operations slept restlessly fearing the inevitable phone call, to the current day monitoring system which act as caretakers to the infrastructure responsible for streaming hundreds of thousands of tracks a day.

Metrics, Metrics everywhere – by Coda Hale

Digging back into the archives, we have Coda Hale’s talk from 2011. While you may think it’s dated, and irrelevant by todays standards. Coda covers some topics which will simply never get old.

Monitoring is Dead. Long Live Monitoring. – by Greg Poirier

Greg Poirer thinks it’s time we stop viewing its metrics in isolation and declare things alive or dead. In this laid back high-energy talk, he covers his opinion on the definition of monitoring, with only a few jabs at DevOps. A must watch.

Better living through statistics. Monitoring doesn’t have to suck. – by Jamie Wilkinson

Jamie Wilkinson goes over what he believes to be the problems in how we currently monitor, and how we can get rid of these problems so that everyone can benefit from logging. A great talk that discusses why our current logs simply aren’t precise enough.

 

The art of performance monitoring. – by Brian Smith

Brian Smith covers the mistakes that he’s made, and the mistakes he keeps seeing developers making. For a 25 minute talk the sheer quantity of technical ideas conveyed in this talk is impressive, but not for the faint of heart.

What should I monitor, and how should I do it? – by Baron Schwartz

In this talk, Baron criticises our approach to monitoring, which is to just stare at a graph and attempt to determine what’s gone wrong. Baron highlights the importance of not just collecting data, but collecting actional data.

Creating a Culture of Observability. – by Cory Watson

Taking a leaf out of the Spotify playbook, this talk covers the culture around logging, rather than logging itself. It’s Cory’s own story of how after joining Strike he managed to instil a culture of observability and monitoring. You’ll hear about his journey towards that goal, with the good, the bad and the downright sneaky.

How monitoring works at scale. – by Ran Leibman

Facebook manage to claim another spot on this top list, this time discussing the challenge which is monitoring the huge amount of infrastructure that make up Facebook around the world. Most companies dont have to monitor on this scale, which leaves an obvious questions, “How exactly do you?”.

The evolution of monitoring systems at google. – by Tom Rippy

Following in the steps of Facebook, Google returns for its second spot in the list. Much like the Spotify talk above, Tony Rippy aims to walk us through the progression of monitoring at Google, and includes some facts that you just wouldn’t believe about the now Tech-Giant.Tony wasn’t present for the whole of this history, but it’s portrayed in a fun and interesting way, which demonstrate whilst your current monitoring solution may not be the best, it doesn’t mean you can’t progress.

Allison McKnights talk demonstrates how monitoring doesn’t have to be expensive. Hailing from Etsy which is known for doing a lot with not a lot. Allison walks us through her experiences of using open source projects to build a monitoring system capable of monitoring the entirety of Etsy’s back end.

 

Hopefully you’ve enjoyed the videos, if you have any that you feel should be added to the list, feel free to drop them below, or tweet @logscape with why.

Failover: Keeping your Environment alive!

Why Everything I Have Is Broken

Computers break… it’s a fact of life! Sometimes it’s a nice quick fix, such as the one cunningly suggested by Randall Munroe. Sometimes it can take hours of trawling through logs. Regardless of the reason, in an ideal world,you want to fix the problem as soon as possible; but the next best thing is to have a Failover – another server that works just as well!

In a Logscape environment, your Management agent is the central point of your environment. It controls alerts, provides users access and runs the entire system: without it, you have nothing. So how do you make sure that your environment is resilient against a Management Agent failure? Simple… you add another one!

The recent 3.2 Logscape release has added new and improved Failover capabilities, making it easier to provide a seamless environment for your users. Having recently implemented this feature in my environment, I thought I’d share with you both the benefits – and the possible pitfalls – of installing this useful bit of kit.

You will need:

  • An existing Logscape Environment with a subscription (Failover is not supported without a license) running at least version 3.2.
  • A server you wish to make your new Failover Agent.
  • A little bit of understanding about ports
  • A pinch of bravery.

Continue reading

Native JSON Support

json_bumper.sh-600x600Working with JSON in Logscape 3.2

Logscape 3.2 introduced native JSON support, meaning that when working with JSON data there’s no need for datatypes, instead Logscape automatically pulls the keys from your structure.

This removes the sometimes daunting configuration step, and instead lets you get straight down to business with visualising your data. With that in mind, today we’re going to be embracing our inner geek, and get to work visualising some JSON from the game EvE Online™.


 

Continue reading

Logscape 3.2 Touches Down

ssksLVBLogscape version 3.2 is now available for public download, you can get it now from the Logscape Website.

A brief rundown of Logscape 3.2 brings with it, and what we’re going to cover today…

  • File Explorer
  • JSON Support (Including JSON Arrays)
  • Failover Overhaul
  • Performance and Stability Changes

 


 

Continue reading

Converting Splunk Searches into Logscape

universal_converter_boxConverting Splunk searches into Logscape

Logscape and Splunk share a lot of overlap, and there is one question we get asked quite often by people looking to migrate from Splunk to Logscape.

 How do we convert Splunk searches and Workspaces into Logscape?

Unfortunately, there’s no magic cure or just click here style solution. Fortunately it is significantly easier than you think.

We’re going to cover converting Splunk searches into their Logscape equivalent.

Continue reading

boot.properties – 5 hints for Logscape Environments

A Logscape Agents is incredibly powerful: it might be a Forwarder shipping data or an IndexStore receiving it. It might even be a Management Agent providing the web front end. Regardless of what it may become, they all start from the boot.properties file. This small, innocuous looking file sitting in the Logscape folder is what makes the difference between a powerful, resource consuming Manager and a small, lightweight forwarder. Here are 5 useful tips for dealing with this file. Continue reading

Advanced data analytics and use-cases in Logscape

Introduction
self_descriptionLogscape Analytics’ are incredibly powerful, however, are you using them to their full potential? In this blog post we’re going to go over some of the less used analytics, show you how to use them, and hopefully inspire you to use your Logscape instance in new and exciting ways. So, without further ado let’s get into some searches. Continue reading

Lifecycle tracking with TXN in Logscape!

The Problem of Ticket Tracking

online_package_trackingSystems tend to have workflows – where an object or ticket are passed around different systems. As someone responsible for monitoring such a system, I need to be able to keep track of the events to ensure that customers get served and orders processed. My main aim in this example is to track how long it takes to process tickets.

This kind of assumes you already have a Logscape environment running. If you don’t, download it now and get started!

Continue reading

Logscape 3.0 is finally here!

How it felt developer Logscape 3.0The long awaited version 3 update to Logscape is finally here.

Logscape 3.0
In version 3 we welcome multiple new and exciting features. We’ve improved search performance, zoning and the search UI as well as a series of bug fixes. Version 3 is sure to surprise even the most experienced user. This release is bigger, better and faster! If you want to find out all you need to know about these excellent new features, and how this update will affect you, read on.

If you just want to get started, download it for free right now from our Website.

If you’ve already tried it, why not tell us what you think? We’d love to hear your feedback. Simply message us and let us know.

Continue reading