In my ever onward quest to show to the world how easy it is to get up and started with Logscape, today I’m going to use a Logscape docker container in order to build visualisations based off some publicly available CSV files in no time at all. If you’ve never used the Logscape docker image, then check out my previous blog.
Here at Logscape it should go without saying that monitoring is sort of a big deal. Some would even go as far as to say it’s even our “thing”. To go with that we’ve collated a collection, of what we think might be the best 10 monitoring talks people should watch. Regardless of whether you’re looking to implement a logging tool, build your own or are just a developer, these talks are worth the time.
Monitoring at Spotify – When things go ping in the night. – by Martin Parm
This non-tech talk covers how logging at Spotify has adapted over the years. From the days where monitoring what just a gleam in the developer’s eyes, to the months where operations slept restlessly fearing the inevitable phone call, to the current day monitoring system which act as caretakers to the infrastructure responsible for streaming hundreds of thousands of tracks a day.
Metrics, Metrics everywhere – by Coda Hale
Digging back into the archives, we have Coda Hale’s talk from 2011. While you may think it’s dated, and irrelevant by todays standards. Coda covers some topics which will simply never get old.
Monitoring is Dead. Long Live Monitoring. – by Greg Poirier
Greg Poirer thinks it’s time we stop viewing its metrics in isolation and declare things alive or dead. In this laid back high-energy talk, he covers his opinion on the definition of monitoring, with only a few jabs at DevOps. A must watch.
Better living through statistics. Monitoring doesn’t have to suck. – by Jamie Wilkinson
Jamie Wilkinson goes over what he believes to be the problems in how we currently monitor, and how we can get rid of these problems so that everyone can benefit from logging. A great talk that discusses why our current logs simply aren’t precise enough.
The art of performance monitoring. – by Brian Smith
Brian Smith covers the mistakes that he’s made, and the mistakes he keeps seeing developers making. For a 25 minute talk the sheer quantity of technical ideas conveyed in this talk is impressive, but not for the faint of heart.
What should I monitor, and how should I do it? – by Baron Schwartz
In this talk, Baron criticises our approach to monitoring, which is to just stare at a graph and attempt to determine what’s gone wrong. Baron highlights the importance of not just collecting data, but collecting actional data.
Creating a Culture of Observability. – by Cory Watson
Taking a leaf out of the Spotify playbook, this talk covers the culture around logging, rather than logging itself. It’s Cory’s own story of how after joining Strike he managed to instil a culture of observability and monitoring. You’ll hear about his journey towards that goal, with the good, the bad and the downright sneaky.
How monitoring works at scale. – by Ran Leibman
Facebook manage to claim another spot on this top list, this time discussing the challenge which is monitoring the huge amount of infrastructure that make up Facebook around the world. Most companies dont have to monitor on this scale, which leaves an obvious questions, “How exactly do you?”.
The evolution of monitoring systems at google. – by Tom Rippy
Following in the steps of Facebook, Google returns for its second spot in the list. Much like the Spotify talk above, Tony Rippy aims to walk us through the progression of monitoring at Google, and includes some facts that you just wouldn’t believe about the now Tech-Giant.Tony wasn’t present for the whole of this history, but it’s portrayed in a fun and interesting way, which demonstrate whilst your current monitoring solution may not be the best, it doesn’t mean you can’t progress.
Allison McKnights talk demonstrates how monitoring doesn’t have to be expensive. Hailing from Etsy which is known for doing a lot with not a lot. Allison walks us through her experiences of using open source projects to build a monitoring system capable of monitoring the entirety of Etsy’s back end.
Hopefully you’ve enjoyed the videos, if you have any that you feel should be added to the list, feel free to drop them below, or tweet @logscape with why.
It’s finally that day, Logscape is now on docker hub. As such I’m going to be walking you through the process of getting Logscape running, and once you’ve got the hang of it, you’ll be able to download, run and start using Logscape all within 60 seconds. Monitoring in a heart beat. Continue reading
Logscape version 3.2 is now available for public download, you can get it now from the Logscape Website.
A brief rundown of Logscape 3.2 brings with it, and what we’re going to cover today…
- File Explorer
- JSON Support (Including JSON Arrays)
- Failover Overhaul
- Performance and Stability Changes
Logscape Analytics’ are incredibly powerful, however, are you using them to their full potential? In this blog post we’re going to go over some of the less used analytics, show you how to use them, and hopefully inspire you to use your Logscape instance in new and exciting ways. So, without further ado let’s get into some searches. Continue reading
Recently we’ve been working on creating new learning materials for the release of Logscape 3.0.Materials appropriate for both the Logscape expert and an individual just picking Logscape up for the first time. The first person to be addressed by this was of cof course the beginner, as such here’s a 10 minute introduction to the basics of Logscape 3.0.
Hopefully this help some of our newer users, and keep an eye out for more advanced tutorials!
New in Logscape 3.0
Logscape 3.0 introduces a new feature that makes working with CSV data easier, and faster. Logscape will now automatically generate a datatype from imported CSV data, you’ll be free to immediately build a workspace around your data rather than having to worry about setting up your datatype. Continue reading
So you have written an app or log – it’s brilliant, it grabs all the data you need and runs like greased lightning. All you need to do now is ensure your output file has a nice clean format – preferably one that means Logscape does all the work for you! So here are some of my top tips.
1) Add a full time stamp to every line. You wouldn’t believe how much trouble can be caused by people using just times or dates. At the best, you have to struggle to get your data properly organised. At worst, you end up with a mess and data appears in the wrong place on the graph. Do it right, set the date and time!
2) Add a time zone to that stamp. My computer will never move time-zone, surely it’ll be fine? Don’t count on it. British Summer Time changing the system time on half your servers, servers being reset to US time, data centres moving locations… All these things can and will happen. Adding the time zone to the stamp gives you a cast iron assurance that the data will always be correct. That peace of mind is worth a few bytes.
Docker 1.5 came out a few weeks ago and with it the new stats api arrived. Before 1.5 there was no standard way to collect the metrics of running docker containers without writing custom scripts to parse files stored in the proc memory file system.
Release 2.5.1 includes enhancements to JMX functionality, improvements to Logscape roll detection aswell as significant search performance optimisation.
2.5.1 Introduces new linking functionality. This allows you to flawlessly link your workspaces and transfer context between them providing a troubleshooting workflow for non-expert users of the Logscape system.
The release notes are here.