Realtime WebSocket streaming from the cloud to you: Part I

This is a 2 part post where iwebSocketClientn Part 1 I build the ‘spike’ using Groovy to run a WebSocketServer to stream data to HTML5-WebSocket & JavaWebSocket Clients. The HTML Client uses the elegant smoothie charts (great for streaming). In Part 2 Ill show you how to run it on Amazons AWS.
At the end we have a real-time feed plotting the data from the cloud; it looks something like the grab on the right.

Networkingly challenged!

One of the challenges with distributed computing, particularly cloud computing is networking. In recent work we have been looking at running services in the cloud but we really need the ability to stream data back to our local servers (we want to keep it). Conventional wisdom says ‘You need a server and the client will send data to it’. In our case this isnt going to work – our servers are nicely wrapped up behind firewalls, switches and associated rules (security blanket). To get the appropriate client-ip holes punched would be a 12-24 hour wait; we are pretty impatient J.

WebSockets to the rescue

Now we rely heavily on websockets in Logscape – they are great; the decoupled nature brings about many benefits (and a little pain). The nice thing about the API is that you can call onMessage either from the client->server or server->client (full duplex). In this case we can run a server in the cloud and stream its data to the client located within our local estate. A JavaClient can receive the feed and do whatever is necessary; pump it into a queue, disk, machine learning; or just keep it for later analysis. After the cloud servers has gone the data resides (yes we could store it in S3).  Learn more about WebSockets

A Groovy WebSocketServer

With a Java background, groovy can make your life pretty easy if you need to whip up some simple classes. For WebSocket support Jetty does the trick. Below is a cut-down version of the WebSocketServer.

class TheWebSocket implements WebSocket.OnTextMessage {
 public TheWebSocket(List<TheWebSocket> allClients) {
 }
 public void onMessage(String json) {
 }
 public void onOpen(Connection connection) {
 }
 public void onClose(int i, String s) {
 }
 public void send(String msg) {
    connection.sendMessage(msg)
 }
}
class WsHandler extends WebSocketHandler {
 public WsHandler(List<TheWebSocket> allClients) {
 }

 public WebSocket doWebSocketConnect(HttpServletRequest r, String s) {
   return new TheWebSocket(allClients);
 }

}
class DefaultHttpHandler extends AbstractHandler {
 public void handle(String s, Request req, HttpServletRequest httpServletRequest, HttpServletResponse res)  {
 }
}
def server = new Server();
def connector = new SelectChannelConnector();
connector.setPort(20000);
connector.setHost("0.0.0.0");
server.addConnector(connector);

def WsHandler webSocketHandler = new WsHandler(allClients);
webSocketHandler.setHandler(new DefaultHttpHandler());
server.setHandler(webSocketHandler);

Firing up the Server

Before running the server we need to send data; the following snippet does this:

executor.scheduleAtFixedRate(new Runnable() { 
 public void run() {
 for (client in allClients) {
 client.send("[{ TIME: " + new Date().getTime() + " , CPU:" + Math.random() + " }]")
 }
 }
 }
 ,1, 1, TimeUnit.SECONDS)

As each client connects, it adds itself to a collection called allClients. The scheduler pumps data into each one.

Inside: webSocketServer.sh (SCRIPT==webSocketServer.groovy)

java -Xms32m -Xmx32m -cp "$CP" groovy.lang.GroovyShell $SCRIPT

Console:

alteredcarbon:WebSocketTest neil$ ./webSocketServer.sh
2014-06-13 09:47:53.055:INFO:oejs.Server:jetty-8.1.9.v20130131
2014-06-13 09:47:53.094:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:20000

WebSocket Groovy Client

For the Java/GroovyClient we are using a github lib. There isn’t much to it, establish a connection and ‘doStuff’ when onMessage() events occur.

class TheWebSocketClient extends WebSocketClient {
 public TheWebSocketClient(URI uri, Connection connection) {
    super(uri, new org.java_websocket.drafts.Draft_17()) 
 }
 void onOpen(ServerHandshake h1) {
 }

 void onMessage(String p1) {
   println("doStuff - parse json etc:" + p1)
 }
 void onClose(int p1, String p2, boolean p3) {
   connection.close()
 }
 void onError(Exception p1) {
   println("onError:" + p1)
 }
}
def client = new TheWebSocketClient(new URI("ws://localhost:20000"), null)
client.connect()

The Streaming HTML Smoothie Chart

Getting the HTML page working is dead simple. The internet is littered with examples, so putting it together is a walk in the park. I modified the sample code to keep retrying the connection until it succeeds.

function testWebSocket() {
 websocket = new WebSocket(wsUri);
 websocket.onopen = function (evt) {
 onOpen(evt)
 };
 websocket.onclose = function (evt) {
 onClose(evt)
 };
 websocket.onmessage = function (evt) {
 onMessage(evt)
 };
 websocket.onerror = function (evt) {
 onError(evt)
 connected = false

When the window ‘load’ event fires we fire up the webSocket

window.addEventListener("load", init, false);

When onMessage(event) fires we convert the String to JSON (eval) and pop it into Smoothie charts (see line1); passing in the TIME(long) and CPU(float) values:

function onMessage(evt) {
 if (added++ > 0 && added % 10 == 0) {
 document.getElementById("output").innerHTML = ''
 }
 writeToScreen('<span style="color: blue;">RESPONSE: ' + evt.data + '</span>');
 var data = eval(evt.data)[0]
 line1.append(data.TIME, data.CPU);
}

Smoothie charts are amazing simple; insert js ala:

var line1 = new TimeSeries();
var smoothie = new SmoothieChart({ grid: { strokeStyle: 'rgb(125, 0, 0)', fillStyle: 'rgb(60, 0, 0)', lineWidth: 1, millisPerLine: 250, verticalSections: 6 } });
 smoothie.addTimeSeries(line1, { strokeStyle: 'rgb(0, 255, 0)', fillStyle: 'rgba(0, 255, 0, 0.4)', lineWidth: 3 });
smoothie.streamTo(document.getElementById("smoothieCanvas"), 1000);

The Result….

webSocketClient

Every 10 lines the RAW output is reset and the chart updates in real-time; nice!

Don’t forget to follow us on twitter for tips, videos and tutorials!


Regards, Neil