WebSockets at The Red Lion

More of a functional one this… I’m working on a personal project at the moment related to some automated diagramming tools I developed some time back. This calls for inter browser communication (between browsers running on different clients) for which web-sockets are the logical choice in these delightful1 HTML5 days. I’ve developed a prototype to test out web-sockets in Java using the same connection for both commands (messages between client and server) and for inter browsers comms (messages between clients).

It’s only a prototype to test that messaging works but if you head over to The Red Lion2 you may be able to join me online for a virtual pint.

Developing web-socket applications in Java is relatively straightforward though the examples online tend to forget about thread blocking by sending messages to clients in sequence (one client blocking could cause delays to other users). The code I’ve developed avoids this via use of Java concurrency libraries which is fine except that it violates JEE rules by spawning it’s own threads. It’s currently limited to a single server and doesn’t support a clustered environment which would require either a persistent datastore (less than desirable) or server-server comms (which I’d prefer). I did some test on the LAN to pump messages pretty quickly through the tiny Atom server I have running and it seemed to hold up to the task. Interestingly Chrome started to lock up though IE, FF and Safari seemed to cope. I should try and get some stats on this.

As a prototype there is no authentication (feel free to lie) and no logging of message content (you’ll have to trust me on that one or check out the code).

In case you’re interested, the code is on github.com,  no warranties (as ever).

1 If you’ve developed in a pre-HTML5 world then you’ll know what I mean… If you’re still having to support non HTML5 compliant browsers then.. sorry 🙁
2 The Red Lion is the most common pub name in the UK.

LocalServe

One of the things I have found irritating in the past is the need to install and configure a web-server each time the urge takes me to try something out. I don’t run a local web-server permanently and being of a JFDI disposition the hurdle needed to get a server running is usually enough to stall what motivation I’ve managed to muster. Then I discovered that from Java 7 onwards it’s fairly simple to implement your own web-server in plain Java – no need for an application server.

LocalServe implements two types of handlers:

1. File-handler –  This serves any content (and sub-directories) in the directory in which localserve is run. Any file which is not found returns a 404 and  any request for the root of a folder (path ending in “/”) attempts to return the index.html file in the directory. Note that localserve does not provide any listing of directories.

If all you want to do is serve static content then the above is sufficient and LocalServe can be run using the command below in the directory you want to serve content from. This will run a webserver on port 8765 by default:

java -jar localserve.jar

The port number can also be changed by adding this to the end – e.g.:

java -jar localserve.jar 5678

2. SQL-handler – Often static content isn’t enough and you need to use a database. This handler provides access to a database that can be called from JavaScript (typically via an AJAX request). A configuration file can be specified on the command line when running localserve. This configuration file provides details of a SQLite database, and the SQL statements that are supported. Each SQL statement has a logical name, SQL statement (including bindings), supported methods (POST or GET) and optionally a redirect (where to send the user on success). Calls to paths starting “/sql/” are sent to the SQL handler and the path element after this is used to match against a logical name in the configuration file. If found the SQL statement is executed with any HTTP parameters matching the bind names being bound accordingly. Two special names “PAGE” and “PAGE_SIZE” are defined such that queries which may return many many rows can be restricted to returning only certain pages of a certain size. Results from SQL commands are returned in JSON format.

The configuration file can be specified on the command line when running localserve as below:

java -jar localserve.jar config.json
or to run on a port other than 8765:
java -jar localserve.jar config.json 5678

(note that the position of these two parameters doesn’t matter).

An example configuration file is shown below:

{ "wsm": {
    "connString": "jdbc:sqlite:/path/to/database/file.db",
    "driver": "org.sqlite.JDBC",
    "operations": [
        { "path": "listbob", 
          "statement": "select name, id from bob", 
          "methods": "GET,POST" },
        { "path": "querybob", 
          "statement": "select id, name from bob where id={ID} order by name desc", 
          "methods": "GET,POST" },
        { "path": "insertbob", 
          "statement": "insert into bob (id, name) values ({ID}, {N})", 
          "redirect": "/sql/querybob", 
          "methods": "GET,POST" }
    ]
}
}

The database here contains one very simple table (BOB) as:

CREATE TABLE BOB (ID VARCHAR(20), NAME VARCHAR(100));

The only database used is SQLite and the main JAR file contains all libraries for this to work. I have tried other databases (notably IBM DB2) which worked fine so long as the JAR libraries can be found.

An example response to something like http://localhost:8765/sql/listbob looks like:

{ "dataset": 
    {  "page": 0, 
       "pageSize": 20,  
       "record": [  
           { "name": "James Brown", "id": "1"} ,  
           { "name": "Simple Simon", "id": "2"} ,  
           { "name": "Ducky Duncan", "id": "3"}  
       ] 
    }
} 

The attribute names are derived from the query names and are usually lower case. HOWEVER, you may find that if you explicitly state a column name the attribute may come back in uppercase (e.g. “select id||’ – ‘||name LONG_NAME from bob” will result in an attribute with name “LONG_NAME”).

Once you have the database setup and working then it’s a relatively simple task to use JQuery to submit AJAX requests to the SQL handler to create/read/update/delete/list the database. A hastily knocked up example is below:

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> 
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/> 
    <meta charset="utf-8"/>  
  
    <script src="jquery-2.0.3.min.js"></script>
    <script src="purl.js"></script>
    </head>
  <body>
      ...
  <script type="text/javascript">
      
      function query(searchId) {
        $.ajax({ url: "/sql/querybob",
            data: { "PAGE": 0, "PAGE_SIZE": 32000, "ID": searchId },
            success: function( data ) {
              var dataset = (JSON.parse(data)).dataset;
              var arr = new Array();
              for (var i=0;i<dataset.record.length;i++) {
                alert(dataset.record[i].id + " " + dataset.record[i].name);
          }
        });
      }
      query(1);
    </script>
  </body>
</html>

Anyway, it’s been useful when I just want to prototype something and this is definitely not intended for any production use. It’s simply a very simple webserver so that I can quickly get on with prototyping something.

If you’re interested then the source can be found on GitHub where the LocalServe JAR file can be downloaded. The code is what I call “prototype quality” which means it’s been made to work by beating it into some shape with a hammer – it is uncommented and not of production quality.

Java 7 and this JAR are all you should need to run LocalServe. As ever, no assurances, warranties, guarantees etc. are provided and; whether you lose a little data or the world goes into meltdown (and everything in-between), I’ll accept no responsibility for any damages caused…

Reuse

Reuse! My favourite subject. Now, are you sitting comfortably? Then I’ll begin…

Once upon a time in a land far far away, the king of computer-land was worried. Very worried indeed. His silly prime minister had borrowed lots and lots of money to build lots of new computer systems and programs and now they couldn’t pay the interest on the debt. Worse still, none of the systems worked together and the land was becoming a confusing mess and there were lots of traffic jams and angry people. No-one knew how it worked and everything kept breaking. It was very expensive and it didn’t work. The villagers were not happy and were threatening to chop the heads off the king and queen because they were French.

Then one day, a strange young prince claiming to be from a neighbouring country arrived bringing promises to sort out all the mess and clean the country up. And what’s more, he’d do it cheaply and the country would have more money and better computer systems. The king and queen were very happy and the villagers were pleased as well – although they still want to chop the heads of the king and queen, because they were French and it would be fun.

So they listened to the prince and liked his ideas and gave him all the money they had left. The prince was going to build even more computer systems but they would all be based on the same design so would be cheap and quick to build. This meant he could spend more money on the design so it would be very good as well as cheap to build.

Then the prince said that he could also make a large hotel and everyone could live under the same roof. This would save on roofs because there would only be one and would be cheaper to run because there would only be one electricity bill. The villagers liked this because they liked going on holiday. The king and queen liked this because they had decided to go on holiday and so the villagers could not chop off their heads even though they were French.

Then the prince started to design the computer systems. He decided to start with the post-box because everyone sent letters. So he spoke to Granny Smith and Mrs Chatterbox about what they needed. They liked his design. It was round and red and pretty – it looked a bit like the old post-boxes.

Then he spoke to the bookshop keeper who didn’t like his design because it was too small for him to post a book. So the prince made it bigger, much bigger.

Then he spoke to the postman who didn’t like it because it was too big and would give him too many parcels to carry but the prince decided to ignore the postman because he was clearly an idiot.

So two of the postboxes were built; one in case the other was full, and the villagers liked them a lot even though the postman did not.

Next the prince decided to build the hotel so asked the villagers how they would like their room to look; because there could only be one design. Some wanted it round, some square, some with a balcony, some with stairs… and everyone want an en-suite with bidet even if they did not know how to use it. So the prince designed a flexible framework consisting of transformable panels which could be positioned wherever the villager chose. No-one liked the tents and the bidet was missing. The villagers were very angry and started to build a guillotine because they were French.

Then some of the villagers started to place their tents at the entrance to the hotel so they could get out quickly. But this stopped other villagers from coming in so made them angry. Then another villager blocked the toilet and all the villagers were angry and the hotel staff decided to go on strike because they were French and they hadn’t had a strike yet.

So the villagers decided to summon the king and queen to come back from holiday and sort out the mess. So they each sent a letter recorded delivery. But the postbox didn’t understand what “recorded delivery” meant because it was just a big round red box and and postman didn’t want to pick up all the letters anyway because there were too many to carry and they hadn’t paid the postage. So the king and queen didn’t return to sort out the mess and the villagers were apoplectic with rage.

So the villagers burnt all the tents and drowned the postman and the prince in the river. Then the king and queen returned from holiday to find the city on fire and lots of angry villagers carrying pitchforks and pointing to a guillotine. But the king and queen were fat and so couldn’t run away. So the villagers decided to form a republic and elected the prime-minister to become the president. The president chopped off the heads of the king and queen and the villagers were happy so gave a gallic shrug; because they were French, and lived happily for the next week or so…

All of which begs the question… what’s this got to do with reuse?

Well, two things.

  1. Design reuse requires good design to be successful. And for the design to be good there must be lots of consistent requirements driving it. All too often reuse programs are based on the notion of “build it and they will come” where a solution is built for a hypothetical problem which it’s believed many requirements face. Often the requirements don’t align and a lot of money is spent designing a multi-functional beast which tries; and often fails, to do too much which increases complexity which increases cost. The additional effort needed to consider multiple requirements from disparate systems significantly increases design, build and maintenance costs.  To make this worse, true cases of reuse are often common problems in the wider industry and so industry standard solutions and design patterns may exist which have been thought out by smarter people than you or me. To tackle these in-house is tantamount to redesigning the wheel… generally badly.
  2. Instance reuse sounds like a great idea – you can save on licenses, on servers and other resources – but this creates undesirable dependencies which are costly to resolve and act to slow delivery and reduce ease of maintenance. Furthermore, resource savings are often limited as you’ll only save on a narrow portion of the overall requirements – you’ll need more compute, more memory and more storage. Getting many parties to agree to changes is also time-consuming and consequently costly and makes management of the sum more of a headache than it need be.

Personally I believe if you’re going to progress a reusable asset program you need to validate that there really exists multiple candidate usage scenarios (essentially  the cost of designing and building a reusable asset must be less than cost of designing and building n assets individually), that requirements are consistent and that you’re not reinventing the wheel. If this is the case, then go for it. Alternatively you may find an asset harvesting program to review and harvest “good” assets may yield better results; technically as well as being more efficient and cost effective. Then there’s the view that all reuse is opportunistic in so much as using something designed to be “re”used is really just “use”, and not “reuse” – as I once noted, “wearing clean underpants is ‘use’, turning them inside out and back to front is ‘reuse'”.

In terms of instance reuse, in my view it’s often not worth saving a few licenses given the headaches that results from increased dependencies between what should be independent components. The problem is complicated with hardware, rack space and power consumption so is often not clear and some compromise is needed. However, the silver bullet here is virtualisation where a hypervisor can allocate and share resources out dynamically allowing you to squeeze many virtual machines onto one physical machine. License agreements may allow licensing at the physical CPU level instead of virtual CPU which can then be over allocated so you can have many guest instances running on fewer host processors. This isn’t always the case of course and the opposite may be cheaper so this needs careful review of licensing and other solution costs.

Reuse isn’t always a good idea and the complexities needed in design and build and additional dependencies resulting may outweigh the costs of just doing it n times in the first place. Use of standards, design patterns and harvesting of good assets should be the first  front in trying to improve quality and reduce costs . Any justification for creating reusable assets should include a comparative estimates of the costs involved; including ongoing cost to operations.