Skip to main content


Showing posts from June, 2014

Government IT Spend

The way that governments are run they'll be a huge amount of duplication and waste in this lot . Situation is even worse when you consider that historically it's mostly proprietary software with n  year support contracts for stuff that's rarely used (but hits the headlines when it is). Not at all surprising. The future for government IT is open-source and cloud based.

Bypassing BT's DNS Service

I suffered from BTs failure yesterday which knocked out many sites though thankfully it didn't seem to affect - phew! What a relief huh? Anyway, BT has now apologised for the incident and is investigating root-cause. Well, feeling lost and detached from reality without full and proper access to the net ( internet access should be a human right ) I naturally did my own investigating which included the obligatory reboots to no avail (my Mac, wife's PC, home-hub) - and you know they'll make you redo these steps if you have to call support... Some sites could be pinged, some couldn't (could not resolve host) which points at a DNS issue. Bypassing BTs DNS isn't that easy though as they have a transparent DNS service in place which means you can't just add Googles free DNS servers to your list ( and if you're interested). Doing this  in my case simply resulted in an error message saying that BT's Parental Controls

Excremental Form

We often think we know what good design is; whether it be system, code or graphic design, and it's a good thing that we strive for perfection. Perfection though is subjective, comes at a cost and is ultimately unachievable. We must embrace the kludges, hacks, work-arounds and other compromises and like the Greek idiom; "whoever is not Greek is barbarian", we should be damn proud of being that little bit barbaric even if we continue to admire the Greeks. The question is not whether the design is good but whether the compromises are justified, sound and fit for purpose. Even shit can have good and bad form.

Eyes on the road

What's it got to do with non-functionals? Hell, I dunno... That you're a single point of failure perhaps?

Chaos Monkey

I've had a number of discussions in the past about how we should be testing failover and recovery procedures on a regular basis - to make sure they work and everyone knows what to do so you're not caught out when it happens for real (which will be at the worst possible moment). Scheduling these tests, even in production, is (or should be) possible at some convenient('ish) time. If you think it isn't then you've already got a resiliency problem (you're out when some component fails) as well as a maintenance problem. I've also talked (ok, muttered) about how a healthy injection of randomness can actually improve stability, resilience and flexibility. Something covered by Nassim Taleb in his book Antifragile . Anyway, beat to the punch again, Netflix developed a tool called Chaos Monkey (aka Simian Army)  a few years back with randomly kills elements of the infrastructure to help identify weak points. Well worth checking out on . For the record

Telco CDNs & Monopolies

Telco CDNs (Content Distribution Networks) are provided by telcos by embedding content caching infrastructure deep in the network close to the end-user (just before the last kM of copper wire). The result is improved streaming to end-users and significantly less load on both the content providers servers and the telcos wider network. It's a win-win-win for everyone. Telcos charge content providers for this service. If the telcos network has a limited client base then perhaps there's not much point in the content provider paying them to cache the content since it'll not reach many end-users. If the telco is a state run (or previously state run) monopoly telco then if you want to make sure your content is delivered in the best quality you'll pay (if you can). The telco could thus be accused of abuse if they are seen to be using a monopoly position to drive ever higher profits through leveraging this sort of technology. It can also be considered an abuse of net neutrality

Resilient WebSphere Session Management

I've been promising myself that I'll write this short piece sometime and since the football today has been a little sluggish I thought I take timeout from the world cup and get on with it... (you know it won't be short either..). Creating applications than can scale horizontally is; in theory, pretty simple. Processing must be parallelizable such that the work can be split amongst all member processors and servers in a cluster. Map-reduce is a common pattern implemented to achieve this. Another; even more common, pattern is the simple request-response mechanism of the web.  It may not sound like it since each request is typically independent from each other, but from a servers perspective it is arguably an example of parallel processing. Map-reduce handles pre-requisites by breaking jobs down into separate map and reduce tasks (fork and join) and chaining multiple map-reduce jobs. The web implements it's own natural scheduling of requests which must be performed in sequ

Cloud Computing Patterns

Found a website today on Cloud Computing Patterns . Bit of a teaser to buy the book really since there's not a great deal of detail on the site itself. Still, a useful inventory of cloud patterns and some nice diagrams  showing how things like workload elasticity  works and the service models; IaaS , PaaS and SaaS , fit together.

Scaling on a budget

Pre-cloud era. You have a decision to make. Do you define your capacity and performance requirements in the belief that you'll build the next top 1000 web-site in the world or start out with the view that you'll likely build a dud which will be lucky to get more than a handful of visits each day? If the former then you'll need to build your own data-centres (redundant globally distributed data-centres). If the latter then you may as well climb into your grave before you start. But most likely you'll go for something in the middle, or rather at the lower end, something which you can afford. The problem comes when your site becomes popular. Worse still, when that popularity is temporary. In most cases you'll suffer something like a slashdot effect for a day or so which will knock you out temporarily but could trash your image permanently. If you started at the higher end then your problems have probably become terminal (at least financially) already. It's a dilemm

Linux! Champion of Big Data

Big data solutions based on distributed databases such as MongoDB (and Hadoop and others) rely on have very many nodes running in parallel to provide resiliency, performance and scalability. This is a step up from the "cluster of 2-nodes" model (primary & failover) used for many legacy SQL installations. Such is simply not big enough to support resiliency with the sort of distributed database model NoSQL solutions provide (even if it could scale). For example, you'll need a minimum of x3 nodes just to allow the election of a primary to work in a replicated cluster and more for sharding using MongoDB. Of course there's a reason why you've chosen a NoSQL solution in the first place - scale - and the choice of horizontal v vertical scaling at these sizes makes sense. This is all good news for Linux since an increase in the number of nodes has costs associated with it which I will likely dictate that Linux will become the OS of choice for such solutions instead of

MongoDB Write Concern Performance

MongoDB is a popular NoSQL database which scales to very significant volumes through sharding and can provide resiliency through replication sets. MongoDB doesn't support the sort or transaction isolation that you might get with a more traditional database (read committed, dirty reads etc.) and works at the document level as an atomic transaction (it's either inserted/updated, or it's not) - you cannot have a transaction spanning multiple documents. What MongoDB does provide is called "Write-Concern" which provides some assurance over whether the transaction was safely written or not. You can store a document and request "acknowledgement" (or not), whether the document was replicated to any replica-sets (for resiliency), whether the document was written to the transaction log etc. There's a very good article on the details of Write-Concern over on the MongoDB site . Clearly the performance will vary depending on the options chosen and the Java driver

Google Maps Usability

Is it me or is the recently revamped Google Maps not as usable as the old one? They've maxed out the map; which is nice, but the menu items are now so obscure you'd not know where they are without flipping endlessly around the map (it's worse on the Android version IMHO). For example, I search for a location, find it, right click, select "directions to here" and it provides directions from that location to itself!? Well ok, I can kind of understand why but it's obviously not what I meant... So I right click on home and it says "What's here?", not "directions from here". To do that I need to type (wtf!) my start location into the drop-down on the left of the screen. Not unusable just irritating... Plus the search menu overlaps the map which can be irritating on smaller screens and seems to slide up-and-down at will as you drag around the map - again, irritating... (but still pretty much the best map service out there...).

WebSockets at The Red Lion

More of a functional one this... I'm working on a personal project at the moment related to some automated diagramming tools I developed some time back. This calls for inter browser communication (between browsers running on different clients) for which web-sockets are the logical choice in these delightful 1 HTML5 days. I've developed a prototype to test out web-sockets in Java using the same connection for both commands (messages between client and server) and for inter browsers comms (messages between clients). It's only a prototype to test that messaging works but if you head over to The Red Lion 2 you may be able to join me online for a virtual pint. Developing web-socket applications in Java is relatively straightforward though the examples online tend to forget about thread blocking by sending messages to clients in sequence (one client blocking could cause delays to other users). The code I've developed avoids this via use of Java concurrency libraries which