Skip to main content


Showing posts from November, 2016

Scalable = Horizontal

There's two ways of scaling; vertical and horizontal, but there's only one which is really scalable. Vertical scaling essentially means bigger nodes. If you've got 8GB RAM, go to 16GB. If you've 2 cores, go to 4.. and so on. Horizontal scaling means adding more nodes. One node to two nodes, to three and so on. As a rule, horizontal scaling is good. Theoretically there's no limit to the number of nodes you can have. As a rule, vertical scaling is bad. You quickly run into constraints over the number of cores or RAM you can support. And for many of todays problems this just doesn't work. Solutions need to be both scalable at the internet scale and available as in 24x7. Relying on large single nodes in such situations is not ideal. (and those supercomputers with 250,000+ processors are really horizontal solutions as well). The problem is, horizontal scaling isn't trivial. The culprits here are data and networking (plumbing really). State and caches need to be d

Instrumentation as a 1st Class Citizen

I wrote previously that we are  moving into an era of instrumentation  and things are indeed improving. Just not as fast as I'd like. There's a lot of good stuff out there to support instrumentation and monitoring including the likes of the ELK ( ElasticSearch , Logstash , Kibana ) and TIG ( Telegraf , InfluxDB , Grafana ) stacks as well as their more commercial offerings such as TICK (Telegraf, InfluxDB, Chronograf , Kapacitor ), Splunk , DataDog , AppDynamics and others. The problem is, few still really treat instrumentation as a real concern... until it's too late. Your customers love this stuff! Really, they do! There's nothing quite as sexy as an interactive graph showing how your application is performing as the load increases - transactions, visitors, response-times, server utilisation, queue-depths etc. When things are going well it gives everyone a warm fuzzy feeling that all is right with the universe. When things are going wrong it helps to quickly focus you