Skip to main content

Data Currency and Exploding Bunnies

There is such a thing as data currency - i.e. how current and up to date the data is. On the web, stale data is a social disease which deservingly leads to isolation and the irritating and distant tut-tut'ing that goes with it - much like zits on a greasy teenager. We're all culpable (me in particular given the last time I updated the mighty but I expect more from The Guardian. So... tut-tut to The Guardian for pressing UK Gov Data from 2010 on their Data home-page today. There was me thinking I'd happen across some interesting nuggets only to find old, stale and consequently misleading data.

Of course I'm not helping by providing a bunch of links to stale content myself but it's time more attention was paid to data currency by net publishers. Perhaps then this wouldn't be such a big problem.

Screen grab below for the hell of it...



From a non-functional perspective an important criteria for data is how current it needs to be. Various caching strategies may; or may not, be viable depending on how critical this is. If you're buying stock then you want the correct price now, if you're browsing the news then perhaps today is sufficient. This affects how deep into your system transactional tentacles reach and the resources that need to be committed to address this. However, the issue on The Guardian site likely relates instead to the algorithms that promote data and how these are either insensitive to the element of time, the subject of data of low velocity (which it isn't in this case) or which are sensitive; and who wouldn't be, to the internet scale viral effect causing excessive temporary popularity. Perhaps content providers need to start saying, "ok, we know it's a funny video of a cat stuck in a washing machine but that was so 2005 and welcome to 2015, so here's a fully interactive 3d experience of a bunny playing with grenades instead"...


Popular posts from this blog

An Observation

Much has changed in the past few years, hell, much has changed in the past few weeks, but that’s another story... and I’ve found a little time on my hands in which to tidy things up. The world of non-functionals has never been so important and yet remains irritatingly ignored by so many - in particular by product owners who seem to think NFRs are nothing more than a tech concern. So if your fancy new product collapses when you get get too many users, is that ok? It’s fair that the engineering team should be asking “how many users are we going to get?”,   or “how many failures can we tolerate?” but the only person who can really answer those questions is the product owner.   The dumb answer to these sort of question is “lots!”, or “none!” because at that point you’ve given carte-blanche to the engineering team to over engineer... and that most likely means it’ll take a hell of a lot longer to deliver and/or cost a hell of a lot more to run. The dumb answer is also “only a couple” and “

Inter-microservice Integrity

A central issue in a microservices environment is how to maintain transactional integrity between services. The scenario is fairly simple. Service A performs some operation which persists data and at the same time raises an event or notifies service B of this action. There's a couple of failure scenarios that raise a problem. Firstly, service B could be unavailable. Does service A rollback or unpick the transaction? What if it's already been committed in A? Do you notify the service consumer of a failure and trigger what could be a cascading failure across the entire service network? Or do you accept long term inconsistency between A & B? Secondly, if service B is available but you don't commit in service A before raising the event then you've told B about something that's not committed... What happens if you then try to commit in A and find you can't? Do you now need to have compensating transactions to tell service B "oops, ignore that previous messag

Equifax Data Breach Due to Failure to Install Patches

"the Equifax data compromise was due to their failure to install the security updates provided in a timely manner." Source: MEDIA ALERT: The Apache Software Foundation Confirms Equifax Data Breach Due to Failure to Install Patches Provided for Apache® Struts™ Exploit : The Apache Software Foundation Blog As simple as that apparently. Keep up to date with patching.