Skip to main content

Computation, Data and Connections

My ageing brain sees things in what feels like an overly simplistic and reductionist way. Not exactly the truth so much as a simplified version of reality which I can understand well enough to allow me to function (to a degree).

So in computing I see everything as being composed of three fundamental (overly simplified) elements; computation, data and connections.

Computation refers to those components which do stuff, essentially acting on data to transform it from one form to another. This may be as simple as taking data from a database and returning it as a web page or as complex as processing video data to identify and track movements of individuals.

Data is the information itself which critically has no capability to do anything. Like Newtons first law, data at rest will remain at rest until an external computation acts upon it. Data may be in a database (SQL or NoSQL), file-system, portable USB drive, memory etc. (memory is really just fast temporary data storage).

And connectivity hooks things together. Without some means of connectivity computation can't get a handle on the data and nothing much happens regardless. This may be a physical ethernet cable between devices, logical database driver, a file-system handle etc.

So in every solution I can define each component (physical or logical) as fulfilling one of these three core functions.

The next step from a non-functional perspective is to consider a matrix of these elements and my four classifications for non-functional requirements (security, performance & capacity, operability and form) to discuss the key considerations that are required for each.

For example, backup and recovery is primarily a consideration for data. Computational elements have no data as such (log files for example would be data output from compute components) and so a different strategy can be adopted to recover these (e.g. redundant/resilient instances, configuration management, automated-build/dev-ops capabilities etc.). Like wise, connectivity recovery tends to be fairly manual since connections can traverse many components, any one of which could fail, and so more effort is required to PD (problem determination) and identify root cause. A key requirement to aid this is a full understanding of the connections between components and active monitoring of each so you can pinpoint problems more easily - unfortunately it's naive to think you understand all the connections between components in your systems.

The matrix thus looks something like this:

































ComputationDataConnections
Security Computation v Security
Performance & Capacity
OperabilityRedundancy, configuration management, build automation.Backup, recovery, resiliency.Connectivity model, monitoring.
Form

 

Now I just need to fill the rest of it in so when I work on a new solution I can classify each component and refer to the matrix to identify non-functional aspects to consider. At least that's the theory...

Comments

  1. […] month I wrote a piece about computation, data and connections with a view to starting to list out some considerations for each of these with respect to […]

    ReplyDelete

Post a comment

Popular posts from this blog

An Observation

Much has changed in the past few years, hell, much has changed in the past few weeks, but that’s another story... and I’ve found a little time on my hands in which to tidy things up. The world of non-functionals has never been so important and yet remains irritatingly ignored by so many - in particular by product owners who seem to think NFRs are nothing more than a tech concern. So if your fancy new product collapses when you get get too many users, is that ok? It’s fair that the engineering team should be asking “how many users are we going to get?”,   or “how many failures can we tolerate?” but the only person who can really answer those questions is the product owner.   The dumb answer to these sort of question is “lots!”, or “none!” because at that point you’ve given carte-blanche to the engineering team to over engineer... and that most likely means it’ll take a hell of a lot longer to deliver and/or cost a hell of a lot more to run. The dumb answer is also “only a couple” and “

Inter-microservice Integrity

A central issue in a microservices environment is how to maintain transactional integrity between services. The scenario is fairly simple. Service A performs some operation which persists data and at the same time raises an event or notifies service B of this action. There's a couple of failure scenarios that raise a problem. Firstly, service B could be unavailable. Does service A rollback or unpick the transaction? What if it's already been committed in A? Do you notify the service consumer of a failure and trigger what could be a cascading failure across the entire service network? Or do you accept long term inconsistency between A & B? Secondly, if service B is available but you don't commit in service A before raising the event then you've told B about something that's not committed... What happens if you then try to commit in A and find you can't? Do you now need to have compensating transactions to tell service B "oops, ignore that previous messag

Equifax Data Breach Due to Failure to Install Patches

"the Equifax data compromise was due to their failure to install the security updates provided in a timely manner." Source: MEDIA ALERT: The Apache Software Foundation Confirms Equifax Data Breach Due to Failure to Install Patches Provided for Apache® Struts™ Exploit : The Apache Software Foundation Blog As simple as that apparently. Keep up to date with patching.