A little chit-chat

It's been a while since I posted anything so  I thought I'd write an article on the pros and cons of chatty interfaces...

When you're hosting a sleep-over for the kids and struggling to get them all to sleep you'll no doubt hear that plaintiff cry "I'm thirsty!", usually following in quick succession by 20 other "Me too!"'s...

What you don't do is walk down the stairs, fetch a glass from the cupboard, fill it with milk from the fridge and take it back upstairs... then repeat 20 more times - you may achieve your steps-goal for the day but it'll take far too long. Instead you collect all the orders, go downstairs once, fill 21 glasses and take them all back upstairs on a tray (or more likely just give them the bottle and let them fight it out).

And so it is with a client-server interface. If you've a collection of objects on a client (say customer records) and you wanted to get some more info on each one (such as the address) you'd be better off asking the server the question "can you give me the addresses for customers x, y and z?" in one go rather than doing it three times. The repeated latency, transit time and processing by the server may appear fine for a few records but will very quickly deteriorate as the number of records rises. And if you've many clients doing this sort of thing you can get contention for resources server side which breaks things for all users - even those not having a wee natter...

Is chatty always bad?

It comes down to the atomicity of requests, responsibility and feedback.

Atomicity...

If it's meaningful to ask a question like "can you give me the address for customer x, account-balance for customer y and shoe-size for customer z?" then by all means do so, but most likely that's really three separate questions and you'll get better reuse and maintainability if you split them out. Define services to address meaningful questions at the appropriate granularity - what is "appropriate" is for you work out...

Responsibility...

You could break the requests out server side and have a single request which collects lots of data and returns this at once rather than having lots of small requests go back-and-forth. This model is common for web-pages which contain data from multiple sources - one requests for the page, server collects data from sources and combines this in one response back to the user. Problem is, if any one of the the sources is performing badly (or down) the whole page suffers and your server chokes in sympathy as part of a cascading failure. So is it the responsibility of the server to collate the data and provide a unified view (often it is) or can that responsibility be pushed to the client (equally often it can be)? If it can be pushed to the client then you can actually improve performance and reliability through more chatty round-trips. Be lazy, do-one-thing-well, and delegate responsibility as much as possible...

Feedback...

Your end-user is a grumpy and impatient piece of wetware incapable of acting repetitiously who seeks instant gratification with minimal effort and is liable to throwing the tantrums of a two year old when asked to "Please wait...". God help us all if you send them an actual error... To improve usability you can trigger little round-trip requests to validate data asynchronously as they go and keep the user informed. This feedback is often enough to keep the user happy - all children like attention - and avoids wasted effort. Filling in a 20 page form only to have it rejected because of some cryptic checkbox on page 3 isn't going to win you any friends. And yet it is still o' so common... I generally work server-side and it's important to remember why we're here - computers exist for the benefit of mankind. User interface development is so much more complicated than server-side and yet simplicity for the user has to be the goal.

So chatty is generally bad for performance and reliability though it can though be good for UX and depending on where the responsibility sits, can improve things overall.

However, as we move towards a micro-services based environment the chatter is going to get louder and more disruptive. Understanding the nature and volumetrics of the questions that are asked of services and ensuring they are designed to address these at the correct granularity will help keep the chatter down whilst making the code more maintainable and reusable. As always, it's never black-and-white...

 

 

ForgeRock License Confusion

IANAL which doesn't help me understand the ForgeRock license situation which looks to be as clear as mud.

ForgeRock states that you need to buy a "subscription" for production.

Blogs like this suggest I'm not the only one to be confused.

Wikipedias page on ForgeRock suggests it's only the binaries you need a license for (if you compile yourself then is this ok?).

And the OpenAM project site suggests two licenses apply. Though one of these (the first one) states "This License represents the complete agreement concerning subject matter hereof" so how can they then go on to apply another "non-commerical" Creative Commons license constraint?

Joy! :(

Don't Build Your Own Security Solution!

There are only three reasons why building your own security solution is a good idea:

1. Security through obscurity - There's less likelihood that anyone will find the holes because no-one else is using it. This is just as well since your home-grown solution is probably riddled with them (see below).


2. Risk v Cost - You've weighed the risk, baulked at the cost and decided the benefits aren't affordable.


3. You're unique! - Some organisations may be unique and/or have unique problems for which there is not an off-the-shelf solution. You may be NASA or a top secret government department, or you could be one of the big internet organisations operating at the edge of technology and scale (fb, Google or the like) or it may just be your business to develop security products.

It's a fair bet that #3 doesn't apply to you and if you think the costs are too high then I suspect you've not really understood the problem.

There are many reasons why you shouldn't build your own security solution and should buy off-the-shelf instead:

1. Risk - The suppliers of security product are much more likely to understand the potential security issues and deal with them correctly than your in-house developers. You may have some really nice people working for you but despite their assurances, they're not the experts. The most basic of attack vectors; weak passwords, code-injection, CSRF, XSS etc., can be left exposed by an in-house team who don't have the experience or knowledge to address them correctly. This point really can't be stressed enough. I do not consider myself to be a security expert but I have seen problems in every single home-grown solution I've come across.


2. Features - Even basic administration capabilities require effort to implement - adding users, granting permissions, password resets, password rotation, revoking access etc. Add to this the expanding list of security patterns and protocols which may need to be supported (basic-auth, forms-based-auth, single-sign-on, WS-Sec, SAML, OAuth, multi-factor authentication etc.) and the range of vulnerabilities you need to address (buffer-overflow, virus scanning, intrusion detection etc.) and a home-grown effort can start to look pretty pathetic. Standards also evolve to improve quality and make interoperability easier and many off-the-shelf products will implement these accordingly. A home-grown effort likely won't efficiently lend itself to being manipulated to support such standards as they evolve - in either a qualitative or interoperable way.


3. Swiss cheese - Buying an off the shelf solution isn't going to solve your problems overnight and your developers will still need to employ secure development practices in case something gets through. However with many layers of security (especially where from a variety of suppliers) comes additional protection. Every product will have holes but you hope the holes don't all line up allowing something through. The more layers the better though home-grown layers have lots of holes.


4. Support - As new vulnerabilities are discovered you'll have (should have) a support contract in place with the supplier to provide patches to address these as they arise. With an in-house solution you'll probably never know you've a problem until it's too late. Furthermore, the obscurity benefit is a millstone around your neck. Finding developers to support the solution will become increasingly difficult (and expensive) as the years go by. No developer wants to be left supporting pre-historic software, it's a dead-end career role.


5. Maintenance - As technology marches on, the environment in which you operate changes. Constant upgrades in browsers, operating systems and devices means you've got to continue to develop solutions just to remain compatible with your users. The time and costs this consumes can quickly mount. The same issue can be charged at commercial vendors and you should ensure your supplier has the mind to consider the upgrade path before you sign on the dotted line.


6. Blame! - Ultimately if something goes wrong you'll have someone to shout at, sue and/or get emergency fixes from. You can still do the same for in-house solutions, it's just that you'll be the one being screamed at and if the breach is bad enough you could well be scanning jobserve sooner than you wanted.


7. Cost - If you can afford to fund development to address the risks, develop, maintain and support a fully featured product then by all means do so. Odds are that you can't.

I could (and do) levy many of these issues at any home-grown solution and not just security but I fear security is the most heinous case as it's taking risks with other peoples data.

Ultimately we need to ask the fundamental question as to whether it's our business to be writing some software product or whether we should focus on our own specific goals and let someone else, someone more qualified, do that for us.

In the case of security it's almost always better to buy off the shelf.

ECB Mode Encryption Deficiencies Explained

 

For the record, I'm not sure where the original came from but there's a copy on Stack Exchange; "Why shouldn't I use ECB encryption", and the ECB penguin clearly has history as noted on PyTux...

Useful article on granting external access to AWS resources

How to use external IDs when granting access to your AWS resources...

Sleep-walking to a Bloody Revolution

According to The Guardian, the EU is about to tighten legislation on user tracking to apply to other means than just cookies.

However, no-one reads those cookie popups or EULAs anyway and you can't do much on the net if you don't blindly accept them. And I mean "blindly" since even though you could read them (most of us don't), you can assume that most users don't have a combined law and computing degree sufficient to be able to understand the implications of them regardless.

I like to think that I understand the computing aspects of these things but I also know that new techniques are developed daily to try to leverage more revenue from the end-user and place more responsibility on them than I can keep up with (or have the mental capacity to understand).

The  likes of Google, Twitter and Facebook are continually trying to capture more data about us and do so through the development of what appears to be "free" services. These services may seem nice but; as always, there's no such thing as a free lunch. They're making a lot more out of the combined mass of data they're collecting on us than we get out of it - after all, they're not charities, they're commercial organisations.

They're not interested in developing this stuff for your benefit; I wouldn't expect them to, and whilst we seem to talk about how relaxed sheep people have become about privacy online, at the same time these organisations have become that much tighter in how they share the data they have on us - individually and in aggregate form (anonymised preferably).

They give you free email or chat services, you let them spy on you so they can feed you with the right ad at the right time in the right place to milk the maximum revenue they can out of you. The algorithms are tuned to this model and they're getting better. We'll soon be letting them install a camera in the bedroom for the benefit of a free daily cappuccino...

So those seeking to extend existing legislation to cover alternative means of tracking which simply relies on "valid consent from the user" are part of the problem. They provide a façade of transparency where there isn't any. They are the lawyers and computer experts who understand the scope of the possible and should be defining the law such as to make some of these techniques illegal without sufficient transparency in what data is captured, how it is used and who it will be shared with.

Ultimately, we need to be paid for the value of the data we provide to these organisations to offset what is becoming a serious discrepancy between the data-rich and the data-poor. The use of vast amounts of data by so few is increasing the imbalance between rich and poor and ultimately will be a disaster for the real economy. That intelligence organisations across the world want to make use of this data themselves and that market leaning governments are ideologically crippled to the point of inaction means we'll not do anything about this until it becomes a real problem - yet another bloody revolution is on the horizon. To address this the law needs to change and no popup is going to help.

Amazon v Amazon

I was reading up a while back about how Amazon (and others) do differential pricing depending on who you are (or who they think you are) including factors such as the type of computer you use. As a very quick test I decided to do two searches...

One (let's call it "open") from my own laptop via my default browser used for everyday activity, logged in to Amazon, full of tracking cookies and history etc....

The other ("closed") from a TOR'ified Linux VM running a clean browser, privoxy and appearing (for now) to be surfacing out of Liberia.

Both searches were for "macbook air" (not that I intend on buying one but they're common and relatively expensive).

Results were pretty much the same, but not quite and the example below shows a difference...

From "open":

 

From "closed":

£686 v £669...

So, if proof were needed.... you can't get Amazon Prime in Liberia! (at least not from amazon.co.uk)... Because that's really the only difference. You can still buy the device (new) for £669 on the "open" version, it's just not the "obvious" option.

However, the first item in the "closed" search did suggest purchase via Prime as it did on the "open" version...



So there is a difference, but how significant is debatable. It may be a classic diversionary sales tactic - the top item via Prime is more expensive than another via Prime lower in the list but this one (which they want you to buy) is still more expensive than you could have got it for... If they can make you buy that one because you think it's a bargain over the top item then it's a good sale!

Who knows... I'm likely reading too much into this after too many beers... In any case it's largely up to us to be conscious that such behaviour goes on and that we; as human beings, are susceptible to all sorts of strange and suspect sales tactics.

Go search elsewhere, multiple times, from other devices, in Liberia... then save the money instead.