IE AppContainers and LocalStorage

IE's EPM (Enhanced Protected Mode) mode provides separate containers for web storage between desktop and Metro mode when using the Internet Zone. There's a page which discusses the detail but never really states why it behaves like this. It seems to me that this is unnecessarily complex and will lead to user confusion and angst - "why does switching to desktop mode lose my session/cookies/storage?" or more simply - "why do I have to login again?". It's also arguably a security risk since users will have multiple sessions/cookies active so could inadvertently leave themselves logged in or could lead to duplicate transactions because items may be placed in the basket in separate containers etc. It would be less of a concern if users couldn't easily switch, but of course they can because MS has kindly put a menu item on the Metro page to "View in the Desktop"!? It all seems to be related to providing enterprise users with the ability to maintain and configure a setup to provide greater access/functionality to  intranet sites than you would want for untrusted Internet sites (enabling various plugins and the like).

To a degree, fair enough, but it's mostly as a result of intranet sites adopting features that weren't standardised or hardened sufficiently in the first place (ActiveX, Java etc.). These need to be got rid of though this will cost companies dearly - replacing existing functionality with something else but with no significant added value to the business bar adherence to standards/security compliance etc. is  a hard sell.

So MS is; from one viewpoint, forced into this approach. The problem is it just adds more weight to my view that MS is so dependent on the enterprise customer and supporting the legacy of cruft they (MS & corporate intranets) have spawned over so many years that MS are no longer able to provide a clean, consistent and usable system (some would say they never were...).

Violation of rule #1 - Keep it Simple!

 

Is lying the solution to a lack of privacy online?

I do wish social networking sites like G+ and FB would stop advertising peoples birthdays. Your birth date is one of those "known facts" used by many organisations (banks, government departments etc.) to verify your identity. Providing this data to social networking sites can result in information leakage and contribute to identity theft and security incidents. Combine this with all the other bits of information they capture and it would be quite easy for someone to bypass those security questions every call centre asks as a facade to security - they only need to gleam a little info from many sources.

This morning G+ asked me if I wanted to say happy birthday to Peter. I know Peter slightly but not well enough to be privy to such information and I have no idea whether it really is his (or your) birthday today, if it is... Happy Birthday! If it's not then congratulations on lying to Google and Facebook - it's good practice (so long as you can remember the lies you tell).

In a world where privacy is becoming impossible, lying may be our saviour. What a topsy-turvy world we're living in...

Windows 7 Incident

Having recently been responsible for an estate wide software upgrade programme for many thousand devices to Windows 7 I sympathise but have to find this amusing. However, it is an interesting approach to achieving a refresh in particularly short order... Make the best of it guys, treat it as an opportunity to audit your estate... I do hope your backup procedures are working though...  ;)

Windows 7 Incident

Pre-emptive Single Task Operating System++

A while ago I wrote a blog entry about a pre-emptive single task operating system that I think the world needs. It seems I'm not the only one and George RR Martin (Game of Thrones) also thinks there's a need for this. His seems to stem from security as well as a productivity perspective but I think I grok what he means. The feature bloat in products such as MS Office these days detracts from their usability. They may be able to boil the ocean but it's not really necessary and just gets in the way of the creative process. However, DOS surely has a limited life and it must be hard to find the h/w components to run this on now. I may fire up a VM with DOS sometime to remind myself of the good-old-days... need-for-a-preemptive-os++

LocalServe

One of the things I have found irritating in the past is the need to install and configure a web-server each time the urge takes me to try something out. I don't run a local web-server permanently and being of a JFDI disposition the hurdle needed to get a server running is usually enough to stall what motivation I've managed to muster. Then I discovered that from Java 7 onwards it's fairly simple to implement your own web-server in plain Java - no need for an application server.

LocalServe implements two types of handlers:

1. File-handler -  This serves any content (and sub-directories) in the directory in which localserve is run. Any file which is not found returns a 404 and  any request for the root of a folder (path ending in "/") attempts to return the index.html file in the directory. Note that localserve does not provide any listing of directories.

If all you want to do is serve static content then the above is sufficient and LocalServe can be run using the command below in the directory you want to serve content from. This will run a webserver on port 8765 by default:

java -jar localserve.jar

The port number can also be changed by adding this to the end - e.g.:

java -jar localserve.jar 5678

2. SQL-handler - Often static content isn't enough and you need to use a database. This handler provides access to a database that can be called from JavaScript (typically via an AJAX request). A configuration file can be specified on the command line when running localserve. This configuration file provides details of a SQLite database, and the SQL statements that are supported. Each SQL statement has a logical name, SQL statement (including bindings), supported methods (POST or GET) and optionally a redirect (where to send the user on success). Calls to paths starting "/sql/" are sent to the SQL handler and the path element after this is used to match against a logical name in the configuration file. If found the SQL statement is executed with any HTTP parameters matching the bind names being bound accordingly. Two special names "PAGE" and "PAGE_SIZE" are defined such that queries which may return many many rows can be restricted to returning only certain pages of a certain size. Results from SQL commands are returned in JSON format.

The configuration file can be specified on the command line when running localserve as below:

java -jar localserve.jar config.json
or to run on a port other than 8765:
java -jar localserve.jar config.json 5678

(note that the position of these two parameters doesn't matter).

An example configuration file is shown below:

{ "wsm": {
    "connString": "jdbc:sqlite:/path/to/database/file.db",
    "driver": "org.sqlite.JDBC",
    "operations": [
        { "path": "listbob", 
          "statement": "select name, id from bob", 
          "methods": "GET,POST" },
        { "path": "querybob", 
          "statement": "select id, name from bob where id={ID} order by name desc", 
          "methods": "GET,POST" },
        { "path": "insertbob", 
          "statement": "insert into bob (id, name) values ({ID}, {N})", 
          "redirect": "/sql/querybob", 
          "methods": "GET,POST" }
    ]
}
}

The database here contains one very simple table (BOB) as:

CREATE TABLE BOB (ID VARCHAR(20), NAME VARCHAR(100));


The only database used is SQLite and the main JAR file contains all libraries for this to work. I have tried other databases (notably IBM DB2) which worked fine so long as the JAR libraries can be found.

An example response to something like http://localhost:8765/sql/listbob looks like:

{ "dataset": 
    {  "page": 0, 
       "pageSize": 20,  
       "record": [  
           { "name": "James Brown", "id": "1"} ,  
           { "name": "Simple Simon", "id": "2"} ,  
           { "name": "Ducky Duncan", "id": "3"}  
       ] 
    }
} 

The attribute names are derived from the query names and are usually lower case. HOWEVER, you may find that if you explicitly state a column name the attribute may come back in uppercase (e.g. "select id||' - '||name LONG_NAME from bob" will result in an attribute with name "LONG_NAME").

Once you have the database setup and working then it's a relatively simple task to use JQuery to submit AJAX requests to the SQL handler to create/read/update/delete/list the database. A hastily knocked up example is below:

<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/> 
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/> 
    <meta charset="utf-8"/>  
  
    <script src="jquery-2.0.3.min.js"></script>
    <script src="purl.js"></script>
    </head>
  <body>
      ...
  <script type="text/javascript">
      
      function query(searchId) {
        $.ajax({ url: "/sql/querybob",
            data: { "PAGE": 0, "PAGE_SIZE": 32000, "ID": searchId },
            success: function( data ) {
              var dataset = (JSON.parse(data)).dataset;
              var arr = new Array();
              for (var i=0;i<dataset.record.length;i++) {
                alert(dataset.record[i].id + " " + dataset.record[i].name);
          }
        });
      }
      query(1);
    </script>
  </body>
</html>

Anyway, it's been useful when I just want to prototype something and this is definitely not intended for any production use. It's simply a very simple webserver so that I can quickly get on with prototyping something.

If you're interested then the source can be found on GitHub where the LocalServe JAR file can be downloaded. The code is what I call "prototype quality" which means it's been made to work by beating it into some shape with a hammer - it is uncommented and not of production quality.

Java 7 and this JAR are all you should need to run LocalServe. As ever, no assurances, warranties, guarantees etc. are provided and; whether you lose a little data or the world goes into meltdown (and everything in-between), I'll accept no responsibility for any damages caused...

Entropy - Part 2

A week or so ago I wrote a piece on entropy and how IT systems have a tendency for disorder to increase in a similar manner to the second law of thermodynamics. This article aims to identify what we can do about it...

It would be nice if there was some silver bullet but the fact of the matter is that; like the second law, the only real way to minimise disorder is to put some work in.

1. Housekeeping

As the debris of life slowly turns your pristine home into something more akin to the local dump, so the daily churn of changes gradually slows and destabilises your previously spotless new IT system. The solution is to crack on with the weekly chore of housekeeping in both cases (or possibly daily if you've kids, cats, dogs etc.). It's often overlooked and forgotten but a lack of housekeeping is frequently the cause of unnecessary outages.

Keeping logs clean and cycling on a regular basis (e.g. hoovering), monitoring disk usage (e.g. checking you've enough milk), cleaning up temporary files (e.g. discarding those out of date tins of sardines), refactoring code (e.g. a spring clean) etc. is not difficult and there's little excuse for not doing it. Reviewing the content of logs and gathering metrics on usage and performance can also help anticipate how frequently housekeeping is required ensure smooth running of the system (e.g. you could measure the amount of fluff hoovered up each week and use this as the basis to decide which days and how frequently the hoovering needs doing - good luck with that one!). This can also lead to additional work to introduce archiving capabilities (e.g. self storage) or purging of redundant data (e.g. taking the rubbish down the dump). But like your home, a little housekeeping done frequently is less effort (cost) than waiting till you can't get into the house because the doors jammed and the men in white suits and masks are threatening to come in and burn everything.

2. Standards Compliance

By following industry standards you stand a significantly better chance of being able to patch/upgrade/enhance without pain in the future than if you decide to do your own thing.

That should be enough said on the matter but the number of times I see teams misusing APIs or writing their own solutions to what are common problems is frankly staggering. We (and me especially) all like to build our own palaces. Unfortunately we lack sufficient exposure to the space of a problem to be able to produce designs which combines elegance with flexibility to address the full range of use cases or the authority and foresight to predict the future and influence this in a meaningful way. In short, standards are generally thought out by better people than you or me.

Once a standard is established then any future work will usually try to build on this or provide a roadmap of how to move from the old standard to the new.

3. Automation

The ability to repeatedly and reliably build the system decreases effort (cost) and improves quality and reliability. Any manual step in the build process will eventually lead to some degree of variance with potentially unquantifiable consequences. There are numerous tools available to help with this (e.g. Jenkins) though unfortunately usage of such tools is not as widespread as you would hope.

But perhaps the real killer feature is test automation which enables you to continuously execute tests against the system at comparatively negligible cost (when compared to maintaining a 24x7 human test team). With this in place (and getting the right test coverage is always an issue) you can exercise the system in any number of hypothetical scenarios to identify issues; both functional and non-functional, in a test environment before the production environment becomes compromised.

Computers are very good at doing repetitive tasks consistently. Humans are very good at coming up with new and creative test cases. Use each appropriately.

Much like housekeeping, frequent testing yields benefits at lower cost than simply waiting till the next major release when all sorts of issues will be uncovered and need to be addressed - many of which may have been around a while though no-one noticed... because no-one tested. Regular penetration testing and review of security procedures will help to proactively avoid vulnerabilities as they are uncovered in the wild, and regular testing of new browsers will help identify compatibility issues before your end-users do. There are some tools to help automate in this space (e.g. Security AppScan and WebDriver) though clearly it does cost to run and maintain such a continuous integration and testing regime. However, so long as the focus is correct and pragmatic then the cost benefits should be realised.

4. Design Patterns

Much like standards compliance, use of design patterns and good practices such as abstraction, isolation and dependency injection can help to ensure changes in the future can be accommodated at minimal effort. I mention this separately though since the two should not be confused. Standards may (or may not) adopt good design patterns and equally non-standard solutions may (or may not) adopt good design patterns - there are no guarantees either way.

Using design patterns also increases the likelihood that the next developer to come along will be able to pick up the code with greater ease than if it's some weird hair-brained spaghetti bowl of nonsense made up after a rather excessive liquid lunch. Dealing with the daily churn of changes becomes easier, maintenance costs come down and incidents are reduced.

So in summary, entropy should be considered a BAU (Business as Usual) issue and practices should be put in place to deal with it. Housekeeping, standards-compliance, automation through continuous integration and use of design patterns all help to keep the impact of change minimised and keep the level of disorder down.

Next time, some thoughts on how to measure entropy in the enterprise...

Feedback - Logging and Monitoring

It seems to me that we are seeing an increasing number of issues such as this reported by the Guardian. A lost transaction results in a credit default against an individual with the result that they cannot obtain a mortgage to buy a house. Small error for the company, huge impact for the individual.

The company admitted that despite the request being submitted on their website they did not receive the request!? So either the user pressed submit then walked away without noting the response was something other than "all ok!" or the response was "all ok!" and the company failed to process the request correctly.

If the former then, well, user error for being a muppet... As end users we all need to accept some responsibility and check that we get the feedback we expect.

For the latter, there are several reasons why subsequent processing could have failed. Poor transaction management so the request never gets committed, poor process management so the request drops into some dead queue never to be dealt with (either through incompetence or through malicious intent), or through system failure and a need to rollback with resulting data loss.

With the growth in IT over the past couple of decades there are bound to be some quality issues as the result of ever shorter, demanding and more stringent deadlines and budgets. Time and effort needs to be spent exploring the hypothetical space of what could go wrong so that at least some conscious awareness and acceptance of the risks is achieved. This being the case I'm usually quite happy to be overruled by the customer on the basis of cost and time pressures.

However, it's often not expensive to put in place some logging and monitoring - and in this case there must have been something for the company to admit the request had been submitted. Web logs, application logs, database logs etc. are all valuable sources of information when PD'ing (Problem Determination). You do though need to spend at least some time reviewing and auditing these so you can identify issues and deal with them accordingly.

I remember one case where a code change was rushed out which never actually committed any transaction. Fortunately we had the safety net of some judicious logging which allowed us to recover and replay the transactions. WARNING: It worked here but this isn't always a good idea!

In general though, logging and monitoring are a very good idea. In some cases system defects will be identified, in others, transient issues will be found which may require further work to deal with them temporarily. Whatever the underlying issue it's important to incorporate feedback and quality controls into the design of systems to identify problems before they become disasters. At a rudimentary level logging can help with this but you need to close the feedback loop through active monitoring with processes in place to deal with incidents when they arise. It really shouldn't just be something that we only do when the customer complains.

I don't know the detail of what happened in this case. It could have been user-error, or we could applaud the company for having logging in place, or they could just have got lucky. In any case, we need to get feedback on how systems and processes are performing and operating in order to deal with issues when they arise, improve quality and indeed the business value of the system itself through continuous improvement.