Articles and talks

Comment: Quality Framework - the First Iteration?

by Louise Ferguson (2003)

[The following article appeared in Usability News as a follow-up to an earlier article I wrote, eGov Feature on e-Envoy Framework causes Questions (and Fur?) to Fly, published in Usability News on 21 August 2003. It now appears in the Usability Views Usability Heretics collection.]

My original article commissioned by eGov Monitor (appearing on UN as eGov Feature on e-Envoy Framework causes Questions (and Fur?) to Fly) only allowed space to reflect a few issues and views raised by the people whom I spoke to. My plan was to write a more usability professional-focused version for Usability News, including more industry views and more discussion of the way forward. I was overtaken by events: cats and pigeons in close proximity, with chickens roosting and fur flying.

Reactions to the original piece have all – so far as I know – been positive: 'A nicely balanced piece!' wrote one; 'It's not always easy to criticise usability initiatives without sounding as if you are criticising the field, but I think you trod the line well in this article,' said another.

One issue most respondents referred to, but that I did not include in the original article – the wrong audience entirely – was the concern about the consultation process behind the document. And here we get to the crux: many agencies that spoke to me said they were unaware of the consultation period, but could not admit this in public as it made them look slightly foolish or ill-informed. The only firm that stated specifically that it had seen the Framework during its consultation phase said their firm had found out about it through knowing someone at the Office of the e-Envoy – but did not have time to file a considered response in the time left. Another firm said it had seemingly input to the Office of the e-Envoy 'at the wrong time'.

I have no idea how many usability firms or others in the industry responded to the official consultation for the Framework, but I find it disconcerting that so many in the industry were unaware of it, in what is, after all, a very small and specialised corner of the new media industry. It strikes me that with so many significant players unaware of the process, there is something that needs improving in either the process or the way it is publicised.

For the record, comments in my article about the Framework came from agencies including amberlight, Bunnyfoot, Circle, Effortmark, Flow Interactive, Optimum Web, Sapient, two others preferring to remain anonymous, a couple of senior academics, the president of the UK Usability Professionals' Association, and the editor of a usability publication. In some cases, there was more than one contribution from a particular organisation, or several contributions from one contributor over a period of days. This was a straw poll, not a statistical exercise (in common with the majority of journalism).

It would be unusual for any group of professionals to express consensus on a document as complex as the Framework (remember the joke about the twelve economists in a room who could be guaranteed to hold thirteen opinions if one of them were Keynes). Everybody, without exception, welcomed the arrival of such a document – 'encouraging', 'to be applauded', 'welcome', 'well done for trying' were some of the descriptions. But there was clear concern expressed by the majority of respondents about both the process of creation and the final contents. The message from most people was very much that this was a version 0.1. And what's wrong with that, I say: is this not a profession that concerns itself with iteration? And doesn't the biggest company on the planet operate in exactly the same way?

For future reference, the main plaudits included:
- wholeheartedly support e-Envoy's Office in raising UCD methods to fore
- sets out Government support for user-centred design
- is realistic about what can be achieved in small departments with low-reach websites
- distinguishes usability from accessibility
- full of great information
- inclusion of guidance about errors on forms
- useful but limited
- a hearty collection of some of the better thinking on usability
- a rich blend of method, approach and substantive knowledge
- may help "committed government website managers"

And the main criticisms included:
- lack of discussion of the specific needs of and issues surrounding public sector websites
- lack of discussion of why websites need to be useful as well as usable (and what makes them so)
- not representative of range of practices in the field
- old school thinking, out of touch with current practice
- idea promoted of usability only as a third party, 'agency' activity, versus its incorporation into the systems and processes of the organisation as part of a human-centred approach (the latter not felt to be addressed at all)
- conflation of testing and HCD/UCD
- does not discuss embedding UCD within the organisation
- downplays the importance of iteration
- impression given that iteration is somehow linear; a circular process bashed into a straight line
- confused coverage of testing methodologies
- not appropriate for Government sites to get away with minimal usability
- a jumble of unrelated methods, models and standards
- lack of presentation of best practice (eg. navigation, downloads, metadata)
- contains generalisations and misleading statements (eg. re testing children, numbers to test, FAQs)
- little about trade-offs or constraints
- lack of integration with online resources and the Web
- insufficient use of concrete examples
- accessibility section woefully inadequate
- poor/strange selection of resources (eg. absence of UsabilityNet), no linking to Web, no advantage taken of new media to create an active, living resource
- unrepresentative or out-of-date resources and toolkit references, such as 'jthom'
- poor use of online resources
- poor organisation, integration and writing, lots of cut and paste, overuse of acronyms, blurry definitions, usability jargon, difficult to read
- poor usability: not useful for or usable by the intended audience
- makes no attempt to speak with accessible common voice
- suitable for budding usability professionals
- won’t have any impact on most web project teams

And in conclusion:
- hopefully this is just the start of a bigger programme – because if this is all there is, it's nowhere near enough!

As I said in the original article, the obligatory nature of the Framework in some public sector bidding processes, at least – and probably to an increasing degree – makes this a potentially important document that could have an influence on the practice of usability in the UK.

This is all the more reason to get the Framework right. UK UPA members have just been asked to contribute their views to an internal process. Ann Light has also invited UN readers to comment.

If I've achieved nothing else with these articles, I hope I have encouraged a wide range of people involved in public sector websites to read the Framework, to discuss it and to have a view. If we, as a community, also manage to contribute to improving the guidance provided by documents such as the Framework, then so much the better.

Read more articles on this website
Check out shorter pieces on the blog