Sunday, February 29, 2004
ETcon, UK style
The UK participants of ETcon (now ETech) decided to run a brief UK event to bring to a UK audience the content of their presentations, together with an overview of the event.
A good time was had by all, and pages can be found at http://www.symbianwiki.com/ConConUK for anyone interested in taking this forward to a full-blown UK one-day event later this year.
2:42 PM|
link to this item
Saturday, February 28, 2004
More info on Brazilian e-voting
Brazil's e-voting system is widely touted by Brazilian officials, and others, as the answer to e-voting problems. PR material is produced in English by the Brazilian authorities to sing its praises.
Academics in Brazil, however, have for some time urged caution and have produced their own - less publicised - versions of the story. And now, as someone who seems to be on the 'usual suspects' lists when it comes to e-voting, I've just received a link to a formal expert report on the Brazilian system, from someone who has posted the material to the Web, but prefers to remain anonymous (a former Electoral Justice dept. worker in Brazil with access to such documents).
One item is a 2002 report commissioned by the then Brazilian opposition from independent experts, "that shows the poor software development made by the TSE and Unisys". The text is in Portuguese.
Here's my own short summary of the report - my Portuguese is not perfect, but I've been told by Brazilian e-voting experts that it's a fair reflection of the contents:
The report is a quality assessment, covering documentation and code, and was commissioned by the Partido dos Trabalhadores (Workers' Party), via the COPPETEC Foundation, and produced in August 2002.
The assessors were Professor Ana Regina Cavalcanti da Rocha, Professor Guilherme Horta Travassos, and experts Gleison Santos de Souza and Somulo Nogueira Mafra.
The results are divided into (a) software documentation and (b) code.
The document contains general comments concerning software documentation - details are contained in Annex 4, not supplied.
Software documentation was divided into two groups. The first group was found to be based on "in some respects out-of-date and incoherent methodology"; specifications were "inadequate, incomplete and inconsistent"; poor project definition; no documentation on quality procedures, configuration management procedures or systematic testing procedures (meaning either no action or no documentation of it); no test plan documentation; when further test planning and results documentation was requested, this was incomplete with results obtained missing.
For the second group of documents, seemingly primarily the responsibility of Unisys, a variety of inadequacies are reported, including lack of documentation of differences between the test and live systems, and of modifications to code. While FTP file transfer is mentioned, there is no documentation of security measures with respect to FTP.
The report points out that [the presence of] "processes for developing quality software are indicators of products of good quality" and goes on to conclude that the software development process was "substantially ad hoc and immature" which "often leads to products of unpredictable quality, strongly dependent on the characterisitcs of the developers".
In the second section, concerning code, the report states that there are sections of inactive code in several applications with no explanation for their presence; evidence of poor project management with different solutions found by different programmers for similar problems; lack of treatment of error conditions...etc. The report also remarks on the lack of a watermark on the voting paper trail.
The report concludes with a set of short-term recommendations, and longer-term recommendations.
There are many other points, and anyone with decent Portuguese should feel free to render a fuller summary, for the benefit of those outside Brazil.
2:12 PM|
link to this item
Friday, February 27, 2004
Baloney
"The focus group is an essential component of user-centered design..." (link to full text)
It's amazing what baloney you can read on the Web, even from so-called reputable sources.
12:11 PM|
link to this item
Sunday, February 22, 2004
Questionnaire design
It never ceases to amaze me how badly questionnaires are designed. And it's particularly irritating when I consider the number of times I hear from people in business that 'quant' studies are the only ones they consider valid.
Today's example: I've received a request by email from a market research company contracted by one of the universities I've attended, UCL. The request is for former students to fill out an online questionnaire. So I click on the link, and go to the first question. And here I hit the first problem: 'What do you think are the key issues facing UK higher education?' asks Question 1, with four and only four options being given (top-up fees, choice of courses, competition, government targets). There is no option for an 'other' response.
Question 1 turns out to be compulsory: I cannot move onto the next page without selecting at least one of the limited choices given, none of which I agree with. The choices, however, do not allow for any indication of what I consider to be the main issue(s): standards and staff pay. So I either lie, conforming to the questionnaire designer's expectations, in order to be able to move on to further questions, or don't complete the questionnaire. Either way, the accumulated bias created by bad design will, I suspect, not be apparent to anyone in either the market research company or the university, unless they receive an onslaught of critical emails.
And this on behalf of a university, and a good one, that ought to know better. I almost feel inclined to answer questions 'incorrectly' just to find out how many other howlers the document contains. But I don't have the time.
Postscript: A response to this post received from a director of the market research organisation concerned:
"I fully understand your anger at being unable to answer ‘Other’ in the alumni questionnaire. This was a warm up question (also used in the qualitative research) and the reason ‘Other’ was not put in was one of cost – i.e. the cost of coding up to 6,000 responses (some of which could be several pages long as we have found in similar alumni studies in the past). That said I have instructed for ‘Other’ to be put in, as well as a code that says ‘Too many issues to list in this way’."
Nice to see that a blog post about questionnaire design can have an impact on its designers and the questionnaire itself, but then he continues..."Overall while I recognize your irritation I do not think the questionnaire is quite as bad as you say on your website."
My response is that if I cannot fill in a questionnaire that I want to fill in, aimed at my specific community (UCL alumni), because of a rogue compulsory question one - and all, it appears, over a warm-up question - then that is as bad as it gets. That's 100% failure.
As Caroline Jarrett - who runs a company specialising in forms design, and has written a book on the subject - said to me, "no question in a questionnaire should be compulsory", adding "you can always throw away the results for 'other'". Caroline is adamant that 'other' is a much neglected category in the forms world, and I have to agree with her.
2:39 PM|
link to this item
Thursday, February 19, 2004
Professional development
The reason for my being in Brussels was an invitation to speak at Namahn, a Belgian user-centred design consultancy. Namahn is doing a great job in bringing together people in Belgium interested in user-centred design ideas, on a regular basis, to listen to and talk with invited guest speakers. This goes some way to making up for the lack of more formal 'trade' organisations in the country. Previous speakers have included William Hudson.
Namahn's offices in Brussels include a superb room know as the library, a sizable space used for meetings, with an extensive collection of literature on user-centred design and many related areas. Namahn takes its library very seriously and it's a great resource. I immediately resolved to get my own resources into some semblance of order, which will mean constructing some shelving....
1:24 PM|
link to this item
Ethereal music
Yesterday I paid a flying visit to MIM, the Brussels musical instrument museum.
Apart from being housed in an attractive Art Nouveau building, the museum has benefited from Belgian lottery funding, enabling it to provide - over infrared headphones - sound samples of all manner of instruments and types of music. It's great being able to wander around and drift from Albanian pipes to battling banjos - the museum has a surprisingly large collection of the latter - though it was sometimes difficult to determine just what instrument you were listening to. The 'soundtrack' is very international, with the only sounds transmitted being some signal 'pips' and the music sample itself, and the reception areas for each sample are not always well defined with repect to the exhibits. And the sound goes when you turn your head. But that's a minor quibble.
One of the highlights of the museum is the large collection of instruments - saxaphones, saxhorns and others - devised by the Belgian Adolph Sax, sitting alongside a reproduction of Sax's original patent application for the saxaphone. It's interesting to see the large number of other musical instruments that fell by the wayside over the years.
My own favourite exhibits were the glass harmonica - an instrument partly developed by Benjamin Franklin - and the ethereal sounds in the back of the 'mechanical music' section, to wit a modern Big Briar theremin, an ondes-Martenot and a trautonium. The ondes-Martenot is apparently still used by orchestras today.
The BBC website has a downloadable desktop theremin (PC and Mac).
12:28 PM|
link to this item
Monday, February 16, 2004
Digital voice recorders
I'll have a rant about these when I get back from Brussels....
12:42 PM|
link to this item
Saturday, February 14, 2004
Camcorder design
I'm trying to find a digital camcorder that will meet my needs, and it's tough.
Extensive discussions on a list used by a number of ethnographers, many of whom use camcorders for fieldwork, revealed the following issues.
First, all camcorders seem designed for the right-handed. I'm left handed. I'm a left hander in a world of right-handed magic, in the words of Geoffrey Bowker and Susan Leigh Star.
Second, the vast majority are designed so that the tape load is in the base. This makes filming using a tripod for more than 60 minutes (a) a fiddle (b) disruptive to the environment being filmed. Taken together, these 'features' make both hand-held and tripod use less than easy.
While camcorder these days come with all manner of functions rarely needed by the vast majority, these two fundamental issues - for me at least - don't seem a concern for manufacturers.
3:17 PM|
link to this item
Thursday, February 12, 2004
Geek clock
A perfect geek gift: a binary clock
There's a downloadable desktop version here.
6:00 PM|
link to this item
Usability goes hip hop
"If your project goes pear shaped, you come get me"
Is this the first ever mention of Norman, Winograd, Shneiderman et al in a song?
5:49 PM|
link to this item
Monday, February 09, 2004
Cost per page
Over at Usability Views, Chris is always coming up with interesting ways of analysing all kinds of data. His latest target is usability reports themselves.
My comment on this would be that having done a range of reports for a variety of clients, I've learned that house style is incredibly varied, and the shorter almost always means the more concentrated. You need to draw up the long version and then edit it down to achieve report nirvana, which is time consuming.
A report spread over 80 pages may give the info, and on a cost per page basis may appear good value; and invariably, this is what the report commissioners have asked for. But the 8-page report that contains highly analysed and summarised data, with visual summaries, built out of the same mass of data, and that has taken even longer to achieve, may prove more immediately usable.
10:16 AM|
link to this item
Oh my, so many places to go
So many places to go!
Here are the places I plan to go to this year, on the travelling fellowship:
9:41 AM|
link to this item
Michigan experiments with Internet voting
There have been a few news reports about Internet voting in Michigan, where the state Democratic Party is allowing registered Democrats to cast votes online for its presidential caucus.
There's pre-caucus coverage from Michigan State University, and post-caucus coverage from Yahoo.
One of my UPA colleagues has used the system and is putting some feedback together.
9:02 AM|
link to this item
Sunday, February 08, 2004
Researching voting in the US
Having been lucky enough to gain a travelling fellowship, I'll be spending some time in the US this year, studying voting tech (old and new) and how people are responding to it. If you're involved in a project or research in this area, please let me know. I'm currently planning my itinerary...
5:27 PM|
link to this item
US electronic voting developments
While I was away from my desk, some interesting developments took place in the US on remote Internet voting. Barbara Simons, Avi Rubin and others came out with a report that blasted the Pentagon's current plans to implement e-voting for overseas service personnel [SERVE project], and the Pentagon promptly abandoned its plans.
Both The Register and The New York Times (registration required) reported on the outcome, while e-week tried putting a positive spin on the dispute, arguing that some scientists still think it's a viable idea. Further coverage in TechWeb.
The NYT's story points to the fact that the 'minority report' authors were the only experts that actually turned up for the six days of briefings on the system: "There is no majority report because the other experts did not write their views" [or turn up at the briefings].
A NYT editorial today, 'Budgeting for Another Florida', reports that Bush's funding for upgrading voting technology across the US has been severely curtailed.
3:19 PM|
link to this item
Mental models of homing pigeons
It turns out that homing pigeons follow the highways, just like we do, according to UK scientists.
Which still makes me wonder what they did before motorways and dual carriageways existed...
3:09 PM|
link to this item
|
