That Technology Magazine

Highlight videos are an essential part of the usability engineer’s toolbox. They are used to “get the message across” concerning usability issues discovered during an empirical test of an interactive system – and with every kind of message, the recipient should be considered when deciding on the exact way of doing so.

The “mechanics” of assembling a highlight video are not hard if you know how to handle software like Morae – or Premiere for the Steven Spielbergs who like to have more flexibility in editing their videos. But there is more to coming up with useful highlight videos than just being a power user of video editing software. (It helps, though).

An important part of the job is carefully considering the intended target group of the highlight video. A very basic distinction in this area can be made between developers and management. When assembling a highlight video for one of those two groups, its specific characteristics and needs should be taken into account. There is no “one fits all” approach to delivering highlight videos.

The aim of a highlight video from the usability engineer’s point of view is – not surprisingly – to prompt the audience to act in a way that will help in enhancing the usability of the system under investigation. Different target audiences have different ways of contributing to the usability of a system – managers can, e.g., contribute by allocating additional resources (in terms of money and time) to user interface design efforts whereas developers can heed the user’s perspective in their daily work and thus work towards improving usability. Because of their different characteristics, the highlight video should be “customized” to suit the intended target groups. The more a video is tailored to match the audience, the higher the probability of having the desired impact. This also means that a badly assembled video can send your usability project right down the drain…

Things to keep in mind when making a highlight video targeted at a management audience include:

• Members of this audience tend to be interested in long-term perspectives of the project and also in (surprise) financial issues.
• Very often they don’t have much knowledge about the technical implementation of the system or only a very high-level view. (That lack of knowledge may also go along with a lack of interest in technical details.)

What does that mean for the development of a “management’s highlight video”?

• Don’t bother them (too much) with technical details. Don’t show them one error after another, without giving them some means to assess the severity in terms of system usability. Stringing together bugs may send them to sleep before the video is over. They may not even recognize a complete “killer” without the necessary background knowledge of the system. This means that showing user reactions (verbal and nonverbal) plays an essential part in helping the management audience in assessing the severity of an error.
• An often seen characteristic of highlight videos is the repeated demonstration of certain critical issues happening to different users. Especially if the usability engineer has already established a good working relationship with the management audience (which should lead to them trusting his judgment) and if the video is presented by the usability engineer as part of a session, it may be enough showing one instance of a critical issue along with giving an estimate of the respective severity. By doing so, a greater diversity of issues can be included in the video and thus giving the audience more insight into the overall usability.

When presenting to developers, things may be different…

• It is not unusual for developers to become emotionally attached to the system they are developing. (But then it is not unusual for anyone to become emotionally attached to work they invest a lot of effort in…at least if they don’t completely hate what they are doing.) And you wouldn’t feel completely comfortable with someone pointing out what’s bad about your “baby”, either.
• Developers are interested in very concrete prompts for action. They don’t want some general assessment of usability that leaves them without a clue on what to do to improve the system. (If you go to see the doctor you wouldn’t expect him to just tell you that you’re sick and send you home afterwards.)

So when assembling the “developer’s cut” of a highlight video, some things may be helpful.

• In contrast to the management audience, it may be necessary to repeatedly show an error happening with different users, even if a working relationship between usability engineer and developers has been established because they may require more proof of the severity of an issue than just the usability engineer stating it.
• It is especially important to include positive aspects of the usability test. By doing so, the usability engineer demonstrates that the developers’ work is appreciated and that he is not trying to engage in “developer bashing” which some usability engineers may have a tendency to do. The second reason for including positive aspects is keeping the developers from removing those good elements when reworking the system. Don’t expect them to know what users especially like about the system if you don’t show them.
• For demonstrating critical issues it should be made sure that the users shown in the video appear in equal proportion. The danger in showing every second example of a usability bug with the same user is that developers may get the impression that the usability engineer selected an especially…ehm…dumb user to demonstrate pseudo-flaws that a moderately intelligent user would not suffer from. So in the worst case problems are attributed to the user and not to the system design.

Did I simplify matters? Yes. Of course the world is not black and white and there’s more to it that the few points made above, but those things are issues that I think should be looked out for when engaging in a usability engineering project that includes assembling highlight videos. Acknowledging the fact that different audiences have different characteristics and needs is the first step to producing effective highlight videos.

Other issues to keep in mind when producing a highlight video are the context in which it is shown (e.g. during a presentation vs. “standalone” as part of a report that is delivered to the customer) and the impact of editing (transitions, captions etc.) on the reception of the video. But since this already has gotten way too long, you have to look up Mr. Spielberg’s blog to get advice on those matters…