Usability Inspections by Groups of Specialists

Perceived Agreement in Spite of Disparate Observations

Morten Hertzum, Niels E. Jacobsen, Rolf Molich

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskning

Resumé

Evaluators who examine the same system using the same usability evaluation method tend to report substantially different sets of problems. This so-called evaluator effect means that different evaluations point to considerably different revisions of the evaluated system. The first step in coping with the evaluator effect is to acknowledge its existence. In this study 11 usability specialists individually inspected a website and then met in four groups to combine their findings into group outputs. Although the overlap in reported problems between any two evaluators averaged only 9%, the 11 evaluators felt that they were largely in agreement. The evaluators perceived their disparate observations as multiple sources of evidence in support of the same issues, not as disagreements. Thus, the group work increased the evaluators’ confidence in their individual inspections, rather than alerted them to the evaluator effect.
OriginalsprogEngelsk
TitelCHI 2002 : Extended Abstracts
ForlagAssociation for Computing Machinery
Publikationsdato2002
Sider662-663
ISBN (Trykt)1-58113-454-1
StatusUdgivet - 2002
Udgivet eksterntJa
BegivenhedCHI2002 - Minneapolis, MN, USA
Varighed: 20 apr. 200225 apr. 2002

Konference

KonferenceCHI2002
LandUSA
ByMinneapolis, MN
Periode20/04/200225/04/2002

Bibliografisk note

Publikationen skal IKKE medtages i årsberetningen, den indberettes til brug i udtræk internt på datalogi

Citer dette

Hertzum, M., Jacobsen, N. E., & Molich, R. (2002). Usability Inspections by Groups of Specialists: Perceived Agreement in Spite of Disparate Observations. I CHI 2002: Extended Abstracts (s. 662-663). Association for Computing Machinery.
Hertzum, Morten ; Jacobsen, Niels E. ; Molich, Rolf. / Usability Inspections by Groups of Specialists : Perceived Agreement in Spite of Disparate Observations. CHI 2002: Extended Abstracts. Association for Computing Machinery, 2002. s. 662-663
@inproceedings{acde2da052bd11dba4bc000ea68e967b,
title = "Usability Inspections by Groups of Specialists: Perceived Agreement in Spite of Disparate Observations",
abstract = "Evaluators who examine the same system using the same usability evaluation method tend to report substantially different sets of problems. This so-called evaluator effect means that different evaluations point to considerably different revisions of the evaluated system. The first step in coping with the evaluator effect is to acknowledge its existence. In this study 11 usability specialists individually inspected a website and then met in four groups to combine their findings into group outputs. Although the overlap in reported problems between any two evaluators averaged only 9{\%}, the 11 evaluators felt that they were largely in agreement. The evaluators perceived their disparate observations as multiple sources of evidence in support of the same issues, not as disagreements. Thus, the group work increased the evaluators’ confidence in their individual inspections, rather than alerted them to the evaluator effect.",
author = "Morten Hertzum and Jacobsen, {Niels E.} and Rolf Molich",
note = "Publikationen skal IKKE medtages i {\aa}rsberetningen, den indberettes til brug i udtr{\ae}k internt p{\aa} datalogi",
year = "2002",
language = "English",
isbn = "1-58113-454-1",
pages = "662--663",
booktitle = "CHI 2002",
publisher = "Association for Computing Machinery",

}

Hertzum, M, Jacobsen, NE & Molich, R 2002, Usability Inspections by Groups of Specialists: Perceived Agreement in Spite of Disparate Observations. i CHI 2002: Extended Abstracts. Association for Computing Machinery, s. 662-663, CHI2002, Minneapolis, MN, USA, 20/04/2002.

Usability Inspections by Groups of Specialists : Perceived Agreement in Spite of Disparate Observations. / Hertzum, Morten; Jacobsen, Niels E.; Molich, Rolf.

CHI 2002: Extended Abstracts. Association for Computing Machinery, 2002. s. 662-663.

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskning

TY - GEN

T1 - Usability Inspections by Groups of Specialists

T2 - Perceived Agreement in Spite of Disparate Observations

AU - Hertzum, Morten

AU - Jacobsen, Niels E.

AU - Molich, Rolf

N1 - Publikationen skal IKKE medtages i årsberetningen, den indberettes til brug i udtræk internt på datalogi

PY - 2002

Y1 - 2002

N2 - Evaluators who examine the same system using the same usability evaluation method tend to report substantially different sets of problems. This so-called evaluator effect means that different evaluations point to considerably different revisions of the evaluated system. The first step in coping with the evaluator effect is to acknowledge its existence. In this study 11 usability specialists individually inspected a website and then met in four groups to combine their findings into group outputs. Although the overlap in reported problems between any two evaluators averaged only 9%, the 11 evaluators felt that they were largely in agreement. The evaluators perceived their disparate observations as multiple sources of evidence in support of the same issues, not as disagreements. Thus, the group work increased the evaluators’ confidence in their individual inspections, rather than alerted them to the evaluator effect.

AB - Evaluators who examine the same system using the same usability evaluation method tend to report substantially different sets of problems. This so-called evaluator effect means that different evaluations point to considerably different revisions of the evaluated system. The first step in coping with the evaluator effect is to acknowledge its existence. In this study 11 usability specialists individually inspected a website and then met in four groups to combine their findings into group outputs. Although the overlap in reported problems between any two evaluators averaged only 9%, the 11 evaluators felt that they were largely in agreement. The evaluators perceived their disparate observations as multiple sources of evidence in support of the same issues, not as disagreements. Thus, the group work increased the evaluators’ confidence in their individual inspections, rather than alerted them to the evaluator effect.

M3 - Article in proceedings

SN - 1-58113-454-1

SP - 662

EP - 663

BT - CHI 2002

PB - Association for Computing Machinery

ER -

Hertzum M, Jacobsen NE, Molich R. Usability Inspections by Groups of Specialists: Perceived Agreement in Spite of Disparate Observations. I CHI 2002: Extended Abstracts. Association for Computing Machinery. 2002. s. 662-663