|
|
|
|
|
|
Lesson#20
|
USER RESEARCH-2
|
|
|
|
As the aim of this lecture is to introduce you the study of
Human Computer
Interaction, so that after studying this you will be able to:
. Understand the
User-Centered approach
. Discuss in detail
the ethnographic interviews
. Understand how to
prepare for ethnographic interviews
In our last lecture we were studying the qualitative research
techniques. Today we
will discuss last technique, ethnographic field study. But,
first let us look at the usercentered
design approach.
20.1 User-Centered Approach
The user-centered approach means that the real users and their
goals, not just
technology, should be the driving force behind development of a
product. As a
consequence, a well-designed system should make the most of
human skill and
judgment, should be directly relevant to the work in hand, and
should support rather
than constrain the user. This is less technique and more a
philosophy.
In 1985, Gould and Lewis laid down three principles they
believed would lead to a
“useful and easy to use computer system.” These are very similar
to the three key
characteristics of interaction design.
. Early focus on
users and tasks: This means first understanding who the users
will be by directly studying their cognitive, behavioral,
anthropomorphic, and
attitudinal characteristics. This required observing users doing
their normal
tasks, studying the nature of those tasks, and then involving
users in the design
process.
. Empirical
measurement: early in development, the reactions and performance
of intended users to printed scenarios, manuals, etc, is
observed and measured.
Later on, users interact with simulations and prototypes and
their performance
and reactions are observed, recorded and analyzed.
. Iterative design:
when problems are found in user testing, they are fixed and
then more tests and observations are carried out to see the
effects f the fixes.
This means that design and development is iterative, with cycles
of “design,
test, measure, and redesign” being repeated as often as
necessary.
Iteration is something, which is emphasized in user-centered
design and is now
widely accepted that iteration is required. When Gould and Lewis
wrote their paper,
however, the iterative nature of design was not accepted by most
developers. In fact,
177
they comment in their paper how “obvious” these principles are,
and remark that
when they started recommending these t designers, the designers’
reactions implied
that these principles were indeed obvious.
Applying ethnography in design
Ethnography is a method that comes originally from anthropology
and literally means
“writing the culture”. It has been used in the social sciences
to display the social
organization of activities, and hence to understand work. It
aims to find the order
within an activity rather than impose any framework of
interpretation on it. It is a
broad-based approach in which users are observed as they go
about their normal
activities. The observers immerse themselves in the users’
environment and
participate in their day-to-day work, joining in conversations,
attending meetings,
reading documents, and so on. The aim of an ethnographic study
is to make the
implicit explicit. Those in the situation, the users in this
case, are so familiar with their
surroundings and their daily tasks that they often don’t see the
importance of familiar
actions or happenings, and hence don’t remark upon them in
interviews or other datagathering
sessions.
There are different ways in which this method can be associated
with design. Beynon-
Davies has suggested that ethnography can be associated with the
development as
“ethnography of”, “ethnography for”, and “ethnography within.”
Ethnography of
development refers to studies of developers themselves and their
workplace, with the
aim of understanding the practices of development (e.g. Button
and Sharrock).
Ethnography for development yields ethnographic studies that can
be used as a
resource for development, e.g., studies of organizational work.
Ethnography within
software development is the most common form of study; here the
techniques
associated with ethnography are integrated into methods and
approaches for
development.
Because of the very nature of the ethnography experience, it is
very difficult to
describe explicitly what data is collected through such an
exercise. It is an experience
rather than a data-collection exercise. However, the experience
must be shared with
other team members, and therefore needs to be documented and
rationalized.
Studying the context of work and watching work being done
reveals information that
might be missed by other methods that concentrative on asking
about work away from
its natural setting. For example, it can shed light on how
people do the “real” work as
opposed to the formal procedures that you ‘d find in
documentation; the nature and
purpose of collaboration, awareness of other’s work, and
implicit goals that may not
even be recognized by the workers themselves. For example, Heath
et al. has been
exploring the implications of ethnographic studies of real-world
setting for the design
of cooperative systems. They studied medical centers,
architects’ practices, and TV
and radio studios.
In one of their studies Heath et al. looked at how dealers in a
stock exchange work
together. A main motivation was to see whether proposed
technological support for
market trading was indeed suitable for that particular setting.
One of the tasks
examined in detail was the process f writing ticket to record
deals. It had been
commented upon earlier by others that this process of deal
capture, using “oldfashioned”
paper and pencil technology, was currently time-consuming and
prone to
error. Based on this finding, it had been further suggested that
the existing way of
178
making deals could be improved by introducing new technologies,
including touch
screens to input the details of transactions, and headphones to
eliminate distracting
external noise.
However, when Heath et al. began observing the deal capture in
practice, they quickly
discovered that these proposals were misguided. In particular,
they warned that these
new technologies would destroy the very means by which the
traders currently
communicate and keep informed of what others are up to. The
touch screens would
reduce the availability of information to others on how deals
were progressing; while
headphones would impede the dealers’ ability to inadvertently
monitoring of other
dealers’ actions was central to the way deals are done.
Moreover, if any dealers failed
to keep up with what the other dealers were doing by
continuously monitoring them,
it was likely to affect their position in the market, which
ultimately could prove very
costly to the bank they were working for.
Hence, the ethnographic study proved to be very useful in
warning against attempts to
integrate new technologies into a workplace without thinking
through the implications
for the work practice. As an alternative, Heath et al. suggested
pen-based mobile
systems with gesture recognition that could allow deals to be
made efficiently while
also allowing the other dealers to continue to monitor one
another unobtrusively.
Hughes et al state that “doing” ethnography is about being
reasonable, courteous and
unthreatening, and interested in what’s happening. Training and
practice are required
to produce good ethnographies.
Collecting ethnographic data is not hard although it may seem a
little bewildering to
those accustomed to using a frame of reference to focus the data
collection rather that
letting the frame of reference arise from the available data.
You collect what is
available, what is “ordinary”, what it is that people do, say,
how they work. The data
collected therefore has many forms: documents, notes of your
own, pictures, room
layouts.
In some way, the goals of design and the goals of ethnography
are at opposite ends of
a spectrum. Design is concerned with abstraction and
rationalization. Ethnography, on
the other hand, is about detail. An ethnographer’s account will
be concerned with the
minutiae of observation, while a designer is looking for useful
abstractions that can be
used to inform design. One of the difficulties faced by those
wishing to use this very
powerful technique is how to harness the data gathered in a form
that can be used in
design.
20.2 Ethnography framework
Ethnographic framework has been developed specifically to help
structure the
presentation of ethnographies in a way that enables designers to
user them. This
framework has three dimensions
1. Distributed co-ordination
2. Plans and procedures
3. Awareness of work
1. Distributed co-ordination
The distributed co-ordination dimension focuses on the
distributed nature of the tasks
and activities, and the means and mechanisms by which they are
coordinated. This
has implications for the kind of automated support required.
179
2. Plans and procedures
The plans and procedures dimension focuses on the organizational
support for the
work, such as workflow models and organizational charts, and how
these are used to
support the work. Understanding this aspect impacts on how the
system is designed to
utilize this kind of support.
3. Awareness of work
The awareness of work dimension focuses on how people keep
themselves aware of
others’ work. No one works in isolation, and it has been shown
that being aware of
others’ actions and work activities can be a crucial element of
doing a good job. In the
stock market example this was one aspect that ethnographers
identified. Implications
here relate to the sharing of information.
Rather than taking data from ethnographers and interpreting this
in design, an
alternative approach is to train developers to collect
ethnographic data themselves.
This has the advantage of giving the designers first-hand
experience of the situation.
Telling someone how to perform a task, or explaining what an
experience is like is
very difficult from showing him or her or even gaining the
experience themselves.
Finding people with the skills of ethnographers and interaction
designers may be
difficult, but it is possible to provide notational and
procedural mechanisms to allow
designers to gain some of the insights first-hand. Two methods
described bellow give
such support.
. Coherence
. Contextual design
Coherence
The coherence method combines experiences of using ethnography
to inform design
with developments in requirements engineering. Specifically, it
is intended to
integrate social analysis with object-oriented analysis from
software engineering.
Coherence does not prescribe how to move form the social
analysis to use cases, but
claims that presenting the data from a ethnographic study based
around a set of
“viewpoints” and “concerns” facilitated the identification of
the product’s most
impotent use cases.
Viewpoints and concerns
Coherence builds upon the framework introduced above and
provides a set of focus
questions for each of the three dimensions, here called
“viewpoints”. The focus
questions are intended to guide the observer to particular
aspects of the workplace.
They can be used as a starting point to which other questions
may be added as
experience in the domain and the method increase.
In addition to viewpoints, Coherence has a set of concerns and
associated questions.
Concerns are a kind of goal, and they represent criteria that
guide the requirements
activity. These concerns are addressed within each appropriate
viewpoint. One of first
tasks is to determine whether the concern is indeed relevant to
the viewpoint. If it is
relevant, then a set of elaboration questions is used to explore
the concern further. The
concerns, which have arisen from experience of using ethnography
in systems design,
are:
. Paper work and
computer work
. Skill and the use
of local knowledge
180
. Spatial and
temporal organization
. Organizational
memory
Paperwork and computer work
These are embodiments of plans and procedures, and at the same
time are a
mechanism for developing and sharing an awareness of work.
Skill and the use of local knowledge
This refers to the “workarounds” that are developed in
organizations and are at the
heart of how the real work gets done.
Spatial and temporal organization
This concern looks at the physical layout of the workplace and
areas where time is
important.
Organizational memory
Formal documents are not the only way in which things are
remembered within an
organization. Individuals may keep their own records, or there
maybe local gurus.
Contextual design
Contextual design was another technique that was developed to
handle the collection
and interpretation of data from fieldwork with the intention of
building a softwarebased
product. It provides a structured approach to gathering and
representing
information from fieldwork such as ethnography, with the purpose
of feeding it into
design.
Contextual design has seven parts:
. Contextual inquiry
. Work modeling,
consolidation
. Work redesign
. User environment
design
. Mockup
. Test with customers
. Putting it into
practice
Contextual inquiry
Contextual inquiry, according to Beyer and Holtzblatt, is based
on a masterapprentice
model of learning: observing and asking questions of the users
as if she is
the master craftsman and he interviews the new apprentice. Beyer
and Holtzblatt also
enumerate four basic principles for engaging in ethnographic
interview:
Context:
Rather than interviewing the user in a clean white room, it is
important to interact
with and observe the user in their normal work environment, or
whatever physical
context is appropriate for the product. Observing users as they
perform activities and
questioning them in their own environment, filled with the
artifacts they use each day,
can bring the all-important details of their behaviors to light.
181
Partnership:
The interview and observation should take the tone of a
collaborative exploration with
the user, alternating between observation of and discussion f
its structure and details.
Interpretation:
Much of the work of the designer is reading between the lines of
facts gathered about
user’s behaviors, their environment, and what they say. These
facts must be taken
together as a whole, and analyzed by the designer to uncover the
design implications.
Interviewers must be careful, however, to avoid assumptions
based on their own
interpretation of the facts without verifying these assumptions
with users.
Focus:
Rather than coming to interviews with a set questionnaire or
letting the interview
wander aimlessly, the designer needs to subtly direct the
interview so as to capture
data relevant t design issues.
Improving on contextual inquiry
Contextual inquiry forms a solid theoretical foundation for
quantitative research, but
as a specific method it has some limitations and inefficiencies.
The following process
improvements result in a more highly leveraged research phase
that better set the
stage for successful design:
. Shortening the interview
process: contextual inquiry assumes
full day
interviews with users. The authors have found that interviews as
short as one
hour in duration are sufficient to gather the necessary user
data, provided that
a sufficient number of interviews (about six well-selected users
for each
hypothesized role or type) are scheduled. It is much easier and
more effective
to find a diverse set of users who will consent to an hour with
a designer than
it is to find users who will agree to spend an entire day.
. Using smaller design teams:
Contextual inquiry assumes a large
design
team that conducts multiple interviews in parallel, followed by
debriefing
sessions in which the full team participates. Experiments shows
that it is more
effective to conduct interviews sequentially with the same
designers in each
interview. This allows the design team to remain small (two or
three
designers), but even more important, it means that the entire
team interacts
with all interviewed users directly, allowing the members to
most effectively
analyzed and synthesized the user data.
. Identifying goals first:
Contextual inquiry, as described by
Beyer and
Holtzblatt, feeds a design process that is fundamentally
task-focused. It is
proposed that ethnographic interviews first identify and
prioritize user goals
before determining the tasks that relate to these goals.
. Looking beyond business
contexts: the vocabulary of
contextual inquiry
assumes a business product and a corporate environment.
Ethnographic
interviews are also possible in consumer domains, though the
focus of
questioning is somewhat different.
182
20.3 Preparing for
ethnographic interviews
As we discussed ethnography is term borrowed form anthropology,
meaning the
systematic and immersive study of human cultures. In
anthropology, ethnographic
researchers spend years living immersed in the cultures they
study and record.
Ethnographic interviews take the spirit of this type of research
and apply it on a micro
level. Rather than trying to understand behaviors and social
ritual of an entire culture,
the goal is understand the behaviors and rituals of people
interacting with individual
products.
Identifying candidates
Because the designer must capture an entire range of user
behaviors regarding a
product, it is critical that the designers identify and
appropriately diverse sample of
users and user types when planning a series of interviews. Based
on information
gleaned form stakeholders, SMEs, and literature reviews,
designers need to create a
hypothesis that serves as a starting point I determining what
sorts of users and
potential users to interview.
Kim Goodwin has coined this the persona hypothesis, because it
is the first step
towards identifying and synthesizing personas. The persona
hypothesis is based on
likely behavioral differences, not demographics, but takes into
consideration
identified target markets and demographics. The nature of the
product’s domain
makes a significant difference in how a persona hypothesis is
constructed. Business
users are often quite different than consumer users in their
behavior patterns and
motivations, and different techniques are used to build the
persona hypothesis in each
case.
The personal hypothesis
The persona hypothesis is a first cut at defining the different
kinds of users (and
sometimes customers) for a product in a particular domain. He
hypothesis serves as a
basis for an initial set of interviews; as interviews proceed,
new interviews may be
required if the data indicates the existence of user types not
originally identified.
The persona hypothesis attempts to address, at a high level,
these three questions:
. What different
sorts of people might use this product?
. How might their
needs and behaviors vary?
. What ranges of
behavior and types of environments need to be explored?
Roles in business and customer domains
Patterns of needs and behavior, and therefore types of users,
vary significantly
between business and technical, and consumer products. For
business products,
roles—common sets of tasks and information needs related to
distinct classes of
users—provide an important initial organizing principle. For
example, in an enterprise
portal, these search roles can be found:
. People who search
for content on the portal
. People who upload
and update content on the portal
. People who
technically administer the portal
183
In business and technical context, roles often map roughly to
job descriptions, so it is
relatively easy to get a reasonable first cut of user types to
interview by understanding
the kind of jobs held by users of the system.
Unlike business users, consumers don’t have concrete job
descriptions, and their use
of products tends to cross multiple contexts. Their roles map
more closely to lifestyle
choices, and it is possible for consumer users to assume
multiple roles even for a
single product in this sense. For consumers, roles can usually
better be expressed by
behavioral variables
Behavioral and demographic variables
Beyond roles, a persona hypothesis seeks to identify variables
that might distinguish
users based on their needs and behaviors. The most useful, but
most difficult to
anticipate without research, are behavioral variables: types of
behavior that behavior
concerning shopping that we might identify:
. Frequency of
shopping (frequent--infrequent)
. Desire t shop
(loves to shop—hates to shop)
. Motivation to shop
(bargain hunting—searching for just the right item)
Although consumer user types can often be roughly defined by the
combination of
behavioral variables they map to, behavioral variables are also
important for
identifying types of business and technical users. People within
a single business-role
definition may have different motivations for being there and
aspirations for what
they plan to do in the future. Behavioral variables can capture
this; through usually
not until user data has been gathered.
Given the difficulty in accurately anticipating behavioral
variables before user data is
gathered, another helpful approach in building a persona
hypothesis is making use of
demographic variables. When planning your interviews, you can
use market research
to identify ages, locations, gender, and incomes of the target
markets for the product.
Interviews should be distributed across these demographic
ranges.
Domain expertise versus technical expertise
One important type of behavioral distinction to note is the
difference between
technical expertise (knowledge of digital technology) and domain
expertise
(knowledge of a specialized subject area pertaining to a
product). Different users will
have varying amount of technical expertise; similarly, some
users of a product may be
less expert in their knowledge of the product’s domain (for
example, accounting
knowledge in the case of a general ledger application). Thus,
depending on who the
design target of the product is, domain support may be a
necessary part of the
product’s design, as well as technical ease of use.
Environmental variables
A final consideration, especially in the case of business
products, is the cultural
differences between organizations in which the users are
employed. Small companies,
for example, tend to have more interpersonal contact between
workers: huge
companies have layers of bureaucracy. These environmental
variables also fall into
ranges:
. Company size
(small—multinational)
. IT presence (ad
hoc—draconian)
184
. Security level
(lax--tight)
Like behavioral variables, these may be difficult to identify
without some domain
research, because patterns do vary significantly by industry and
geographic region.
20.4 Putting a plan together
After you have created a persona hypothesis, complete with
potential roles,
behavioral, demographic, and environmental variables, you then
need to create an
interview plan that can be communicated to the person in charge
of providing access
to users.
Each identified role, behavioral variable, demographic variable,
and environmental
variable identified in the persona hypothesis should be explored
in four to six
interviews (some time more if a domain is particular complex).
However, these
interviews can overlap: it is perfectly acceptable to interview
a female in her twenties
who loves to shop; this would count as an interview for each of
three different
variables: gender, age group, and desire to shop. By being
clever about mapping
variables to interviewee screening profiles, you can keep the
number of interviews to
a manageable number. |
|
|
|
|