|
|
|
|
|
|
Lesson#19
|
USER RESEARCH-1
|
|
|
|
As the aim of this lecture is to introduce you the study of
Human Computer
Interaction, so that after studying this you will be able to:
. Understand the
difference in qualitative and quantitative research
. Discuss in detail
qualitative research technique
The outcome of any design effort must ultimately be judged by
how successfully it
meets the requirements of both the user and the organization
that commissioned it. No
matter how skillful and creative the designer. If she does not
have a clear and detailed
knowledge of the users she is designing for, what the
constraints of the problem are,
and what business or organizational goals the design is hoping
to achieve, she will
have little chance of success.
What and how questions like these are best answered by
qualitative research, not
metrics or demographics (though these also have their purpose).
There are many types
of qualitative research, each of which plays an important role
in filling in a picture of
the design landscape of a product.
Qualitative versus Quantitative Research
Research is a word that most people associate with science and
objectivity. This
association isn’t incorrect, but it biases many people towards
the notion that the only
valid sort of research is the kind that yields the supposed
ultimate in objectivity:
quantitative data. The notion that numbers don’t lie is
prevalent in the business and
engineering communities, even though we all rationally
understand that numbers—
especially numbers ascribed to human activities—can be
manipulated or reinterpreted
at least as dramatically as words.
Data gathered by the hard sciences like physics is simply
different from that gathered
on human activities: electrons don’t have moods that vary from
minute to minute, and
the tight controls physicists place on their experiments to
isolate observed behaviors
are next to impossible in the social sciences. Any attempt to
reduce human behavior
to statistics is likely to overlook important nuances, which
though they might not
directly affect business plans, do make an enormous difference
to the design of
products. Quantitative research can only answer questions about
how much or how
many along a few reductive axes. Qualitative research can tell
you about what, how
and why in rich, multivariate detail.
Social scientists have long realized that human behaviors are
too complex and subject
to too many variables to rely solely on quantitative data to
understand them. Usability
practitioners, borrowing techniques from anthropology and other
social sciences, have
developed many alternative methods for gathering useful data on
user behaviors to a
more pragmatic end: to help create products that better serve
user need.
170
The value of qualitative research
Qualitative research helps us understand the domain, context and
constraints of a
product in different, more useful ways than quantitative
research does. It also quickly
helps us identify patterns of behavior among users and potential
users of a product
much more quickly and easily than would be possible with
quantitative approaches. In
particular, qualitative research helps us understand:
. Existing products,
and how they are used.
. Potential users of
new or existing products, and how they currently approach
activities and problems the new product design hopes to address
. Technical,
business, and environmental contexts—the domain—of the product
to be designed
. Vocabulary and
other social aspects of the domain in question
Qualitative research cab also help the progress of design
projects by:
. Providing
credibility and authority to the design team, because design
decisions can be traced to research results
. Uniting the team
with a common understanding of domain issues and user
concerns
. Empowering
management to make more informed decisions about product
design issues that would otherwise be based on guesswork or
personal
preference
It is the experienced that qualitative method, in addition to
the benefits described
above, tend to be faster, less expensive, more flexible, and
more likely than their
quantitative counterparts t provide useful answers to impotent
questions that leads t
superior design:
. What problems are
people encountering with their current ways of doing what
the product hopes to do?
. Into what broader
contexts in people’s lives does the product fit and how?
. What are the basic
goals people have in using the product, and what basic
tasks help people accomplish them?
19.1 Types of qualitative
research
Social science and usability texts are full of methods and
techniques for conducting
qualitative research. We will discuss following qualitative
research techniques:
. Stakeholder
interviews
. Subject matter
expert (SME) interviews
. User and customer
interviews
. User
observation/ethnographic field studies
. Literature review
. Product/prototype
and competitive audits
171
Stakeholder interviews
Research for any new product design, through it must end with
understanding the
user, should start by understanding the business and technical
context in which the
product will be built. This is necessary not only to ensure a
viable and feasible end
result, but also to provide a common language and understanding
among the design
team, management, and engineering teams.
Stakeholders are any key members of the organization
commissioning the design
work, and typically include managers and key contributors from
engineering, sales,
product marketing, marketing communications, customer support,
and usability. They
may also include similar people from other organizations in
business partnership with
the commissioning organization, and executives. Interviews with
stakeholders should
occur before any user research begins.
It is usually most effective to interview each stakeholder
one-on-one, rather than in a
larger, cross-departmental group. A one-on-one setting promotes
candor on the part of
the stakeholder, and ensure that individual views are not lost
in a crowd. Interviews
need not last longer than about an hour, though follow-up
meetings may be called for
if a particular stakeholder is identified as an exceptionally
valuable source of
information.
The type of information that is important to gather from
stakeholders includes:
. What is the
preliminary vision of the product from each stakeholder
perspective? As in the fable of the blind men and the elephant,
you may find
that each business department has a slightly different and
slightly incomplete
perspective on the product to be designed. Part of the design
approach must
therefore involve harmonizing these perspectives with those of
users and
customers.
. What is the budget
and schedule? The answer to this question often provides a
reality check on the scope of the design effort and provides a
decision point
for management if user research indicates a greater scope is
required.
. What are the
technical constraints? Another important determinant of design
scope is a firm understanding of what is technically feasible
given budget,
time, and technology.
. What are the
business drivers? It is important for the design team to
understand what the business is trying to accomplish. This again
leads to a
decision point, should user research indicate a conflict between
business and
user needs. The design must, as much as possible, create a
win-win situation
for users, customers, and providers of the product.
. What are the
stakeholders’ perceptions of the user? Stakeholders who have
relationships with users (such as customer support
representative) may have
important insights on users that will help you to formulate your
user research
plan. You may also find that there are significant disconnects
between some
stakeholders’ perceptions of their users and what you discover
in your
research. This information can become an important decision
point for
management later in the process.
Understanding these issues and their impact on design solutions
helps you as a
designer to better serve your customer, as well as users of the
product. Building
172
consensus internally will help you to articulate issues that the
business as a whole may
not identified, build internal consensus that is critical for
decision making later in the
design process, and build credibility for your design team.
Subject matter expert (SME) interviews
Some stakeholders may also be subject matter experts (SMEs):
experts on the domain
within which the product you are designing will operate. Most
SMEs were users of
the product or its predecessors at one time, and may now be
trainers, managers, or
consultants. Often they are experts hired by stakeholders,
rather than stakeholders
themselves. Similar to stakeholders, SMEs can provide valuable
perspective on a
product and its users, but designers should be careful to
recognize that SMEs
represent a somewhat skewed perspective. Some points to consider
about using SMEs
are:
. SMEs are expert
users. Their long experience with a product or its domain
mean that they may have grown accustomed to current
interactions. They may
also lean towards expert controls rather than interactions
designed for
perpetual intermediate perspective. SMEs are often not current
users of the
product, and may have more of a management perspective.
. SMEs are
knowledgeable, but they aren’t designers. They may have many
ideas on how to improve a product. Some of these may be valid
and valuable,
but the most useful pieces of information to glean from these
suggestions are
the causative problems that lead to their proposed solutions.
. SMEs are necessary
in complex or specialized domains such as medical,
scientific, or financial services. If you are designing for a
technical or
otherwise specialized domain, you will likely need some guidance
from
SMEs, unless you are one yourself. Use SMEs to get information
on complex
regulations and industry best practices. SME knowledge of user
roles and
characteristics is critical for planning user research in
complex domains.
. You will want
access to SMEs throughout the design process. If your product
domain requires use of SMEs, you should be able to bring them in
at different
stages of the design to help perform reality checks on design
details. Make
sure that you secure this access in your early interviews.
User and customer interviews
It is easy to confuse users with customers. For consumer
products, customers are
often the same as users, but in corporate or technical domain,
users and customers
rarely describe the same sets of people. Although both groups
should be interviewed,
each has its own perspective on the product that needs to be
factored quite differently
into an eventual design.
Customers of a product are those people who make the decision to
purchase it. For
consumer product, customers are frequently users of the product;
although for
products aimed at children or teens, the customers are parents
or other adult
supervisors of children. In the case of most enterprise or
technical products, the
customer is someone very different from the user—often an IT
manager—with
distinct goals and needs. It’s important to understand customers
and their goals in
order to make a product viable. It is also important to realize
that customers seldom
173
actually use the product themselves, and when they do, they use
it quite differently
than the way their users do.
When interviewing customers, you will want to understand:
. Their goals in
purchasing the product
. Their frustrations
with current solutions
. Their decision
process for purchasing a product of the type you’re designing
. Their role in
installation, maintenance, and management of the product
. Domain related
issues and vocabulary
Like SMEs, customers may have many opinions about how to improve
the design of
the product. It is important to analyze these suggestions, as in
the case of SMEs, to
determine what issues or problems underline the ideas offered,
because better, more
integrated solutions become evident later in the design process.
Users of a product should be the main focus of the design
effort. They are the people
(not their managers or support team) who are personally trying
to accomplish
something with the product. Potential users are people who do
not currently use the
product, but who are good candidates for using it in the future.
A good set of user
interviews includes both current users (if the product already
exists and is being
revised) and potential users (users of competitive products and
non-automated
systems of appropriate). Information we are interested in
learning from users includes:
. Problems and
frustrations with the product (or analogous system if they are
potential users)
. The context of how
the product fits into their lives or workflow: when, why,
and how the product is used, that is, patterns of user behavior
with the product.
. Domain knowledge
from a user perspective: what do users need to know to
accomplish their jobs
. A basic
understanding of the users’ current tasks: both those the product
requires and those it doesn’t support
. A clear
understanding of user goals: their motivations and expectations
concerning use of the product
User observation
Most people are incapable of accurately assessing their own
behaviors, especially
outside of the context of their activities. It then follows that
interviews performed
outside the context of the situations the designer hopes to
document will yield less
complete and less accurate data. Basically, you can talk to
users about how they think
they behave, or you can observe it first hand. The latter route
provides superior
results.
Many usability professionals make use of technological aids such
as audio or video
recorders to capture what users say and do. Care must be taken
not to make these
technologies too obtrusive: otherwise the users will be
distracted and behave
differently than they would off-tape.
Perhaps the most effective technique for gathering qualitative
user data combines
interviews and observation, allowing the designer to ask
clarifying questions and
direct inquiries about situations and behaviors they observe in
real-time.
174
Literature review
In parallel with stakeholder interviews, the design team should
review any literature
pertaining to the product or its domain. This can and should
include product
marketing plans, market research, technology specifications and
white papers,
business and technical journal articles in the domain,
competitive studies. Web
searches for related and competing products and news, usability
study results and
metrics, and customer support data such as call center
statistics.
The design team should collect this literature, use it as a
basis for developing
questions to ask stakeholders and SMEs, and later use it to
supply addition domain
knowledge and vocabulary, and to check against compiled user
data.
Product and competitive audits
Also in parallel to stakeholder and SME interviews, it is often
quite helpful for the
design team to examine any existing version or prototype of the
product, as well as its
chief competitors. Doing so gives the design team a sense of the
state of the art, and
provides fuel for questions during the interviews. The design
team, ideally, should
engage in an informal heuristic or expert review of both the
current and competitive
interfaces, comparing each against interaction and visual design
principles. This
procedure both familiarizes the team with the strengths and
limitations of what is
currently available to users, and provides a general idea of the
current functional
scope of the product.
19.2 Ethnographic interviews
It has been experienced that combination of one-on-one
interviews and work/lifestyle
observation is the most effective and efficient tool in a
designer’s arsenal for
gathering qualitative data about users and their goals. The
technique of ethnographic
interviews is a combination of immersive observation and
directed interview
technique.
Hugh Beyer and Karen Holtzblatt have pioneered an ethnographic
interviewing
technique that they call contextual inquiry. Their method has,
for good reason, rapidly
gained traction in the industry, and provides a sound basis for
qualitative user
research.
Contextual inquiry
Contextual inquiry, according to Beyer and Holtzblatt, is based
on a masterapprentice
model of learning: observing and asking questions of the users
as if she is
the master craftsman and he interviews the new apprentice. Beyer
and Holtzblatt also
enumerate four basic principles for engaging in ethnographic
interview:
Context:
Rather than interviewing the user in a clean white room, it is
important to interact
with and observe the user in their normal work environment, or
whatever physical
context is appropriate for the product. Observing users as they
perform activities and
questioning them in their own environment, filled with the
artifacts they use each day,
can bring the all-important details of their behaviors to light.
175
Partnership:
The interview and observation should take the tone of a
collaborative exploration with
the user, alternating between observation of and discussion f
its structure and details.
Interpretation:
Much of the work of the designer is reading between the lines of
facts gathered about
user’s behaviors, their environment, and what they say. These
facts must be taken
together as a whole, and analyzed by the designer to uncover the
design implications.
Interviewers must be careful, however, to avoid assumptions
based on their own
interpretation of the facts without verifying these assumptions
with users.
Focus:
Rather than coming to interviews with a set questionnaire or
letting the interview
wander aimlessly, the designer needs to subtly direct the
interview so as to capture
data relevant t design issues.
Improving on contextual inquiry
Contextual inquiry forms a solid theoretical foundation for
quantitative research, but
as a specific method it has some limitations and inefficiencies.
The following process
improvements result in a more highly leveraged research phase
that better set the
stage for successful design:
. Shortening the interview
process: contextual inquiry assumes
full day
interviews with users. The authors have found that interviews as
short as one
hour in duration are sufficient to gather the necessary user
data, provided that
a sufficient number of interviews (about six well-selected users
for each
hypothesized role or type) are scheduled. It is much easier and
more effective
to find a diverse set of users who will consent to an hour with
a designer than
it is to find users who will agree to spend an entire day.
. Using smaller design teams:
Contextual inquiry assumes a large
design
team that conducts multiple interviews in parallel, followed by
debriefing
sessions in which the full team participates. Experiments shows
that it is more
effective to conduct interviews sequentially with the same
designers in each
interview. This allows the design team to remain small (two or
three
designers), but even more important, it means that the entire
team interacts
with all interviewed users directly, allowing the members to
most effectively
analyzed and synthesized the user data.
. Identifying goals first:
Contextual inquiry, as described by
Beyer and
Holtzblatt, feeds a design process that is fundamentally
task-focused. It is
proposed that ethnographic interviews first identify and
prioritize user goals
before determining the tasks that relate to these goals.
. Looking beyond business
contexts: the vocabulary of
contextual inquiry
assumes a business product and a corporate environment.
Ethnographic
interviews are also possible in consumer domains, though the
focus of
questioning is somewhat different. |
|
|
|
|