Every piece of user research is part of an ongoing research program,
even if that program is informal. However, making a program formal
provides a number of advantages: It gives you a set of goals, a schedule
that stretches limited user-research resources, and results when they’re
needed most. It also helps you avoid unnecessary, redundant, or hurried
Over the next few weeks, we’ll introduce the three major parts of a user
- Goals: Why you’re doing the research.
- Budget: How much it’s going to cost.
- Schedule: When to do the research.
This week, we’ll outline the goal-setting process and explain how to
develop a workable budget for your user research plan.
Before you begin writing a research plan, determine corporate priorities
and set research goals that can meet them. Hopefully everyone in an
organization knows the general goals of a project (“Get more revenue,”
“Bleed less money,” “Get more exposure for the brand,” etc.), but there
will be a natural spectrum of understanding about what this means. Each
department team invariably has a different method for figuring out how
goals apply to them and for measuring their success at completing goals.
Thus, you’ll need to do some internal discovery before you can make a
complete research plan.
Identify Stakeholders and Their Needs
Stakeholders are the people in a department who represent what that
department wants and needs from the project. The departments working on
the project may not be the only ones affected by it, and the people you
talk to don’t have to be top managers. Seek out groups that are directly
affected and people who can provide accurate, honest representations of
how your plans will impact their group. Ask them these three sets of
- In terms of what you do on a day-to-day basis, what are the goals of
- Are there ways that it’s not meeting those goals? If so, what are
- Are there questions you want to have answered? If so, what are they?
If you get answers, how will you use them?
Stakeholder answers may be contradictory or conflicting, but they will
give you an idea of what aspects matter most and what questions your
research needs to address.
So, for example, say you identify three main needs as stated by
- Help visitors use the search engine better and more often.
- Tell people about related services of which they may be unaware.
- Guide visitors to country-specific information from the worldwide
Now let’s take these needs and prioritize them.
If you’re not sure how to prioritize, try the following exercise,
borrowed from Total Quality Management:
Make a column next to your list and label it “Importance.” Go down the
list and rate each item on a scale of one to five, where five means that
you must have an answer to this question in order for the project to
succeed and one means it would be nice to have an answer.
Next, make a second column and label it “Severity.” This will reflect
how bad the problem is today. Rate each item from one to five again,
where five represents terrible problems affecting the bottom line right
now, and one means information that would be good to know.
Now multiply the two entries in the two columns, and write the result
next to them in a third column called “Priority.” Ordering the list by
the third column amplifies the differences between priorities and often
helps sort items that would otherwise appear similar.
Next, rewrite the needs as user-specific research questions or
information to be gathered. Broaden narrow questions into general topics
to get at the root causes of problems.
The questions should be simple so that they give the most bang for the
buck. Pick one or two questions for each need that, when answered, will
address the underlying goal.
For example: If visitors need to use the search engine better and more
often, “How do people navigate the site, especially when they’re looking
for something specific?”
To inform people about related services of which they may be unaware,
“Where do people find related service information? Where do they expect
it?” or “Where would inclusion of service information be intrusive?”
To guide visitors to country-specific information from the worldwide
homepage, “What information is country-specific?” or “Where are non-U.S.
visitors expecting to find that information? How much is
language-specific versus country-specific?”
When you’re doing user research, don’t try to prove things right or
wrong. Never create goals that seek to justify a position or reinforce a
perspective. The process should aim to uncover what users really want
and how they really are, not whether an opinion (yours or a
stakeholder’s) is correct.
For more information about setting priorities, read Janice Fraser’s
essay on the subject.
The budget will be based on the cost of resources available to you, but
it’ll probably come in three big chunks: people’s time (including your
time and the research team’s time), recruiting and incentive costs, and
In my experience, useful times for calculating the duration of a
qualitative user research project such as a usability test (including
project management time and typical inefficiencies) are roughly as
- Preparation for a single project — ten hours
- Recruiting and scheduling — two to three hours per person
- Contextual inquiry/task analysis — five hours per person
- Focus groups — three hours per group
- Usability tests — three hours per participant
- Analyzing contextual inquiry/task analysis — five hours per person
- Analyzing focus group results — four hours per group
- Analyzing usability tests — two hours per person
- Preparing a report for email delivery — twelve hours
- Preparing a one-hour presentation — six hours
The effort necessary for quantitative research, such as surveys and log
analysis, will vary greatly based on the complexity of the task and the
tools and expertise available. What a good statistician can do in a
couple of hours can take days for someone with less training.
Incentive costs for interviewees vary, but as of Summer 2003 in San
Francisco they tend to fall around $100 per person per ninety-minute
Likewise, equipment costs will vary based on how much documentation and
space you require. It’s possible to spend $5,000 a day renting a
usability lab for a series of highly documented usability tests, or
testing can be free if you conduct a focus group in your conference room
with a borrowed video camera and use the tape from last years’
forgettable holiday party.
Revise your research plan every time new knowledge comes in. Everything
will change as your team and the company’s understanding of the user’s
experience grows. The research goals, especially, should be
re-evaluated, refined, and rewritten to take every piece of additional
knowledge into account.
Since the research you do is likely to affect your understanding of a
number of different research goals, you should consolidate your
knowledge about the user experience whenever possible. A good way to do
this is to create a mini-site for your intranet that contains all the
reports and goals. Link each goal to the information that applies, and
enter all problems into the development group’s bug-tracking software.
Eventually, you should have a set of interlinked documents that,
together, make up a more-or-less complete picture of your user