Article ID:
19971208
Published:
December 1997
Author:
Walt Dickie
Seven rules for observational research: how to watch people do stuff
Editor's note: Walt Dickie is a partner at Creative & Response Research Services, Inc., Chicago.
Observational research, ethnography, or, in plain English, watching people do stuff, seems to be hot these days. Newsweek touts it ("Enough Talk," August 18, 1997), which means it's getting to be mainstream, but I find that a lot of clients aren't very comfortable with it.
Certainly, compared to traditional focus groups, mini-groups, or one-on-one interviews, observational research accounts for a pitiably small portion of most research budgets. Yogi Berra's famous line that "You can observe a lot just by watching" is widely acknowledged, but observation remains the most under-utilized qualitative technique in marketing research.
One of the reasons seems to be that many clients (and researchers) just don't know how to get value out of watching. Nothing sours people on a good approach more permanently than a few "interesting but useless" projects.
Learning from watching is, in fact, hard. If you ask a not-very-deep question in a focus group, you still may get a deep and revealing answer. But if you don't know how to think about what you'll see when you watch normal people doing stuff, you won't learn much from it. And in observational research, as in all qualitative research, it's the "thinking about" that's the key.
Since observation skills don't get sharpened up in real life the way questioning skills do, you need to train yourself to see, learn, and think when you watch people do stuff. It takes some practice, and some discipline. I don't pretend to have mastered the art, but I've learned some techniques that will help. So here are my "Seven Rules for Observational Research."
Look for the ordinary, not the extraordinary
Remember the qualitative project when the lady in the third seat on the right side of the table told the story that really made it all come clear to you? You know how you wait behind the mirror for the moderator to show the new concept so you can hear real consumers respond to it for the first time and all the questions that have been running around your mind for weeks will finally be answered? That's probably not going to happen in an observational study.
Most observational projects I've worked on have begun with a pretty nervous period while we all get past our first impression that nothing's happening! People aren't "doing" anything! They're just going about their business, and nothing that they're doing looks surprising! They're making lunch for their kids, the same way I would if I were in their shoes. They're waiting for their cars to be serviced, the same way I do. If my clients are along, they begin to get very antsy at this point, because they're seeing the same thing I am: nothing out of the ordinary.
Rule 1 for observational researchers: "Ordinary" is what you're there to observe. If you don't go looking for something extraordinary, you won't be so anxious when it doesn't appear. What you're really looking for are the insights hidden in "ordinary."
Observation gives you the chance to answer those questions such as "What do you do when that happens?" that come up all the time in focus groups. Suddenly you're not restricted by respondents' memories, or their reluctance to discuss the issue in a group, or their desire to conceal what they really do in order to present a more admirable face to the rest of the group.
Nothing people do is "natural"
The first time you try observational research, I guarantee that you'll find yourself wondering what there is about the things you're seeing that requires an explanation. You may watch people walking into a retail environment. They'll walk in, look around to get their bearings, walk over to a display or proceed down an aisle, maybe pick up an item or two or compare prices. "Of course," you'll say to yourself, "that's just what I'd do in their shoes. It's just common sense."
Rule 2: Whatever you saw could have happened differently. Your shoppers could have taken more time to get their bearings, or less time. They might have gone down a different aisle. They might have picked up more items, or not as many. They might have sought help from an employee. They might have, but they didn't. What they did needs to be explained.
Start noticing the regularities: do most people need a period of time to get their bearings when they come into the store? Where are they when they do this? Where do they look? What do they see there? Is there something about the store environment that makes them do things they way they're doing them? Is the way they're behaving the optimum way you want your customers to behave? Look at the "rule breakers." Who are they? What regularities are they defying?
Once you recognize that everything people do is the result of something, you can begin looking for that something. Maybe it's something about them. Or the people they're with. Or the environment they're in. Or something. How do you find it?
"I am the master of the obvious"
When I was first learning to conduct and analyze focus groups, Saul Ben Zeev, who founded C&R Research and is now its chairman, told me that the psychologist, Bruno Bettelheim, his teacher, referred to himself as "the master of the obvious." For Bettelheim, it was the secret of his success. Saul trotted that out whenever one of us was stuck for a place to start working on qualitative data (and still trots it out from time to time, now that I mention it).
Think about the last series of focus groups you conducted. What was the most obvious thing about what you saw and heard? That's where to start: If it's really obvious, then it must be really basic. What does it mean? How did people get there? What does it lead to? This was one of the first things I ever learned about qualitative, and remains one of the few really valuable generalizations I know about qualitative analysis.
The same thing is true about observation. Rule 3: Be the master of the obvious. Take the most obvious thing you've observed. Maybe you were watching people wait to have their cars fixed, and they "didn't do anything." Maybe they actually nodded off in the waiting area! Maybe they spent the whole time looking bored. That's about all you saw, and you've been poring over your field notes looking for something to get a handle on ever since.
Ask yourself why they were so bored - and remember that boredom isn't natural. Humans are the most curious creatures on earth. The room had a TV, a bunch of magazines, today's newspaper, some sales material and POP. Why didn't they get interested in any of that?
Were they interested in anything? Not really - they'd get up, check on the progress of their cars, then sit down and nod off again. But maybe that's it: all they were interested in was their cars - not the TV or the magazines or newspapers, and certainly not the POP. They wanted to see what was happening with their cars! And that's all they wanted to see. How's that for obvious?
Don't fear the details
The car repair story is real - I once spent a week watching people nod off waiting for their cars to be repaired. I was Jane Goodall and they were the chimps. And I got more and more panicky as I saw less and less "happening." Then I started thinking about the obvious things I could see.
One seat in the waiting room actually had a pretty good view of the car repair bays, and two or three had decent views. None of the others really let you see your car at all. Luckily, I had detailed notes: I knew where people had sat and how long they sat in each seat.
As I reconstructed scenes, it became more and more clear that people tended to sit in one of the "good" seats unless they were occupied or someone was sitting in the next seat and there were a lot of other empty seats available. When the waiting room was empty, I looked carefully at the carpet and the upholstery of the "good" seats and, sure enough, the wear patterns showed that what I had seen that week had been going on for a long time. There really were good seats and bad seats and you could tell which was which by checking out the sight lines.
Since the project was about developing criteria for understanding waiting-area designs, this was an important piece of information. A good design would put the car center stage and use the fact that customers were riveted to that stage as a way to organize the space and its communication elements. The path for the rest of the analysis was pretty clear.
Rule 4: God is in the details. Take good notes. Make videotapes. Think about where people walk, stand, sit, and look. For how long. Doing what. With whom.
The whole activity
After "master the obvious," the next most valuable thing I've learned about observation is, "identify the whole activity."
Here's an example: We were observing people using a newly designed gasoline pump on a summer day some years ago. One of the first "pay at the pump" designs, it allowed drivers to insert a credit or ATM card so they could pay without having to walk to the cashier's station. We noticed a number of motorists driving up to the pump, getting out and looking at it, then climbing back into their cars, apparently searching for something. They'd get back out of the car, go back to the pump, and read the directions - which seemed to present some difficulty. At a certain point we began walking up to people who had done this odd little in-and-out-of-the-car dance and asking what they were doing: "Looking for my reading glasses."
There are two points to this little vignette: The first is that a concept isn't reality. In this case we found that: (a) drivers don't wear reading glasses to drive (although lots wear sunglasses), so pump directions need to be designed for legibility even without glasses (or with the wrong glasses), (b) this particular design failed because the user couldn't make it work without reading the directions, and (c) respondents in several focus groups leading up to this test hadn't noticed the problem, since they had their reading glasses on, nor had the experienced researchers working on the design (us, unfortunately).
The second is that the observational perspective redefines the object of study. We went into this project thinking, as the client did, that we were going to study people pumping gas. But we quickly saw that pumping gas was part of a larger activity - people driving their cars from point A to point B - and that it had to be altered to fit into that activity. By failing to appreciate the demands of the whole activity, our client had neglected to think about glasses, or driving glasses vs. reading glasses, or sunglasses. All their research had abstracted pumping gas as the activity of interest - setting up experimental situations or taking pump designs into focus groups - and it took observation to put it back into its context.
Rule 5: The "whole activity" is the key to what the consumer is trying to accomplish. Think of activities as rings of context. Pumping gas takes place inside the "driving somewhere" ring, which takes place inside the "going home from work" ring, and so forth. Most research projects involve single activity units like pumping gas, or kitchen clean-up, or visiting a fast-food drive-thru; but these aren't generally whole activities. The whole activity is a set of behaviors that includes these small units plus at least one layer of context. It's "what's going on" from the consumer point of view, and it may be very different from what you (and your client) think is going on.
To get clues about a whole activity, look at how people enter the activity you're trying to observe, and how they exit. What's going on just before and just afterward? How do they get to the point you're interested in? What and who do they bring with them? What mental state are they in? How do they leave? What do they take with them and what do they leave behind?
The whole activity defines the parameters for the unit activity you want to understand.
Let the arrow find the target
It's a Zen idea. If you strive to place the arrow in the bullseye, you'll miss. If you let the arrow find the bullseye, it will fly unerringly. Observation, like all qualitative techniques, takes some Zen. If your task is too tightly defined, all you'll see is what you expected to see.
This doesn't mean that you should leave everything up in the air. The project won't define itself. You need to put together observation forms and some kind of debriefing protocol. You need to keep your notes up to date, and debrief yourself regularly. (I find that talking into a tape recorder as I drive from observation point to observation point works best for me.) But make sure you leave a lot of room for "other" in your materials.
Every observation form I make has space for what the client and I think the key issues and behaviors are; specific areas we want detailed information on. But every one also has a big space for comments or something equally open-ended. And as projects go on, those comment areas always seem to get more and more filled up.
This is where you'll find the things that suddenly seem obvious, and where all the context issues will land. I guarantee that you won't find either the most obvious aspects of the activity you're observing or the clues to the whole activity in the detailed parts of your note forms. In fact, if you do, I think you should be dubious about your findings because you've probably missed something (unless you're a lot luckier and smarter than I am). Rule 6: the most obvious things are obvious only in hindsight, and context doesn't appear until it appears in real action.
Marry observation with traditional qualitative
You can learn a lot by watching, to rephrase Mr. Berra, but you can learn even more by watching and talking.
There is absolutely no better way to go into focus groups, one-on-ones, or mini-groups than with your mind full of observational detail and insight. No better way to look at collages, photo albums or other projective vehicles than with a firm grounding in real behavior. Nor can I think of a better way to follow up on qualitative analysis than going out and observing people doing stuff. Each layer adds dimension and analytic richness, and the richer the stew of data, the more savory it is.
Observation isn't the be-all and end-all of research, and neither are focus groups or any other silver-bullet solutions - which seem to be proliferating at an almost frightening pace. We've been doing collage research here at C&R for quite a few years now, and we really like it. But it's not the One True Technique that you'd think it was if you believed its press. Same thing for giving people disposable cameras, having them wear beepers, or (I swear I heard a serious discussion of this) hypnotizing them to retrieve their deeply repressed memories (about their childhood experiences with a client's breakfast cereal, or whatever). Do you have the feeling that someone could sell focus groups done in a swimming pool because respondents would be more relaxed while floating in warm water?
My own feeling is that the deepest understanding social phenomena comes from combining an analysis of what people do with an analysis of what people have to say - observation plus traditional qualitative. So, as Rule 7, I offer that marriage as the strongest foundation on which to erect a qualitative analysis.
Powered by ScribeFire.
