Contents
Are you a product manager or designer interested in learning more about user experience (UX) research methods and don’t have a dedicated UX researcher in your organization to support your team? Maybe you’re looking to learn more about how to reduce business risk or increase adoption. Whether you need the rundown on research methods to make informed product design decisions, or are generally curious about how to collect insights, I’ll do my best to describe the surface.
Most formal research has a distinct criteria for rigor, or accuracy. Rigor is often defined by concepts of reliability and validity. Reliability as in: “Do we feel that what we are seeing is consistent and repeatable under similar circumstances?” And, validity as in: “Do we feel this is accurate? Does this ring true regardless of repeatability?”
A casual understanding between the two main types of research can help you choose the best exercises for your effort, or at least send you down a proper digital rabbit hole to learn more. Perhaps a slightly better use of screen time than deviled-egg TikTok (I’m projecting). The two types of research we’ll discuss are Quantitative and Qualitative. Mixed-Methods is a third type of research covered in a future post.
Quantitative Research
Quantitative answers questions such as, “How many, how much and how often?” Quant strives for consistent and measurable observations through numbers and scoring. These numeric values are observed to drive insights and analysis, with a goal of statistical significance or generalizability in most cases. Generalizability refers to when your sample is reasonably representative of your population, and the numeric thresholds that are agreed-upon as acceptable (confidence intervals).
Notable quant-oriented folks outside of academia can often be found in finance, such as “Quant King” Jim Simons, a hedge fund founder. The subject of Michael Lewis’ Moneyball, Houston Rockets general manager Daryl Morey, is famous for his use of predictive analytics to bolster the qualitative opinions of talent scouts when drafting players for the NBA.
Examples of quant in corporate environments include A/B testing (What led to more conversions: Subject Line A vs Subject Line B?), the rating portion of Net Promoter Scores (NPS) (How likely are you to recommend this product to a friend?) and user analytics.
Qualitative Research
Rather than the quantity of data, qualitative research focuses on the qualities of the data, the context and the why’s. Qual is not typically meant to be generalizable or representative of a large population or data set. More significantly, qual is intended to shed light on the narrative, the thought processes and the cognitive triggers that influence behavior and expectations.
Examples of qualitative research are focus groups (Why does this product resonate with you?), interviews and the open-ended question that accompanies an NPS rating (“Tell us more, any other feedback you’d like to share?”).
A well-known example of a qualitative researcher, Brené Brown, specializes in shame and vulnerability and uses grounded theory (which can uncover social processes and behaviors) as her primary method. According to Brown,
“I thought, you know, I am a storyteller. I’m a qualitative researcher. I collect stories; that’s what I do. Maybe stories are just data with a soul. And maybe I’m just a storyteller.”
The founders of behavioral economics, Amos Tversky and Daniel Kahneman also leaned qualitative. The two researchers brought a qualitative perspective to cognitive research, particularly around rational choice theory; by challenging the notion that humans are capable of making decisions in an economic, machine-like manner. The researchers shared evidence that bias can influence decision-making, particularly in how information is framed. A UX example of this is when we present a ‘Goldilocks’ of choices on a payment plan screen. If you want users to choose ‘x,’ sandwich ‘x’ between higher and lower tiers, regardless of intrinsic value. The information is framed in a way to steer behavior, by introducing bias and overriding a closer evaluation.
Areas of Focus
Across quantitative, qualitative and mixed-methods is a further subset of research approaches inclusive of generative, evaluative, attitudinal and behavioral lenses; and can be leveraged throughout the product development lifecycle.
- Generative implies blank slate, trying something new, mining for ideas. “Are there any barriers to adoption if we roll out a new technology at our company?”
- Evaluative typically pertains to an existing or known problem or product and vetting whether the solution is successful. “How do users feel about the app updates?”
- Attitudinal is a little more abstract and gauges user sentiment, great for assessing appetite in discovery efforts. “How open are our users to accepting this new process change?”
- Behavioral is often task-oriented, and great for understanding triggers and motivations. “Do users log back in after we send them a cart abandonment email?”
This UX research intro is far from comprehensive. Each type of research has its own set of methods, theories, and best practices. However, getting the basics down can help you choose the most effective approaches to answer your design questions. And remember,
“Research is just formalized curiosity. It is poking and prying with a purpose” — Zora Neale Hurston.
