
“…the only thing that matters is whether people can use machines we design responsibly, with intention, that is the meaningful question, everything else is fantasy.”
-Jaron Lanier, "Father of VR" | IQ2 Debate: Don't Trust The Promise Of Artificial Intelligence’
https://youtu.be/Qqc0t8ghvis?t=802
​
UX RESEARCH PROCESS
STAKEHOLDER/
ENVIRONMENTAL ANALYSIS
Coordination with research scientists, marketing teams, designers, engineers and product managers to identify research questions and formalizing the shared goals and assumptions. These include:
​
-
KPIs/SMART Goals
-
T-Shirt Sized Research Goals/Questions
-
Subject Matter Expert (SME) Interviews and Workshops
-
Stakeholder assumptions and testable hypotheses (may include hypothesized use cases.)
-
Evaluation for new or existing UX Metrics (ex: meeting, exceeding, not-met, unknown)
-
Target Market Demographics (if not specified in assumptions)
-
Commitment of resources (observation, design, participant compensation, tools, stimuli links, reporting collaborators etc.
​
Pre or Post Environmental Scan/Non-Experimental Research:
​​​
-
Competitive Analysis
-
Market Research
-
Business, Social Science or Company Literature Reviews
-
Meta-analysis of Studies
-
Heuristic Reviews
-
Cognitive Walkthroughs
-
Past meeting minutes
​
SCOPING & EXPERIMENTAL DESIGN
Research Plan includes:
​
-
Main Research Questions 1-2, Formal Hypotheses, Assumptions model/Causal Diagrams
-
Participant Screening Logic, Questions, Consent Forms
-
Stimuli Tracking design
-
Discussion Guide Writing (Scenarios, Tasks, Questions)
-
Data Capture Design, Protocols (ex. Success/Struggle/Fail Criteria)
-
Stimuli Creation and Human Resource Requirements
-
Testing & Reporting Timelines
-
Methodology/Mixed methods specifications (examples):
-
Moderated/Unmoderated/Remote/Live
-
Usability Testing
-
Survey Design
-
Counterbalancing, randomization
-
SUS/validated indices
-
Card Sorts
-
Tree Tests
-
Ethnography​
-
1st Click Tests
-
5 Second Value Prop Testing
-
Heat Maps
-
Persona Mining/Narrative Testing
-
Diary Studies
-
Interviews
-
Concept Testing
-
A/B Testing
-
OBSERVATION & DATA ANALYSIS
Testing Environment Setup
​
-
For onsite studies: Testing multi-camera setup, two-way mirror labs, live observation streaming
-
Piloting session timing and between session debrief schedules, Assessor-Moderator syncing protocols
-
See ethnography highlights below
​
Receiving, managing and analyzing data from user studies with:
​​
-
Null Hypothesis Testing & Interpretation with bivariate tests
-
Correlation, T-Tests, ANOVA, Predictive Analytics (OLS Multivariate Regression, Binary Interactions, Logistic Regression)
-
-
Qualitative Analysis
-
Grounded Theory/Text Analysis -
-
Open Coding
-
Axial Coding
-
Parent/Sibling Theming
-
Video Analysis
-
-
Tools:
-
STATA,
-
SPSS,
-
Qualtrics (including Stats IQ)
-
UserTesting.com
-
Chalkmark
-
​
Ethnography:
​
-
Location(s) of Expected Population Traffic
-
Key Timing
-
Gatekeepers
-
Anthropological/Ergonomic Issues or Barriers
-
Fly-on-the-wall tactics
-
Blinded Conversations
-
Artifacts/Tools/Devices used and variations
-
Workarounds; unique adjustments from subjects
-
Visual/Audio/Tactile Recording Methods
-
Group Dynamics, rules, roles, and socio-political observations
REPORTING & COLLABORATION
Identify, promote and share user testing best practices and techniques with the broader organization:
​​
-
Executive summary
-
Abstracts
-
Tl;DRs
-
-
Methodology
-
Key findings
-
Hypothesis statuses:
-
SUPPORTS​
-
NOT SUPPORTED
-
NEEDS RESEARCH
-
-
Results on UX Metrics (as defined in Proposal)
-
Storyboards/Customer Journey Maps
-
User Themes
-
New or updates to Personas or archetypes
-
Red Routes
-
Use Case Maps
-
Component/Design element pain points
-
Wireframes
-
Test Artifacts
-
Lo-Fi Single Step Prototyping for reports (PPT, Keynote)
​
Collaborate cross-functionally with design, engineering and product development groups​
​​
-
1:1 Participatory Design sessions
-
Empathy Maps
-
Service Blue Printing
-
Highlight Reel Production
-
Design Workshops/SME Visioning (Designer uses Figma, Sketch, Axure, Invision, etc., Research Moderates and acts)
​

Tell me more...
STAKEHOLDER/
ENVIRONMENTAL ANALYSIS
Coordination with research scientists, marketing teams, designers, engineers and product managers to identify research questions and formalizing the shared goals and assumptions. These include:
​
-
KPIs/SMART Goals
-
T-Shirt Sized Research Goals/Questions
-
Subject Matter Expert (SME) Interviews and Workshops
-
Stakeholder assumptions and testable hypotheses (may include hypothesized use cases.)
-
Evaluation for new or existing UX Metrics (ex: meeting, exceeding, not-met, unknown)
-
Target Market Demographics (if not specified in assumptions)
-
Commitment of resources (observation, design, participant compensation, tools, stimuli links, reporting collaborators etc.
​
Pre or Post Environmental Scan/Non-Experimental Research:
​​​
-
Competitive Analysis
-
Market Research
-
Business, Social Science or Company Literature Reviews
-
Meta-analysis of Studies
-
Heuristic Reviews
-
Cognitive Walkthroughs
-
Past meeting minutes
​
SCOPING & EXPERIMENTAL DESIGN
Research Plan includes:
​
-
Main Research Questions 1-2, Formal Hypotheses, Assumptions model/Causal Diagrams
-
Participant Screening Logic, Questions, Consent Forms
-
Stimuli Tracking design
-
Discussion Guide Writing (Scenarios, Tasks, Questions)
-
Data Capture Design, Protocols (ex. Success/Struggle/Fail Criteria)
-
Stimuli Creation and Human Resource Requirements
-
Testing & Reporting Timelines
-
Methodology/Mixed methods specifications (examples):
-
Moderated/Unmoderated/Remote/Live
-
Usability Testing
-
Survey Design
-
Counterbalancing, randomization
-
SUS/validated indices
-
Card Sorts
-
Tree Tests
-
Ethnography​
-
1st Click Tests
-
5 Second Value Prop Testing
-
Heat Maps
-
Persona Mining/Narrative Testing
-
Diary Studies
-
Interviews
-
Concept Testing
-
A/B Testing
-
OBSERVATION & DATA ANALYSIS
Testing Environment Setup
​
-
For onsite studies: Testing multi-camera setup, two-way mirror labs, live observation streaming
-
Piloting session timing and between session debrief schedules, Assessor-Moderator syncing protocols
-
See ethnography highlights below
​
Receiving, managing and analyzing data from user studies with:
​​
-
Null Hypothesis Testing & Interpretation with bivariate tests
-
Correlation, T-Tests, ANOVA, Predictive Analytics (OLS Multivariate Regression, Binary Interactions, Logistic Regression)
-
-
Qualitative Analysis
-
Grounded Theory/Text Analysis -
-
Open Coding
-
Axial Coding
-
Parent/Sibling Theming
-
Video Analysis
-
-
Tools:
-
STATA,
-
SPSS,
-
Qualtrics (including Stats IQ)
-
UserTesting.com
-
Chalkmark
-
​
Ethnography:
​
-
Location(s) of Expected Population Traffic
-
Key Timing
-
Gatekeepers
-
Anthropological/Ergonomic Issues or Barriers
-
Fly-on-the-wall tactics
-
Blinded Conversations
-
Artifacts/Tools/Devices used and variations
-
Workarounds; unique adjustments from subjects
-
Visual/Audio/Tactile Recording Methods
-
Group Dynamics, rules, roles, and socio-political observations
REPORTING & COLLABORATION
Identify, promote and share user testing best practices and techniques with the broader organization:
​​
-
Executive summary
-
Abstracts
-
Tl;DRs
-
-
Methodology
-
Key findings
-
Hypothesis statuses:
-
SUPPORTS​
-
NOT SUPPORTED
-
NEEDS RESEARCH
-
-
Results on UX Metrics (as defined in Proposal)
-
Storyboards/Customer Journey Maps
-
User Themes
-
New or updates to Personas or archetypes
-
Red Routes
-
Use Case Maps
-
Component/Design element pain points
-
Wireframes
-
Test Artifacts
-
Lo-Fi Single Step Prototyping for reports (PPT, Keynote)
​
Collaborate cross-functionally with design, engineering and product development groups​
​​
-
1:1 Participatory Design sessions
-
Empathy Maps
-
Service Blue Printing
-
Highlight Reel Production
-
Design Workshops/SME Visioning (Designer uses Figma, Sketch, Axure, Invision, etc., Research Moderates and acts)
​
STAKEHOLDER/
ENVIRONMENTAL ANALYSIS
Coordination with research scientists, marketing teams, designers, engineers and product managers to identify research questions and formalizing the shared goals and assumptions. These include:
​
-
KPIs/SMART Goals
-
T-Shirt Sized Research Goals/Questions
-
Subject Matter Expert (SME) Interviews and Workshops
-
Stakeholder assumptions and testable hypotheses (may include hypothesized use cases.)
-
Evaluation for new or existing UX Metrics (ex: meeting, exceeding, not-met, unknown)
-
Discover Target Market Demographics (if not specified in assumptions)
-
Commitment of resources (observation, design, participant compensation, tools, stimuli links, reporting collaborators etc.
​
Secondary Research Outputs:
​​​
-
Competitive Analysis
-
Market Research
-
Business, Social Science or Company Literature Reviews
-
Meta-analysis of Studies
-
Heuristic Reviews
-
Cognitive Walkthroughs
-
Past meeting minutes
​
SCOPING & EXPERIMENTAL DESIGN
Research Plan includes:
​
-
Main Research Questions 1-2, Formal Hypotheses, Assumptions model/Causal Diagrams
-
Participant Screening Logic, Questions, Consent Forms
-
Stimuli Tracking design
-
Discussion Guide Writing (Scenarios, Tasks, Questions)
-
Data Capture Design, Protocols (ex. Success/Struggle/Fail Criteria)
-
Stimuli Creation and Human Resource Requirements
-
Testing & Reporting Timelines
-
Methodology/Mixed methods specifications (examples):
-
Moderated/Unmoderated/Remote/Live
-
Usability Testing
-
Survey Design
-
Counterbalancing, randomization
-
SUS/validated indices
-
Card Sorts
-
Tree Tests
-
Ethnography​
-
1st Click Tests
-
5 Second Value Prop Testing
-
Heat Maps
-
Persona Mining/Narrative Testing
-
Diary Studies
-
Interviews
-
Concept Testing
-
A/B Testing
-
OBSERVATION & DATA ANALYSIS
Testing Environment Setup
​
-
For onsite studies: Testing multi-camera setup, two-way mirror labs, live observation streaming
-
Piloting session timing and between session debrief schedules, Assessor-Moderator syncing protocols
-
See ethnography highlights below
​
Receiving, managing and analyzing data from user studies with:
​​
-
Null Hypothesis Testing & Interpretation with bivariate tests
-
Correlation, T-Tests, ANOVA, Predictive Analytics (OLS Multivariate Regression, Binary Interactions, Logistic Regression)
-
-
Qualitative Analysis
-
Grounded Theory/Text Analysis -
-
Open Coding
-
Axial Coding
-
Parent/Sibling Theming
-
Video Analysis
-
-
Tools:
-
STATA,
-
SPSS,
-
Qualtrics (including Stats IQ)
-
UserTesting.com
-
Chalkmark
-
​
Ethnography:
​
-
Location(s) of Expected Population Traffic
-
Key Timing
-
Gatekeepers
-
Anthropological/Ergonomic Issues or Barriers
-
Fly-on-the-wall tactics
-
Blinded Conversations
-
Artifacts/Tools/Devices used and variations
-
Workarounds; unique adjustments from subjects
-
Visual/Audio/Tactile Recording Methods
-
Group Dynamics, rules, roles, and socio-political observations
REPORTING & COLLABORATION
Identify, promote and share user testing best practices and techniques with the broader organization:
​​
-
Executive summary
-
Abstracts
-
Tl;DRs
-
-
Methodology
-
Key findings
-
Hypothesis statuses:
-
SUPPORTS​
-
NOT SUPPORTED
-
NEEDS RESEARCH
-
-
Results on UX Metrics (as defined in Proposal)
-
Storyboards/Customer Journey Maps
-
User Themes
-
New or updates to Personas or archetypes
-
Red Routes
-
Use Case Maps
-
Component/Design element pain points
-
Wireframes
-
Test Artifacts
-
Lo-Fi Single Step Prototyping for reports (PPT, Keynote)
​
Collaborate cross-functionally with design, engineering and product development groups​
​​
-
1:1 Participatory Design sessions
-
Empathy Maps
-
Service Blue Printing
-
Highlight Reel Production
-
Design Workshops/SME Visioning (Designer uses Figma, Sketch, Axure, Invision, etc., Research Moderates and acts)
​