CURIOSITY: A CORE VALUE

“Tell me more…” One of my UX research mentors is the master of this phrase. I observed numerous sessions where these three simple words were expertly used to seek more from a research participant. His goal was to gain a better understanding of whatever the participant had just said and ultimately, uncover the answers to the research questions he was tasked with finding.

This act of digging deeper to uncover more or learn is one of my favorite parts of research. It is also another core value of my UX career; CURIOSITY.

curiosity.png

Last week I connected with a college student who recently began her own pursuit of a UX career. She reached to me “to get a more authentic understanding of this field,” and for some advice on seeking a design or research internship.

One question she asked reminded me of the importance of curiosity in UX research. “What have you found to be the most crucial questions to ask participants in usability testing, and have any results been surprising? (In that maybe some participants have experienced/noticed something that the designers/researchers didn't think the same way about.)”

Here’s how I answered her questions:

The questions asked during testing are so dependent on the product, project goals, and research questions that there is no one or two crucial questions that should be asked. That said, this is one crucial thing you should do in all your research sessions: when something a participant says (or does) that seems interesting or is somewhat unclear, say, "Tell me more." UX research is about finding the "why." You might ask a participant, "On a scale from 1 to 7, how easy or difficult was it to find the information you wanted? (Where 1 is very difficult and 7 is very easy.)" but if you don't ask them "why" you're only getting half the answer.

For me, the “why” is what drew me to research and I love getting to learn more about people's understanding, expectations, habits, thoughts, actions, interpretations, frustrations, ideas, etc. Knowing this enables informed design decisions, resulting in products and services that better serve their users.

In addition to understanding the "why," research should uncover those things that end users experience or notice that the designer or researcher didn't anticipate. The designer isn't always going to be a power-user of a product or they may not use it the same way as others. (This is where observation and empathy are crucial.) Research helps to guide initial designs and should be used again and again in an iterative process to point out issues and opportunities which can help create the best products possible.


During research sessions there are often many moving parts, lots to track, and key questions to cover so, in midst of balancing all that, it never hurts to have friendly reminder of staying curious. In our remote research labs (labs dedicated to sessions with remote participants) I’ve placed this inspirational “artwork” to help keep “Tell me more…” top of mind.

A friendly reminder…

A friendly reminder…

NO UX LAB? NO PROBLEM!

-or-

Overcoming common obstacles preventing UX Research

In the last few years I’ve had several people come to me seeking advice about setting up easy-to-use, inexpensive, and effective usability testing kits. This includes an Amazon Research Team, a Facebook Research Manager, and a Principal User Experience Designer at REI. Why are these and others reaching out to me? Because they understand that research is vital to the success of their products and they don’t want technology, time, or money to prevent them from doing research.

These requests got me thinking about some common obstacles that prevent UX research from happening and what can be done to overcome them. Guided by the wisdom of one of the UX forefathers, Steve Krug, other seasoned UXers, and my experiences in UX tech and research I’d like to share three truths to show that you can overcome these obstacles and ensure UX research is not overlooked or dismissed.

You can easily create your own inexpensive and effective “UX Lab.”

Obstacle #1: Equipment is expensive and/or complicated.

During my time at Blink assisting with the technology side of UX research I was tasked with supporting numerous projects with needs beyond simple usability testing of a website. ‘Wizard of Oz’ testing of a smart kitchen appliance, testing of a app connected to a newly-designed gas-pump interface, testing the in-run experience of a redesigned running app, testing several iterations of components connected to the U by Moen smart shower, and a two-week diary study of a voice-activated speaker in participants’ homes are just a few of the more unique setups I’ve designed creative solutions for. While these projects have unique technical challenges, the majority of projects I’ve supported or led as a UX researcher are much simpler. The technical essentials are usually capturing a computer or phone screen, the participant’s face, and of course, audio from both the participant and the moderator.

I’ve utilized and I recommend two techniques for effectively capturing research sessions. With a few pieces of equipment and some training anyone can put these to use.

Technique 1 - Utilize Web Conferencing Software

Web conferencing software options are abundant and many of them are free or inexpensive. The tool that I recommend and have had great success with over the years is Zoom.

Utilizing Zoom to run remote moderated usability testing.

Utilizing Zoom to run remote moderated usability testing.

Zoom is designed to effectively capture whatever is needed during your research session. Zoom will automatically capture the participant’s face and audio and give you options for sharing either the participant's desktop or even mobile device. Additionally, the moderator is able to share their screen and give the participant control of the mouse. This is very effective when testing an HTML prototype or new designs you don’t want participants to have access to after the research session. Zoom also has the ability to automatically or manually record the meeting to your computer or in the cloud so, at the conclusion of your session, you have a PiP-configured MP4 file ready to be shared out or cut it highlight reels.

Getting stakeholders to watch sessions couldn’t be easier; just send them the Zoom meeting link and they can view from anywhere. The only downside is that there is the potential for your participant to feel like 10 people are watching them, especially if people are coming in and out of the meeting during the session. One way to limit this is to make an easy ‘Zoom Lab + Observation Room.’ Just do the following: the participant uses one computer in the 'lab' and one observer joins the meeting (muted w/ no video) and connects their computer to a large monitor, giving you a simple but effective 'observation room.'

Zoom’s many features, including all their screen sharing options make it a great, off-the-shelf research tool and the free version may be all you need.

Technique 2 - Stream Your Sessions to YouTube

If you’re feeling a bit more tech savvy and want more video/capture options then utilizing the free video production software OBS in conjunction with YouTube is another great way to capture and share research sessions.

Field kit for usability testing of a mobile app. Photo by Greg Hansen.

This technique takes advantage of free software as well as a free and very accessible location for live viewing and storing sessions. OBS is a free and open-source software designed for video recording and live streaming. For UX Research only the most basic features are needed but if you do have a study that has unique requirements it should be able to accommodate them.

OBS has designed live streaming to YouTube built right in and there are numerous resources out there to learn how to properly set everything up. Viewing sessions on YouTube is easy and secure; just ensure your live streams are 'Unlisted' so only those with the proper link have access, not matter where they are. This eliminates the 'many meeting participants' problem discussed earlier. The downsides to this technique are that there is more setup between sessions, some basic knowledge of setting things up in OBS (but I don't believe it's too complicated), and an investment in dedicated equipment.

For more specifics regarding building out your own adaptable and capable UX lab see my recommendations list.

You can find the ‘right’ participants provided you take an iterative approach.

Obstacle #2: Participants are hard to find and/or expensive.

In his book, Don’t Make Me Think, Steve Krug lays out several true things he knows about testing:

  • “Testing one user is 100 percent better than testing none.”

  • “Testing one user early in the project is better than testing 50 near the end.”

  • “The importance of recruiting representative users is overrated.”

  • “Testing is an iterative process.”

Some common perceptions around research or testing are that many participants are needed and that you have to find the exact right people to test with. This tends to make research into more of a high-stakes process when it doesn’t need to be. Krug’s philosophy is that testing should be early and often.

He believes that it’s good test with people similar to those who will use your product or service but more weight should be put on making testing an iterative process. The following diagram illustrates how testing twice with three participants in each test will identify more problems then doing just one test with eight participants. The key difference is that the problems identified during the first test are fixed before the second test.

Jakob Nielsen, of the Nielsen Norman Group, reiterates this philosophy in his article, “Why You Only Need to Test with 5 Users. Through years of research and some sophisticated math, Nielson and another researcher found that five participants will provide approximately 85% of the insights your study will uncover. After the fifth participant, “you are wasting your time by observing the same findings repeatedly but not learning much new.”

When you follow the iterative testing process Krug and Nielson lay out research doesn’t need to be a high-budget, high-stakes thing. Multiple tests with a smaller number of participants will help make research a routine part of your design and development process and identify more insights for the same or even less cost.

With the right tools and equipment (discussed earlier), research costs should be lower and studies can be done anywhere; in your ‘Zoom Lab + Observation Room,’ with remote participants, or out in the field.

The iterative approach evangelized by Krug and Nielson should also reduce the anxiety of finding the ‘right’ research participants. Because testing is happening more frequently there should be less pressure compared to finding the perfect fifteen participants for a ‘big, all-or-nothing research study.’

When it comes to finding people to participate there are numerous creative ways to find them as well as industry tools specifically designed to supply you with research participants. I’ve had great success with Dscout, UserTesting, and Respondent, which are just some of the tools you can utilize.

With some creativity and persistence you can build a culture of informed decision making.

Obstacle #3: There isn’t enough time for research.

This may be the hardest obstacle to overcome and the one that doesn’t have clearly defined solutions. Creating a culture where research is prioritized is hard work but it can be done. I was fortunate to work on a team at Microsoft where design and development wanted to make decisions informed by research before too much time was devoted to design efforts or before elements went live.

Each organization is going to differ on how much they prioritize research and the techniques used to create more buy-in will differ from situation to situation but there are some tried-and-true methods worth noting.

Get People to Watch Research Sessions

While there may be some truth in the obstacle of time, I don’t believe people are lacking interest. So it is crucial you give them an opportunity to watch research happening. I’ve heard of and seen several tactics practiced that are effective.

Invite all your stakeholders to the research sessions. Set up calendar invites, make posters, send out all-company emails, personally invite them; just get them in the room. Steve Krug believes, “try to get [them] to at least drop by; they’ll often become fascinated and stay longer than they planned.”

Set up live viewings in common areas. Alaska Airlines broadcasted research sessions in one of their cafes and found it a great way to evangelize research efforts happening on their website and mobile app.  

Make viewing interactive—this will keep observers engaged and can even help with analysis. Provide an observation worksheet or something like the Rainbow Spreadsheet to fill out or have them jot down observations related to key questions on sticky notes—perfect for affinity diagramming later. Their insights are valuable and can help with analysis and recommendations.

Again, with the right tools and equipment, your stakeholders and observers can watch live UX research via Zoom or even YouTube.

Once they’ve had the opportunity to observe participants experiencing frustration with an account verification code thought to be working perfectly or the joy of successfully using a voice command to activate an IOT device you’ll have them hooked.

Create Compelling/Interesting Research Deliverables

Research reports are often stereotyped as dry and boring but there’s no reason they have to be. The insights and findings uncovered in research can have profound impacts so do what you can to make engaging, interesting, and empathy-inducing deliverables.

I’m particularly fond of highlight reels. They are quickly viewed, can be easily shared, and can have a huge impact on those watching them. For a multi-part study focused on DIY soap makers I created this highlight reel of participants sharing how soap making has impacted their lives. The stories shared in this video helped to build excitement around the design and research work we were doing.  

When it comes to effectively communicating your work don’t be afraid to build on the successes of others. There are numerous ways to learn from others who’ve gone before us. I’m fortunate to work with tremendous researchers and designers who are eager to share the lessons for creating compelling work they’ve learned along the way. Seek out UXers around you for advice and wisdom. I’m found that most are willing and excited to help others in the UX industry. I’ve also found the podcast Mixed Methods and their Slack group to be very helpful. Additionally, Medium is full of articles like this one, with great tips on how to ensure your research isn’t boring and people will read it.

Try Something New

When I was completing my User-Centered Design Graduate Certificate I had two instructors, Rebecca Destello and Justin Marx, who had previously worked together at a start-up where they developed a unique solution to their organization’s lack of research buy-in. The need to “quickly instill a culture of user validation,” as well as other needs, prompted them to develop a strategy they called “Witness Wednesdays” or “usability sprints.”

Rebecca and Justin define usability sprints as “a series of rapid-fire user studies, emphasizing team collaboration and organizational buy-in.” These sprints would take five days to run and were often executed as a four-week series. The basic outline for one week looked like this:

Monday

  • Researchers and designers collaborate on the week’s session guide and prototype.

  • Observers are reminded of their upcoming obligation on Wednesday.

Tuesday

  • Researcher finalize the session guide and ensure the lab/observation room are ready.

  • Designers make any necessary tweaks to ensure the prototype matches the session guide.

Wednesday

  • Researcher conducts five, one-hour sessions.

  • Observers record issues, quotes, ideas, surprises, etc. on sticky notes.

  • Team debriefs and does quick affinity diagramming after each session.

Thursday

  • Researchers, designers, and stakeholders discuss the results and ideate on top-line issues.

  • Researchers generate a brief email report of the findings for the whole organization (because everyone on the email was in the observation room, an email is all that is needed!).

Friday

  • Discussion and ideation continues, if necessary

  • Given time, researcher begin next week’s session guide, and designers begin iterations to the prototype.

While this system had its challenges it also produced numerous great results for their organization. According to Rebecca and Justin, the biggest outcome of “Witness Wednesdays” was that it broke “the stereotype that ‘research take too long!’” Check out their talk at Convey UX 2018 for more about the unique system they developed.

⬡ ⬡ ⬡

Pivotal to overcoming these obstacles and deploying these creative tactics is ensuring that your research can be observed. Whether it’s during a collaborative live-viewing or in a compelling highlight video it all starts with an effective UX lab. Don’t have a UX lab? That’s not a problem! You can see there are simple, elegant, and effective solutions out there ready to adapt to your needs and the creative solutions you develop to promote UX research.

For more specifics regarding building out your own adaptable and capable UX lab see my recommendations list.


What are the ways you’ve overcome these obstacle? Please share your UX tech tips, strategies, and creative solutions by leaving a comment below. Thanks!

GRATITUDE: A CORE VALUE

For about five years I’ve been ending nearly all my work emails with one word: “thanks.” It’s a small thing and I’m not sure if my colleagues even notice it, but for me it’s a simple reminder to show my appreciation and gratitude for their part in the work we’re doing together.

GRATITUDE is another of the core values I strive to exemplify in the work that I do.

gratitude.png

The amount of support I received in my UX pursuit, in the form of wisdom, energy, time, money, encouragement, etc. from numerous people, is impossible to calculate. Without these people I would not be where I am today. I tried my best to thank them in this post.

And the support continues… Successful research projects don’t happen because of just one researcher; it’s thanks to the work of research participants, designers, product managers, recruiting partners, technical support, project managers, other researchers, and many other stakeholders.

I will always strive to work hard as I share my thanks and appreciation for all those involved.

EMBEDDED AT MICROSOFT

If you do a quick search on in-house vs. consultancy UX work you’ll find a slew of articles by UXers like this, this, or this sharing their thoughts and observations. And when I think back on my classes at UW there were many conversations discussing the merits and/or hazards of both types of work.

From what I’ve read and discussed at school and with other UXers, here are three brief considerations regarding working in both environments.

Variety vs. Singularity. At a consultancy you have the opportunity to work on a wide variety of projects while in-house you will most likely be dedicated to one specific product. In working on that one product you will get to see the impact of your work as you watch the product grow, develop, and hopefully come to fruition. In contrast, at a consultancy your work may or may not be implemented and you have very little control over this.

Slow Grid vs. Fast-Paced. In-house the work or progress made might be slow or drawn out due to numerous factors including business needs, legacy products, size of the company, etc. This, then can allow you the opportunity to make sure it’s designed and built based on informed research. At a consultancy the work can be fast-paced and you may feel the pressure of deadlines defined by the work contracted. Again, this can give you variety in your work – the opportunity to focus on one specific area of a product and then do something completely different on your next project.

Work-Life Balance. This one really depends on the company and the individual (more on this in a moment). I’ve read that in-house understands this balance better but I’ve also heard from others that they gained much better work-life balance after switching to consultancy work. At the same time I’ve heard from people who’ve left work-focused consultancies to go in-house where they found a much better balance.

A common qualifier that is a part of all these conversations and articles is that these statements are contingent on the organization and the person doing the work. Each are unique so in the end the choice to work in one environment over another ultimately comes down to the person and the place they’ll work.

embedded2.jpg

My UX experience up to this point has been entirely at a consultancy but I’ve just started on a two-month project where I’ll be embedded as a UX Researcher on a team at Microsoft. I’ll be developing and executing on a variety of research initiatives. I’m extremely excited to have the opportunity to work in-house for a few months; to experience the other side of the coin. This will be a tremendous opportunity for me to learn and grow as a UX Researcher and to also see what working at a large company, on a small team, focused on a few products is like.

⬡ ⬡ ⬡

What has been your experience working in-house or at a consultancy? Please share your thoughts by leaving a comment below. Thanks!

THE DO'S AND DON’TS OF DIARY STUDIES

For the past seven months I was part of the research team on a large-scale project that incorporated two research methods repeated over the course of seven waves of participants. The first method was a hybrid of contextual inquiries and usability testing where researchers went to participants’ homes and watched them interact with hardware and software for a new in-home device. The second method, and the subject I’m discussing today, was a digital diary study, of which I was the lead moderator.

Diary studies will vary from research project to research project but in the context of our work participants were given devices to use in their home and each day, over the course of about a week, they would complete activities with the device and answer questions using an online questionnaire or digital diary tool. The diary portion helped us collect valuable quantitative and qualitative feedback, enabled us to interact with participants in different geographical areas, and helped us to discover how behaviors and attitudes changed over time.

Repeating this over seven months with seven waves of participants allowed us to also iterate on how we administered the diary portion (as well as the in-homes sessions) of this study. That iteration has produced some key “Do’s and Don’ts” of diary studies. Use these to help ensure your research is the most effective it can be.

do01.png

Use the right tool - Try to pick a tool that matches your users’ tech proficiency. Revelation and Dscout are robust enterprise tools which require participants to create accounts but a free and familiar-to-most tool like Google Forms or Typeform may be all that you need.

Schedule a 30-minute on-boarding meeting with participants - This give you a chance to meet your participants face-to-face, build rapport, and assist with any technical setup necessary. This should also create more buy-in from your participants. Use the great online meeting tool Zoom to connect with your participants and show them how to use your selected diary tool.

BONUS: During this meeting you can also convey the importance of the diary study they are participating in. Let them know that their feedback and insights will be used to improve the product/service/experience you’re studying. This can also help with participant fatigue (more on that later).

Set proper expectations - Let participants know what will be required of them and how much time they should commit to completing the work. Clearly defining the activities and diary schedule is crucial for two reasons. This will aid your participants in completing the activities in the timeframe you’ve designed and should result in more accurate and thoughtful feedback.

Ask a mix of quantitative and qualitative questions - With a mix you will uncover the “why” behind the hard numbers. Because your ultimate aim is to find the answers to your research objectives it’s important to get a mix of data points. Your quantitative data will help you understand success/failure rates and satisfaction ratings. Your qualitative data will help you better understand what is causing those success/failure rates and satisfaction ratings.

Ask questions that get at the impact value versus just satisfaction rating - This will ensure you are uncovering the impact of the product/service/experience you’re researching and help provide actionable findings. Using scales that show the ultimate impact on participants will provide more valuable insights. Consider scales similar to the following:

1 = I had a significant problem and was unable to complete the setup successfully
2
3 = I had a few problems but was still able to complete the setup successfully
4
5 = I had no problems and was able to complete the setup successfully

OR

1 = The instructions were not clear, and I was unable to continue
2
3 = The instructions were not clear, but I was still able to continue
4
5 = The instructions were clear, and I was able to continue

Provide regular updates to clients or stakeholders regarding the progress of participants - This gives everyone a sense of how your participants are advancing through the study. Additionally, by including some initial findings in your progress report, clients and stakeholders can feel they are a part of the research process as well.

Create a spreadsheet to collect all your quant and qual data - It makes for easier analysis. As data comes in, quickly scan it and input it into your data results spreadsheet. This will help with analysis because everything is in one place and you’ve already reviewed it once.

Our data results spreadsheet from the final wave of testing.

Our data results spreadsheet from the final wave of testing.

don't02.png

Assume your participants will dedicate as much time to the study as you think or hope they will - Despite your best efforts to clearly communicate expectations things will come up and participants may show signs they’re giving less effort (simple one-word answers, skipping questions, lagging behind, etc.). Reach out to these participants and see if they can expand their thoughts or encourage them to complete late activities when they can.

Ask too many questions - Participant fatigue is real. As your participants progress through your diary study help ensure they continue to devote quality time and maximum effort by asking the appropriate number of questions. Even consider reducing the number of questions per activity as participants get closer to the end of the study.

Devote time and energy to areas outside the original scope - Keep your study on track and more manageable by concentrating on the pertinent questions to help with your research goals. It can be tempting to add questions that help inform other research questions you may have but stay focused on your original questions. You don’t want your data set to grow too large and don’t forget about participant fatigue!

Be afraid to ask clarifying questions - Ask follow up questions to better understand what they really meant. Participant’s answers may be vague or unclear and may not help you understand the “why” you’re after. Respectfully ask participants to clarify or expand their thoughts on the answer they’ve provided - most are happy to oblige.

Do it all alone - Depending on the number of participants and the number of questions, there can be a lot of data points so don’t be afraid to get support from others. Divide and conquer or ask a colleague to focus on a specific aspect of your study. Google Docs’ ability to do real-time collaboration helps tremendously here.

Overcomplicate the implementation of your activities/questions - Keep the schedule simple so participants can focus on completing the activities and providing quality answers. It is very likely that your participants will have different schedules so give them the freedom to complete activities when it best suits them. We had the most success when directing participants to complete the day’s activities when it fit into their schedule.


These ‘Do’s and Don’ts’ came from our desire to continually improve our process. Hopefully you can utilize them in your next diary study. And, as every study is different, you’ll likely discover other ways to run more effective and insightful research.

⬡ ⬡ ⬡

Do you have experience running diary studies? Please share your ‘Do’s and Don’ts’ by leaving a comment below. Thanks!

TIM BRIDGHAM, UX RESEARCHER

Roughly three years ago I launched this site and began my pursuit of a career in User Experience. During this time I spoke with numerous friends, colleagues, classmates, teachers, and acquaintances who all shared their time and advice in support of my goal. I shifted my focus on becoming a UX Researcher instead of a UX Designer. At my job, I was able to contribute my skills to more client-focused projects, learning a great deal along the way. And with each project my involvement focused more and more on UX research to where I was moderating usability sessions and collaborating on reporting. During this time I completed the User-Centered Design Certificate from the University of Washington learning from amazing UX professionals and working with driven and talented classmates to do some fun and engaging work. And throughout this entire journey, I had the incredible support from my wife who has lovingly encouraged me and has made numerous sacrifices to help me achieve my goal.

Today I officially became a UX Researcher at Blink UX and I am incredibly excited for what’s to come.

Geoff Harrison, Head of UX Services & Partner, announcing my new title. Cheers!

So thank you to Beth and Karen, Kelly, Kristina, Mark, Tom, John, Geoff, Brigitt, Randy, Skip, Layne, Jonathan, Jake, Louise, Nasahn, Sarah, Julie, Stephanie, Allan, Nick L, Piper, Roxane, Siri, Byron, Greg, Ankitha, Tristan, Darshana, Katie, Jill, Leili, Laura, Rebecca, Justin, Telle, Jenna, Ellis, Hayley, Nick W, Colin, Chris, Caitlin, Lu, Mom & Dad. To all of you, and the others - too numerous to mention, thank you for all that you have done to get me here. Without you this would not have been possible.

And my pursuit is not over. I’ll be striving to learn more, to continue to grow, and to become a better UX Researcher with each project I take on. In addition to continuing to share my journey, I’d also love to share others’ unique UX stories - all with the hope of helping the next person on their pursuit of a career in User Experience.

EDUCATION: USER-CENTERED DESIGN GRADUATE CERTIFICATE PROGRAM

I have benefited tremendously from working with amazing people at Blink UX; they bring passion to their work, exude empathy, and deliver creative and excellent work to our clients. As I’ve mentioned in the past, they have inspired me to grow and pursue a career in UX research. 

During my time at Blink UX, I have assisted on numerous projects including several prototype usability studies, field research projects, a data analysis and survey study, a microenthnographic in-home study, a retail intercepts study, and most recently, an application usability study and an unboxing + mobile app usability study (where I moderated seven sessions, created video clips, and assisted with the initial and final reports for both). I have even constructed a large-scale physical prototype to aid with testing. Also, I have been frequently sought out to create video clips in support of final deliverables. 

The large-scale prototype I constructed to support research. Read more about the benefits of recreating physical products in the lab  here . 

The large-scale prototype I constructed to support research. Read more about the benefits of recreating physical products in the lab here

These experiences have all been opportunities that have come organically to me over the past few years. On a few occasions I was involved from start to finish but most often I was brought in to support a specific piece of a project. Because of this I have obtained many ‘dots’ of knowledge spanning user experience work so I have chosen to connect those dots through formal education in the University of Washington’s User-Centered Design Certificate program.

In this program I will devote time to learning and going through the entire user-centered design process as well as focusing in specifically on usability testing. I view this UCD certificate program as another opportunity to surround myself with other successful UX leaders, be they students, faculty, or guest lecturers. I am confident the UCD certificate will give me the academic grounding, in conjunction with all my experiences and passions, to excel in a UX research career.

*Photo for by Mark Gsellman.