Results by Design: UX Insights for Business Leaders

Description: In this episode, host Michael Woo sits down with UX Research Director Scott Klein to explore how organizations can develop an effective research program, even with tight budgets and minimal resources. Research is often seen as a luxury, but with the right approach, it becomes a powerful tool for improving business outcomes, optimizing user experiences, and driving measurable success.

What You’ll Learn:

• Why research is essential, even for small teams or solo researchers
• How to leverage affordable tools and creative recruiting methods
• The key differences between internal research teams and external agencies
• Strategies to align research with business goals to maximize impact
• How to showcase ROI and gain stakeholder buy-in for continued investment

Interview Participants:

• Michael Woo, Head of Design @ UpTop
• Scott Klein, UX Research Director

Transcript

Intro:
Welcome to Results by Design UX Insights for Business leaders, the podcast that dives deep into the world of UX design, strategy and insights. Tune in, take action, and design your way to success.

Michael Woo:
Hi, I’m Michael, and I’m your host today for the Results by Design podcast. Today we’re diving into how organizations can build an impactful research program even with limited resources. I’m here with a good friend and someone I’ve been fortunate to work with back in our time at Big Fish Games where he led the company’s UX research efforts and I was one of several UX designers sitting in the lab behind the two-way mirror. Watching him interact with customers, I observed firsthand how to moderate the ideal interview and usability session and the type of energy and empathy that one should bring to the table. I’m really excited to have him here today, Scott Klein.

Scott Klein:
Hey, Mike. Great to be chatting with you again today. It’s been a while since our big Fish Games days, but I really, it’s really exciting to get to talk with you again, and it was great to be a part of that evolution of big fish games with you, and I’m glad that you got to learn about research and stuff. I love this field and I’m excited to reminisce with you and talk about research.

Michael Woo:
Yeah, absolutely. Those were great times at Big Fish and especially a great group of folks to work with there, wasn’t it? Man. Before we jump into this topic, can you share a little about your background and how you became passionate about research and its application in ux?

Scott Klein:
Yeah, yeah, yeah, I’d love to. I often say I didn’t start my career in this. I studied management information systems in college, didn’t really know much about sociology and psychology, but the first job out of school was at a usability company. I was living in Texas at the time, and so it was a company in Dallas called Usability Sciences that they were basically hiring folks in. It was the.com era and this wasn’t really a field UX or it was called Usability, so they were just hiring folks in that they thought had good skills that could come in and do this. So they taught me, they kind of helped me understand what usability testing was and moderation and interviews and stuff, and I really kind of fell in love with it. Then I had always gotten in trouble at school for talking too much, so I was like, well, this seems like a field, or I can leverage those skills to talk with folks as well as understand observations and stuff, and I’m always just curious why do people do what they do?

So that love kind of fostered there. After that, I just kind of moved around to different field or different product spaces if you would. I went into more consulting after that job, kind of an interaction agency, working in the telecom community more or less, and then I moved into mash.com of all places that, unlike dating was a thing early in the two thousands, and then I went to hotels.com. Then I went into a restaurant business for a while, started understanding advertising, research, things like that. So just over time just really started saying, okay, I can take this field of social sciences and usability and stuff and apply it to any product space. I landed in gaming shortly after, I guess, I dunno, five or 10 years into my career, and went into there to do website usability for them. They had a web portal where they were just distributing PC games and then the founder was like, Hey, do you think you can apply this to games research?

And I said, sure, why not? Let’s figure out how to do that. And that kind of just evolved into that, and so it expanded out. It went into more into games research, market research, marketing research and all the above. So just throughout that time, my love for UX has just grown more and more because there’s not a product space that can’t take that application for, and there’s not a product space that you can’t take research and make it better, make that product space better because of it, and then I get to apply my business skills to that. Overall as I moved into leadership and stuff, it’s thinking about, okay, how do we do, we solve business problems now leveraging research and design and stuff like that. So it’s just really been something I’ve been able to stick with and go throughout my career and do it in different product spaces. It’s been a lot of fun.

Michael Woo:
That’s awesome, man. I mean, you’re absolutely right. We all take some crazy turns in terms of how we got into the field, and ultimately though, I think it’s an absolutely exciting field to be in, and I know you’re enjoying it as much as I am, so that’s great to hear.

Scott Klein:
Yeah, there’s not, a product that can’t use an improvement. We spend time on them. We all have our own. We go into and we just like, oh my gosh, this is horrible. As a usability person and as a designer, I’m sure over time you sit at a product and you’re just like, oh my gosh, why aren’t they doing this? Why aren’t they doing that? This could be that much better. It’s hard for me to look at anything now without a usability lens over time, but it’s what it’s, that’s part of what you were end up in this space.

Michael Woo:
Yeah. There’s no shorter opportunity out there, is there? No, not at all. Not at all. So in our conversation today, we’re going to cover practical steps to create a research program on a tight budget, including the tools, the strategies and tips for getting buy-in from stakeholders, and let’s just start with the big picture, Scott. It’s 2025. User research and user experience design has been in the mainstream conversation of what can make or break successful businesses for more than a decade. Now you have a good number of companies that, like you said, they understand their products are not perfect and are always striving to improve them. These businesses either have an established research and user experience design teams or they work with external partners. Can you explain why having a research program is essential even for organizations with limited resources?

Scott Klein:
Yeah, for sure. I mean, having a research team or even a single researcher is typically a win for parts of a business. First, they’re typically the advocate for the user. The researchers is the advocate for the user in all situations, and it really just helps the business make better business decisions when they’re grounded in user needs and focused on delivering the ideal experience. Right. Second way more research will be conducted, which will lead to all folks within the company having a better understanding of their user. These two alone will lead to improved business metrics. UXR will have a higher ROI because of that, I always like to say researchers or bridge builders, meaning they have information that is valuable to everyone in the company, user needs and expectations so far. They often talk to many different disciplines within the company, providing them the same information about the user and us building alignment and when alignment is built around those needs and expectations, better experiences are built. I mean, lastly, researchers are awesome and we’ll bring joy to your company, so hire them. I mean, ultimately we love talking to folks. We love sharing that information, so bring them in and let ’em be a part of that.

Michael Woo:
Yeah. It’s interesting that you called researchers Bridge Builders. I’ve noticed that most researchers embody this trait and they’re actually very well liked across teams because of their value and ability to unlock customer insights. Have you noticed the difference between what internal research teams versus external experts bring to the table? The idea that you can’t read the label from inside the jar.

Scott Klein:
The differences between outside agencies and internal is that I think outside agencies often have the ability to hire more researchers with varying experience, which can make them seem better or more experienced, but in the end, you can hire those same folks within your own internal teams. It’s just that their job is research, so they have the ability to get more and more. They’re running a business on that side of it. But when I build teams, my goals for each person I hire to bring new complimentary skills and experiences to the team, ultimately building an internal research agency, if you would, where very few researchers where research needs go unmet, thus eliminate the need for outside support. I started off my career in the agency life and currently in that world again, and it really is. They’re similar in that side of it, so you’re really just trying to find the right people overall.

Don’t get me wrong, though, agencies are great and should be leveraged when needed for special audiences. Say exploring a market where no expertise or skills in the language or culture as Us thing recently where we were trying to explore South Korea and Japan, I’m obviously not a native from there. I don’t speak the language and I don’t understand the cultures very well. I haven’t lived in those spaces. So going and finding agencies that can help you with stuff like that is definitely very useful, and you can still hire those folks in. If you’re building a business around that, then you should obviously hire the researchers this step internally to do that so that you can build the synergy between inside the company and what you’re learning from your users. But if not, agencies are fantastic for that sort of thing too. So it really kind comes down to a resource need based. So whether the agency can supply that or if you have the need that’s strong enough or to be able to hire those people in, then you should go that route.

Michael Woo:
Yeah, that totally makes sense. I’d probably build the same team or build a team the same way as you described. If you have the resources, great. If not, to your point, it can get costly to build that team. So I could see why some folks look externally for that expertise.

Scott Klein:
I think teams should leverage both whenever they can. What do you prefer?

Michael Woo:
Absolutely. Let’s talk about the impact that research has on business outcomes. What are some ways research has helped businesses make better decisions, especially when resources are constrained?

Scott Klein:
Yeah, so I love talking about this because research is typically treated as a cost center versus a can’t live without part the business. In my experience, it’s for a few reasons. Sometimes researchers forget to think about the business side of things or realize that research needs to lead business outcomes and not just the perfect execution of a cool methodology or a product team is overconfident in their understanding of the user and will dismiss it or dismiss and not do research with their, it doesn’t align with the business goals or we already know what to do. Or the business owner simply is unversed in the value of research and they don’t have experience with it or the experience they’ve done in the past has not been ideal. So ultimately research will save money and time, which equals money at every stage of product development. At the concept stage, if you’re really trying to understand what the game should look like, maybe a few ideas and you want to put that into place, you can really help.

Research can help narrow down those ideas. If the product is in development, you can maybe have a vertical slice of something or you have an early prototype of this, you can test that and understand where the usability issues are and how to kind of move that forward or post-launch. You can help understand what is and isn’t working in the product and help marketing understand the best way to sell it. For folks who want to grow a research team or just hire a single researcher, find a business owner who has a problem and talk to them about it and understand how you can help solve their problem, then go out and do that research, solve their problem and make them look like a star essentially. I think that’s always my goal. This is one of the surest ways that you can get what you want, but I would say be careful once you succeed, those flood gates will definitely open up wide and you may have more work to do than you know what to do with, but that’s a good problem to

Michael Woo:
Have. That is great advice for how to shine the spotlight on research. I know we’ve experienced the same at up top because we’ve helped so many business leaders solve their problems. They’ve looked like a star as this has led to repeat engagements for us, but your point about aligning both research and business goals is spot on. In order for research to really carve out of place in business, they need to be aligned with business at all times. What do you think are the biggest misconceptions about starting a research program, particularly on a smaller scale?

Scott Klein:
I think a few things come to mind. One is that I think oftentimes companies think that they need to go find that unicorn, that person who knows how to do all the research, both quant, fancy methods, project management, et cetera, et cetera, I think people feel they have only listed one shot. I need to get that amazing person or so I think one of the issues, I think the other ones is they think it takes a big budget to run research, which isn’t really the truth. You can hire an agency for pretty small or you can use a lot of tools out there that we could talk about or it is not that expensive, honestly, to do it, you’re going to save money in the long run or that research is too time consuming. I heard this time and time again, it’s going to slow down the process. We’re going to have to put milestones in for research, and it’s like, well, really not if research is done correctly. It flows into the overall process and you may have to add time for it, but that time is better spent now than it is later, as I’ve already talked about earlier. So I think those are some of the misconceptions about starting programs that kind of pop into my mind.

Michael Woo:
You’re doing some mythbusting here. That’s cool, man. So for organizations just starting out, what’s the first thing they should focus on when building a research program?

Scott Klein:
I kind of alluded a little bit in this. I think finding a researcher with experience, just conducting interviews and maybe some basic survey writing skills, those go a long way. Those are the foundations of research. In my experience, it’s not too difficult to find those people and they can conduct a lot of the early research that you might need, or at least we’ll be able to help you maybe find an agency as we talked about too, that can help you with this work. So I’d say most of the low hanging fruit in a company for a beginning team will be in usability testing or basic user surveys where they’re simple satisfaction survey or a product needs research. So you’re not doing a complex max diff survey. You’re just doing a simple set of few questions to gather some information, and then once that person is in and they’ve shown some value from that early work, people will come asking for more research as I said before, and then that’ll give you the direct path to growing more. It’s like, okay, well this person doesn’t have the skills, or maybe we need to increase our budget so that we can hire more agencies to help us in the interim. Adding more people has extra cost as well, but once you’ve shown that ROI, it’s usually not a hard sell up the way.

Michael Woo:
So what I’m hearing from you is that there’s higher value in qualitative insights from interviews and surveys and hence that that’s where you should start.

Scott Klein:
Yeah, for sure. I think oftentimes it’s just trying to gather the right information and gather some information. You don’t have to have thousands of responses from participants to understand usability, for instance. So fixing usability doesn’t take large sample sizes. So qualitative stuff works there and you can still do qualitative style work using a survey and get a few hundred responses, but it’s still be more qualitative in its nature, more open ends and stuff like that. So you’re not trying to do necessarily scientific research on that. You’re just trying to get more who are these users, what are their basic needs and stuff like that. So yeah, starting with that qualitative, I mean maybe I’m biased in that because that’s kind of my foundation, but there’s a need for both, but I think you can start with qualitative and get yourself moving forward quite a bit.

Michael Woo:
Yeah, that’s great. So is there a way companies can make the most of the resources they already have, whether it’s time people or tools?

Scott Klein:
Yeah, I think so mean especially with designers. I think designers often maybe what they went to school for especially have some research in their experience with that, but you also don’t have to have a researcher. I think you can get anybody to conduct a simple interview or a usability study by leveraging, there’s a lot of online guidance or help on how do you build a discussion guide, how do you conduct an interview? I think one of the things mostly to remember is you just don’t want to be leading in that, right? You’re not trying to put words into the user’s mouth. You want to ask a big, broad open question and sit back and listen. I think early in my career, you get uncomfortable early on as an interviewer with dead air and it’s like, well, no, dead air is fantastic. Dead air means no one’s saying anything, but people want to fill that space.

So if you as a moderator sit back and just be quiet, the user’s going to fill that and you’re going to get great little nuggets. They’re just going to be like, wait, I didn’t answer my question. I’m going to keep adding more onto it. So I think that’s a great way to just begin yourself in that you can also hire moderators out there. There are folks that are professional moderators and they will just do that portion of it for you. You as a team can put together the set of questions you want ’em to say, Hey, can you go moderate a few sessions for me? It’s pretty inexpensive. You’ll probably pay hourly for it. So if you have 10, 15, 5, 10 years or whatever, you might spend a few hundred bucks or a thousand dollars or something. It’s pretty cheap in the grand scheme of things.

Michael Woo:
So that’s great advice. With that said, what are the most critical types of research to focus on initially for maximum impact?

Scott Klein:
Yeah, usability is an easy one. There’s not a product out there as I kind of alluded to earlier that couldn’t use some help with that and with the alternatives that people have out there, you need to fix usability right away because it’s just easy for people to skip over. So you can improve retention or other conversion metrics like things like at a checkout funnel on the e-commerce or the first time user experience in a game. Those simple interviews are a great way to gather user needs and desires. A light survey can help you with this to allow us for a larger sample, which can help sell the results more for our skeptical cross-functional partners. I think a lot of times we talked about qualitative and people might question, oh, it’s only three people or it’s only 10 people, or whatever. It’s like, well, you can run a survey, get a little bit larger sample size on there and maybe help people say, look, we’ve got enough here to feel confident in those results, but there’s a lot of support around various types of data out there.

Michael Woo:
You mentioned first time user experience in a game and whether it’s a game or a product, I feel like that initial part of the experience often goes or gets missed by both design and product, and it’s one of the last things that is typically worked on, but can be one of the most important parts because what they say, if you only get one chance to make a first impression, users can be so fickle and I know I’m one of those users. If it ain’t right in the first few seconds, I just bail once something isn’t going the way I would expect it to go.

Scott Klein:
Yeah, you’re so right. What came comes to mind is a Google study. It was a few years back, but it was round gaming. Something around, and don’t quote me on it, but it’s pretty relevant in that within the first five seconds is when people make a decision on whether they’re going to stick with that game or spend money in that game, it doesn’t take very long. I think we all make quick judgements and the cost to change is really, really low. There’s thousands of games or other products that are out there that they could switch over to with low switching costs. It doesn’t take much to download another one and go in and try it. So I think the first time user experience is really, really vital. How do you get that person connected into what they’re after? If you truly understand why that person might be coming here, you build that experience around it, you’re going to get them into that flow a lot faster. So it’s super important to get that right. I think games over years have said, oh, well as free to play has come in, we’ll fix it down the road sort of thing. But as the game evolves and they build more late game stuff, that early experience is still bad and then there’s no cohesion. So anyway, I ramble on about that, but it is really, really important to get that first stuff and to always keep revisiting it because as the game evolves or the product evolves, maybe that first experience needs to evolve with it.

Michael Woo:
Yeah, a hundred percent. Let’s pivot really quick and talk about some tools and techniques. If you’re on a budget, what are some affordable or free tools if there are any you’d recommend for conducting research on a shoestring budget?

Scott Klein:
I don’t know if I know any free ones per se, but there’s definitely stuff out there. I mean, we mentioned chatGPT. That could be a great tool. Well, we didn’t mention chat. I’m in my head mentioning it, but it’s like we can talk about chatGPT or these other ones out there that there’s resources. I guess I talked about that earlier of resources online for moderating, but in my head it was thinking that anyway, there’s cost-effective ones I think to user interviews.com or when I was at Meadow, we used Dscout was a popular one that they used there. I don’t know what the costs are particularly on that, but I’ve used Optimal workshop in the past. It’s a great tool for collecting qualitative insights. We used that when I was at Big Fish and I do think there’s free versions of those or the low cost versions of those to be able to use.

Maybe it’s kind of a paired down feature set or whatever it is on those, but there’s lots of ’em out there to just get you started, whether it be for running surveys, I know most of the survey platforms or even Google Forms I think is a free one that you could create simple surveys with so you don’t have to pay money on those. So a little bit of searching, a little bit of chat GP work will help you kind of figure out where those free ones are, but start with those. There’s not a wrong path necessarily of a tools perspective, but use them to help you collect information. Even chatGPT can be used to analyze, I don’t want to say analyze, but to maybe help you think through the dataset we have if we have a lot of open ended questions, you want to see where patterns or arriving and stuff like that. So you can use tools for that sort of thing.

Michael Woo:
Yeah, I think with some of the tools that you named, they have remote conferencing as a part of the platform, which you may have to pay for, I can’t recall right now, but you can definitely use Zoom or Teams or Google Meet as the free versions of those, and really all you need is a participant and to be able to conduct interviews and stuff like that. So there’s definitely a way you can kind of go on a free budget, if you will, or a free trial for a lot of this stuff. So the barriers are pretty low in terms of getting started general.

Scott Klein:
Yeah, that’s a great call out for the interviews and stuff as well. You can do those remotely in a conversation like this with that. Yeah, great call out as well.

Michael Woo:
So for product or design teams that don’t have a researcher on staff, what methods can they use to conduct research themselves? We just kind of talked about that a little bit, but without hiring external agencies or consultants.

Scott Klein:
Yeah, I kind of talked about chatGPT a little bit, but I think there’s other more pointed resources out there such as that’ll help you understand interviewing techniques, usability testing and surveys. Nielsen Norman group’s been out forever. Jacob Nielsen is kind of, we call the godfather of usability, if you would, and him and Jared Spool, I’m probably dating myself, but those are the founders when I was starting, and there’s definitely a lot of folks that have come since then, but those two, Nielsen Norman Group has all kinds of online courses and newsletters and everything else that you can kind of browse through and get a lot of great information from. I mentioned chatGPT too. I leverage it these days for you can throw, if you have a bunch of open ends and you want to see kind of Starwood looking for somatic patterns in there, it’s good at doing that.

I wouldn’t necessarily use it for deep analysis yet. I think there’s a lot of bias inherent in it, but I think there’s ways to leverage it in a good way, such as I’ve used it too for what are ways to rewrite a survey question or what are some right ways to write a survey question or what are some methodologies or what is the industry saying about a certain methodology or if I don’t have an expertise in it, I can leverage it for that sort of thing. So there’s a lot of great online resources for that and frankly, I think there’s already great mentoring tools out there as well and other ways to get access to folks like myself or other ones that have been in the industry a long time. I love talking about research, hence why I’m chatting with you about this, but I’m always trying to help mentor and guide people down those paths. So I think there’s other human resources as well that you can leverage.

Michael Woo:
Those are all great resources and I’ve used all those myself. And as far as chatGPT goes, similarly, I’ve used it from a framework perspective. Research is so much details to remember that oftentimes it’s a little bit easier to just ask chat to remind you of some of those details and methodologies and it’s been so invaluable I think, and will continue to be a bigger part of our lives as researchers, as designers, so I’m happy it’s here. Personally, Scott, recruiting participants can be tough, especially with strong customer relationships and paid panels, pretty pricey. What are some low cost ways to find users for research?

Scott Klein:
Yeah, one of the first ones that comes to mind on this is what I call snowball recruiting, and I didn’t make that term. That’s what I’ve heard it as. It is just essentially who are your friends and family and ask them to, are they participants if you would, sorry, are they the right audience that you’re looking for? Now there’s inherent biases with that that you have to deal with, but you can leverage them that you find other friends and family and so you kind of get the snowball effect, right? Everyone’s asking a few of their friends and you can screen through some of that bias by creating a real screener that they would ask. So you can create an online Google form that kind of ask the questions that you’re looking for that would help qualify this audience member. So you can send that out.

It’s free. Everyone fills that out, and then you do your recruiting through that. You can also offer those folks a small incentive, maybe it’s like a $10 Amazon card or something like that to thank them for their time. You’re probably not doing an hour long interview. It might be a 30 minute interview that you’re trying to do, so it doesn’t take a whole lot of money. Oftentimes these people will do it for free. They think it’s fun, but it never hurts to thank somebody for their work. Some other options that come to mind are online forums or newsletters of a particular product that you’re after or a competitor set that you’re looking for. Those are great ways to forums some. My encouragement there is make sure you talk to a forum moderator though you can get, they sometimes look frowned upon if you’re in there trying to do research within their space without being authorized to definitely do the right thing there so you don’t get yourself in a bad spot.

But if you want to go really, really old school, you can put up flyers. I did this in my time, particularly if you’re looking for a certain audience and maybe they’re college students, there’s bullets and boards and flyer places that you can put those up and it has your name or an online URL or a QR code these days that they can scan and kind of fill out a form. So there’s different ways of doing that. A couple other ones that come to mind are one’s called Mechanical Turk, which is an Amazon built thing. I’ve only used it a little bit and I think it’s for help finding more around technical people. You can kind of hire people, so you put in kind of an ad in there and they specifically say, I’m willing to pay this much for time and these are the people I’m looking for and the folks can reply in.

And so you can find folks through that user interviews.com. I think you can find folks in a similar fashion with that. So again, there’s a lot of those resources to find the people that you’re after fairly inexpensively. I was always fortunate at big fish games inside the game. We could easily pop up a survey in there and get tens of thousands of responses really, really quickly. But I’ve had the challenges of finding really unique folks that are not rampantly available and that costs money. I’ve done research on doctors or lawyers and you have to pay those folks 500 bucks an hour to get them to even talk to you. So busy. So there are challenging methods, but there’s ways to get to those.

Michael Woo:
I learned. So new things I hadn’t heard a snowball recruiting before and I definitely have not heard of Mechanical Turk by Amazon, so I’ll have to look into those. But you definitely shared some great methods that I think give people choice in terms of how to get participants for the research study, so that’s great. With all the great tips you’ve shared so far, you and I know it’s all for N if we can’t get stakeholder buy-in, we’ve long heard how a user-centered approach boosts profits and strengthened brands yet I don’t know why, but teams are still struggling to justify research internally. How do you convince stakeholders to invest in research, especially with tight budgets these days?

Scott Klein:
I know bribery, maybe I go back to Amazon gift cards and we’re just paying people to do it. No, I mean in all seriousness, I think I have two typical methods I would call out on this, maybe my first one, and I kind of teach this, I mentioned mentoring earlier. I think this is my best one with researchers, is to talk to the highest level stakeholder you can and try to get in there and understand what their business problems are, what do they feel that they are trying to solve, and then see if you can solve it using research. If you feel you can, maybe you pitch that idea to the stakeholder and say, you know what? If you’ll foot the bill, I’ll go out and do this research for you. And then if you’ve gotten that buy-in, then you go out and you do it.

You do that great research and you solve their problem for them, and that’s a huge win. And when that happens, snowball effect happens there as well that they’re going to be coming to you and be like, Hey, I want more of this. How did you do that? And then you have friends and then those friends will find out about it and you’ll grow the business that way. I think another one is to kind of figure out what development costs are. So it’s an ROI sort of configuration research. You have to prove ROI for any team and a company has to have a value add, right? And research often doesn’t do a good job of selling that. So I think how I often pitch it is, okay, how much does it cost to develop? What are designers cost? What are engineers cost to build this product to a certain point, and what does it then cost to fix something down the road if it wasn’t done the way that the user wanted it and then go, okay, well that’s that cost.

Here’s what research would’ve cost to spend the time doing it before and that we would’ve gotten it right and not had to fix it later. Hopefully that’s a better ROI calculation there. So it’s just showing them the value of that research and the ROI that you’re going to get out of doing it versus having to take something down the road. So that to me is a way I’ve solved it a lot of times. I mean, there’s no guarantees in either one of these methods, but I’ve had more success using them than I have in others that I’ve tried overall.

Michael Woo:
Yeah, I love it. Two really great approaches. Question for the first method that you stated, should there be any concerns about bucking the org structure if you have to take your pitch a few levels up?

Scott Klein:
Yeah, I get what you’re saying. I think overall, we talked about researchers being bridge builders, and I think you should have built partnerships at all levels. So hopefully you’re not having to, hopefully those folks are coming along with you to go up that, but if you’re having to circumvent somebody because you’re not feeling like your partners at a lower level are doing that, I think that’s part of, you kind of have to deal with that, but I think if you can get the win higher up, that should trickle down and make everyone’s job easier. I guess the metaphor is sometimes you had to break a few eggs to make an omelet, right? It’s like you’re not always going to have that alignment, but I think it’s better. Your goal as a researcher is to focus on the user and if you feel like you’re doing the right thing for the user, going around somebody else should be the right thing overall. I don’t necessarily suggest doing it unless you just really haven’t found another way. But sometimes doing the work of getting leadership to buy in will often win you the points with those closer to your level because it takes the load off of them ultimately or helps them build clearer goals and objectives for themselves. Ultimately, you end goals to really help those folks be more successful as well. So that should trickle down overall,

Michael Woo:
This is great. I know a client or two right now that could actually benefit from the strategy. So definitely thank for sharing that. What’s the best way to showcase the ROI of research to gain continued support and potentially increase resources?

Scott Klein:
What I see too often is a project is pitched assumptions about the metrics that can be impacted are made, and then the people are disappointed when that impact wasn’t what they expected. So for instance, the stakeholders thinks we’re changing one thing and the researcher thinks we’re changing another, and then those don’t align. So what I often suggest is the first thing to do is to make sure you’re clear on what those metrics are. What are you trying to impact with the researcher you’re conducting? Are you trying to change retention rates? Are you trying to affect that first time user experience as day one? Retention with thinking in the gaming space? Are you trying to get conversion on that first, not pinch point, but first purchase point if you would in a product space or getting them through that. So understanding what those are and then making sure that you’re in alignment with the stakeholder on that.

So when you have those discussions of like, okay, we’re coming here today, you want to do research on this, this is what we think we can measure and what we think we can affect and here’s what we can’t. Then once you’ve done that research and implemented the designs from the research, you’ll be able to show the stakeholder the impact of those exact metrics that you agreed upon and minimizing that whole, but I thought you were helping improve this other metric discussion, which we hear too many times. And then I think once you’ve shown impact on those agreed upon metrics, you can spread the word through conversations with others essentially. So it’s like, Hey, we did X, Y, Z, and this is the impact we had. And you can do that through conversations. You can do that in business review meetings, newsletters, all hands meetings, et cetera.

I think just don’t be shy. Product managers and business execs aren’t shy when their work moves the needle, so researchers should need or to your own horn, you got to be an internal marketer to kind of do that. For instance, while I was at Big Fish, I helped with all hands meetings and made it a point to make sure that research work was showcased in some way. In each meeting, the game teams would often have a showcase, oh, look at this cool game we made and here’s all the stuff we did. It’s like, well, but research played a big role in that too, so let’s make sure we get a shout out into how you got to this point and how the game became what it was, because research was vital in that. So I think that those are things that research just naturally probably doesn’t do, and I think we’re empathetic people. We’re probably softer spoken in some ways. I don’t know, maybe I’m making a big generalization, but I do think we are in this field because of certain traits that we have, so may not be the biggest loudest folks out there, but I think by doing that, you’re growing a cohesive view of who the customer is overall and what research can do to help with that from a business perspective. And anytime you can do that, it helps the company overall.

Michael Woo:
Yeah, I think that’s a great tip and a good survival tactic, if you will. To your point, a lot of people don’t quite understand what research is, how people don’t understand what UX is, and so sometimes you do, to your point, you have to be your own evangelist or champion, and sometimes being humble, isn’t it a good thing in this situation, you have to really spread the positive results that you’re actually creating for the company.

Scott Klein:
Yeah,

Michael Woo:
Not feeling bad about it.

Scott Klein:
Yeah, exactly. Yeah, that’s the kicker. Don’t feel bad about it because it’s not a bad thing. You’re helping the company overall, and you’re helping research grow as well because of it.

Michael Woo:
Yeah, I love it. Once the basic program is in place, what are some tips for gradually scaling it as the organization grows?

Scott Klein:
This is the fun part, honestly, in the research team, it’s like, okay, we’ve gotten buy-in now and how do we kind of grow this? There’s a huge demand. I think when I’m scaling a team, I’m always evaluating the skill sets and the expertise I have on the team with the goal of each new person I add brings something new to the group. It can be a deep expertise on a particular method that would be valuable to the team. It could be do they have previous work experience that looks interesting that could really be valuable to the team, or is it related in some way or another? Or they have skills around being a mentor and teaching new skills, or they had great consulting skills where they can expand the research within the organization overall. So maybe they are a great marketing person and evangelist as well, and they have that sort of gift for gab.

It can go out there and really spread the word. So I’m always kind of looking at how do we implement, how does each person add to the overall greater thing? So as a team grows, the overall team structure will also need to evolve. For instance, a big fish. Our team started as a central hub and at that point we needed to grow out further. We were getting a lot of bit of out of sight, out of mind happening in the different studios, and so I embedded researchers out in, I hired senior folks in that had some of those consulting skills and could sit in those game teams and really champion research as they do it. So you might need that kind of growth to happen as well. It can happen in reverse. Sometimes researchers are hired and they’re just in a team, and then maybe everyone’s coming at and needing usability testing in a common way.

So you build a kind central hub. So there’s different kind of flows that you go into, but it’s kind of a fit for purpose that’s needed. And then I think lastly, as I said before, you need to continue to talk about and showcase the work that research is doing and the impact it’s having. Product team members can be very, I guess myopic if you would, and by pushing information to them, your work may be used by more teams than it was conducted for. So meaning you did it for one product game or one product and there’s similar products your company is developing. If they can learn about that research, they can leverage that without having to do new research. So again, being that voice and making sure you’re sharing all that information, essentially the squeaky wheel gets the greased, so be that squeaky wheel in a good way. You want to be loud and proud of the work that you did and then make that spread around overall.

Michael Woo:
Scott, as we wrap up or discussion today, what’s the one piece of advice you’d give to business leaders who’d like to start building a research program right now with minimal resources?

Scott Klein:
I think my theme here might be make sure research knows what the most important business problems are and focus on helping and have them focus on helping solve those problems. This is a sure way to quickly prove the value of the research, build good relationships and advocates who will support the team’s growth when those discussions are inevitably are had within the company of the years, like are meeting those ROI discussions. In my experience, engineers and product managers or key stakeholders, researchers should focus on building strong relationships with, I mean, Mike, I know you’re a designer, so I’m not discounting your value whatsoever, but design usually has a clear understanding of research, so it’s like I don’t have to work very hard on that. Whereas engineers and PMs often take a little bit more convincing because that’s not their day-to-day. They also came into the engineers in particular like, oh, I know how to build this.

I’m going to build this way I have in my mind. So helping them make sure that they’re getting the vision for the user is really important. Building those relationships with those kinds of folks and finding those key stakeholders is good. And then with that said, know your limits and know what you can safely execute for the leadership with the resources that you have available. A quick way to failure is trying to accomplish too much and validating the bias that research is expensive and low ROI as we talked about. So making sure you’re doing the right work at the right time and making sure it’s having the impact that you planned on it having is how you’re wanting those business leaders to see that value and continue to grow.

Michael Woo:
Scott, thank you for sharing your expertise and really making research feel more accessible to our listeners. It’s been a great conversation today, and I know our listeners are definitely walking away with some actionable insights.

That’s it for today’s episode. If you enjoyed today’s conversation, please hit the like and subscribe button. Join us next time as we explore more innovative approaches to enhance your products and services, optimize customer interactions, and ultimately drive success for your organization. Tune in, take action and design your way to success.

Outro:
Thanks for tuning in to Results by Design. If you liked this episode, be sure to share and subscribe to our YouTube channel. We are also playing on all your favorite audio streaming platforms, so stay connected and join us for the next one. Results by Design is brought to you by UpTop. Our mission is to equip business leaders like you with the knowledge and tools needed to leverage UX methods and strategies to achieve tangible business outcomes and create lasting value. Whether you’re a seasoned executive or just starting to explore the world of ux, results by Design is your go-to resource for unlocking the potential of user experience to achieve remarkable results. Tune in, take action, and design your way to success.