Yahoo! Local/Maps Competitive Benchmarking
The Story
While on the Yahoo! Local team, I helped establish a Bi-annual quantitative benchmarking study. The goal of this project was to benchmark key UX metrics (satisfaction, success/failure on key tasks, readability of content, etc.) between our key competitors (Google, Yelp, Microsoft, YellowPages.com). This benchmarking study was conducted online via a tool called Keynote.
Yahoo! had been outshining our competitors for years, until one day we were no longer at the top. I performed a factor and driver analysis to find which key metrics were most affecting this change, then conducted qualitative research observing their experience and understanding the “why” behind the numbers. Together the qualitative and quantitative data helped us understand why we were no longer at top illustrating 3 key areas that we needed to focus on to keep up with competitors.
Key Research Questions To Solve
How does the Yahoo! Local user experience compare against our key competitors?
Why is Yahoo!, who historically had been leading competitors, now falling behind?
Methods Used
Quantitative UX Competitive Benchmarking, Usability study.
Example of one of our metrics. Yahoo! Local is represented via the yellow line against 4 competitors. The quantitative data raised the red flag that our scores were slowly slipping over time against our competitors.
I collected 30 metrics and used factor and regression analysis to determine which scores and tasks were having the biggest impact.
In our qualitative research, I sought to find out why our metrics were slipping, focusing on the key metrics revealed in our regression analysis.
Here we learned why the efficiency metric was a key driver. How many products does it take to plan a dinner and a movie? With Yahoo! 3 (Yahoo! Maps, Local, and Movies). With Google, just one: Google Maps.
Why This Project Matters To Me
As a case study, this project has everything and will always be a shining star as far as highlighting my capabilities However, personally, this project grew me as a researcher like no other. For one it was the most extensive quantitative project I’ve been involved in and really flexed my statistical muscles harkening back to when I was once a statistics major in college. It also was the first time in my career that I really realized the power of mixing qualitative and quantitative data together. I learned how to use quantitative data to draw attention to a problem and then use qualitative data to explain the “why” and “how” to solve the problem. Finally, it was one of the first time where I was able to experience research impact at the highest level. I’ll never forget presenting the data to the senior executives of the search team, which became a rather simple story because due to the strength of the data triangulation, the data practically spoke for itself.
Want to learn more about this research and its impact? Let’s chat!