Context

I joined the Knowledge Base initiative at Etsy in August 2020 in order to help the team improve the quality of raw data that we collect from sellers who are listing items on Etsy — an experience that we internally refer to as “the listing process.” The first order of business for me and my PM was to conduct qualitative research on this complex experience with our users to understand where their biggest pain points were.

From September through December 2020, I worked with and managed an outside research agency to conduct usability interviews with over 60 Etsy sellers. We did an additional round of research with around 10 sellers who sold on other platforms (e.g. Amazon, eBay, Poshmark, etc).

The opportunity

Through our extensive research I realized that, even though sellers often told us that “the listing process was fine,” what we observed qualitatively as well as in our usage metrics told a different story. Our number one goal was to improve both the accuracy and coverage of key structured data because we believed this would ultimately better serve the buyer experiences, yet sellers almost never filled this data out. The reasons for this boiled down to the following:

Sellers cared most about their listings showing up in Etsy’s search — yet they had seen no evidence that adding more listing data would improve their search ranking.
Creating a listing from scratch took nearly 12 minutes for sellers — even though they said this was “fine,” we observed that they often skipped every single optional field in the listing form in order to try to save time. Often, these optional fields were fields that would improve our structured data.
Because of how long it took to create a listing from scratch, sellers often copied previous listings and then simply made a few edits from there. Because of this extremely common habit, sellers rarely bothered to fill out more data on different listings. Instead, they defaulted to the same data on every listing over and over again, even when the data may have been inaccurate or incomplete.

After several rounds of design exploration, I came to the realization that small tweaks to the user experience would not be enough to meaningfully shift seller behavior at scale, so I began to explore the potential of a more thorough redesign.

Etsy’s product development culture tends to be very risk-averse — “redesigns” are seen as expensive and had historically been deprioritized in favor of smaller tracks of work that could yield immediate results. As such, I set out to articulate a vision that would energize and motivate leadership and my team to invest in longer-term work. At the time, my team only had a few relatively junior contract engineers because the scope of this work was believed to be “small.” My goal was to articulate a vision that got leadership excited enough to then increase our engineering staffing.

We conducted 60+ usability interviews to understand behavior and sentiment around the existing experience.

The solution

In order to achieve of goal of creating buy-in and advocating for more engineering resources, in the first half of 2021, I…

Conducted 5+ rounds of user research to gain confidence in a vision
My biggest design hunches were around re-structuring the information architecture of the entire listing process, but there would likely be serious risk to doing this without first understanding if we were deviating too far from sellers’ existing mental models. I leveraged a variety of qualitative research methods (e.g. card sorting, interviews, provocations, etc) to arrive at a redesign that I felt successfully improved ease of use while also maximizing our potential to elicit better data entry.
Presented this vision to leadership (see deck below)
I focused on explaining why smaller design tweaks wouldn’t be enough, and I highlighted all the potential gains to be had if we could radically shift how sellers thought about the value and purpose of listing data entry. The crux of the vision was to incorporate structured data into search ranking, which would be a massive technical undertaking that had historically faced many failed attempts, so I knew I also had to facilitate discussion with leadership about how to attempt to do this again successfully.
Facilitated product and technical strategy discussions (see deck below, starting on slide 39)
Incorporating not-yet-existent data into search ranking was a bit of a chicken-and-egg problem. Without high coverage, any work to incorporate this data (i.e. via search indexing and as a feature in search ranking) was (at the time) seen as a waste of time by the Search team. But without this data being incorporated into Etsy’s search, sellers were unmotivated to bother filling anything out. Inferring data would be great, but much of the raw data we had from sellers was in the form of minimally structured freeform text, so accuracy was hit or miss, depending on what attribute we were focused on. I facilitated discussions with our team to align on a best path forward and to define principles by which we would operate as a team, especially in our relationship with the Search team.

The outcome

The feedback from user research was overwhelmingly positive and promising, and I was able to use this to get buy-in with both my immediate Product and Eng partners, as well as our counterparts in management.

From here, my PM, EM, and I more tactically outlined a detailed staffing proposal that also articulated business tradeoffs. Separately, I increased the visibility of this work internally and positioned it as innovative and radically different from other more typical projects at Etsy. As a result of our staffing proposal, we were able to get approval for headcount, and as a result of the exciting internal reputation or this work, 4 engineers from other teams asked to be transferred to our team, and 2 requested to do month-long rotations. This enabled us to begin development on the redesign almost immediately, which surpassed our initial staffing expectations!

As of August 2021, we are now about the launch the first MVP release of the redesign.