I recently sat down with Brian Burke, Director of eBay Seller Standards & Feedback, regarding some targeted tests being planned for the eBay Marketplace.
It has been a topic of great interest on the blog in the past so I thought a transcript of our conversation would be good for all involved. In a nutshell, a recent test was undertaken in which participating sellers receive feedback from buyers stating how likely they are to recommend the seller.
Here is a screen grab of that email test:
Over the coming weeks, a new wrinkle will be added to the test in which one of three Feedback “flows” will be presented to buyers in order for them to provide detailed feedback to sellers.
Those 3 different Feedback flows are as follows (click for full screen):
Lots to cover… and I’m sure this will generate quite a discussion. So here is a transcript of my conversation with Brian… I’ve also included his responses to my “5 Minutes with…” at the foot of this post.
Cheers!
RBH
Can you start off by giving us a brief overview of the Feedback tests?
As you can imagine, we regularly receive input from our members about our Feedback system. We gather input on potential changes to the Feedback system in a variety of ways, including focus groups, online surveys, and site tests like the one we are about to conduct. Each of these methods provides different types of input that help us better understand how potential changes will impact the system. This test will present three (possibly four) different new leave Feedback flows to buyers. It will only affect a small number of sellers, about 600, and only a small percentage of their listings, up to 2%.
The selected sellers are larger sellers with large volume so the 2% will not impact their overall Feedback or DSR scores, but just in case, buyers will be given the opportunity to leave conventional Feedback and DSRs after being presented with the new flow.
When do the tests begin? How long do we anticipate them to run?
The test will begin next week and run for 4-6 weeks.
Who is being targeted for these tests?
The selected sellers are sellers who are participating in an existing Net Promoter Score (NPS) test – where we ask their buyers an email asking how likely they are to recommend the seller.
Can you define Transactional NPS?
Net Promoter Score or NPS is an industry standard tool for measuring customer loyalty. Net Promoter is based on the question” How likely are you to recommend to a friend or a colleague?” In fact, we (eBay) and hundreds of other companies – large and small — use it to measure our customer’s loyalty and build strategies to improve it.
Transactional NPS is applying this customer loyalty metric after each transaction, much like we do with Feedback today.
What are we hoping to learn through these tests?
The goal of the test is to determine if we will receive similar buyer Feedback asking the NPS question via the leave Feedback flow vs. our current pilot of sending an email to buyers. NPS provides sellers with much richer information about the transaction and the sellers who’ve participated in the pilot have pleased with the added information.
How will Transactional NPS replace or complement the existing DSR format?
First, no changes are pending or planned for this year, this is a test to gather information. If successful we would likely expand the test to validate the information against a larger set of sellers or even in a country.
There are two elements of the existing FB system, conventional Feedback and DSRs. The system we are testing would apply to both, so we are looking at changes to both aspects of the system.
Will DSRs be changing? Going away all together?
DSRs are an important way for buyers to provide input on a seller’s performance. We are evaluating altering how we ask buyers to rate sellers, moving to more objective questions and when we have information about an aspect of the transaction, such as delivery confirmation within the time specified by the seller, we don’t need the buyer to rate the seller.
I’ll take that as a yes. Is the intent that the new feedback flow will go beyond indepth field research for businesses?
The goal is to provide sellers with more detailed, actionable information, while removing some of the emotion that drives inappropriate behaviors from the existing system.
What kind of language are you planning on instituting in these tests to ensure that buyers still understand the need to leave traditional DSR feedback IN ADDITION to the new feedback flow?
All buyers who leave the “test” Feedback will be presented a confirmation page. On that page will be a link to the conventional Feedback system so they can leave conventional Feedback for the seller if they wish.
Why should sellers be pleased to hear about this test?
We’ve heard their concerns about the accuracy of the current system and are looking at ways to obtain more accurate Feedback from buyers.
This should help our best performing sellers.
Testing should also signal to the sellers that we don’t want to make changes without understanding the impact to the business first. Changes to the Feedback system can be very disruptive since sellers rely on this data for evaluating performance and we use it for determining seller benefits. It’s not always possible to test every change, but whenever we can we will.
We launched eBay Ink 13 months ago and in that time, the changes to feedback – specifically the DSR feedback system – have created the most comments (predominantly negative) to the blog. Is this step the first in many that see us “fixing” the feedback system in the eyes of our sellers?
All systems have weaknesses, the eBay Feedback system is no exception. Over the years we have looked to enhance the system, and this is part of that evolution.
Enabling buyers to leave more accurate Feedback and measuring sellers against actual performance when we have the data will enhance the overall marketplace. Feedback and DSRs are much maligned, but they actually provide sellers and eBay a wealth of information about seller performance that we can’t get elsewhere. Every seller should want to know what their buyer thinks about the transactional experience so that the seller can improve performance or address the buyer’s issue.
Anything you’d like to add in closing?
Just to emphasize that this is a test. Based on what we learn we may make tweaks and further test. In the meantime, sellers should leverage DSR reports, especially the new reports which allow them to see how many low (meaning a 1 or a 2) DSRs they receive.
Also, I would like to plug this reporting page we developed for sellers last year. Sellers should book mark it – it’s a one stop page for sellers to go to report Unpaid items, Feedback extortion and other buyer violations. http://pages.ebay.com/help/sell/report_problem.html
5 Minutes with… Brian Burke
Full Name: Brian Burke
Nickname: Brian
Official Title: Director, Seller Standards & Feedback
What You Really Do: Responsible for developing minimum selling standards and consequences for those who don’t meet them (only when necessary) and for developing enhancements to the Feedback system, including DSRs.
Your First Day at eBay: June 1, 1999 – wow, time flies!
Education/Schooling: undergrad – San Jose State University, grad school – University of Phoenix
Interests: My three kids, volunteer on my kids school board, college sports, especially football
Favorite Quote: You can’t escape the responsibility of tomorrow by evading it today – Abraham Lincoln
Most memorable eBay experience: Too many – here are two, Voices Three (my first Voices) and the entire first eBay Live in Anaheim
Most recent eBay purchase/sale: Bought the book the Wind & the Willows for my son, after taking him on Mr. Toad’s Wild Ride at Disneyland last month
Favorite Albums: Beatles – White Album, U2 – Sunday Bloody Sunday
Favorite Movies: Any Hitchcock movie, Saving Private Ryan, Finding Nemo
What You Wanted to Be Growing Up: An NBA basketball player
Fill in the blanks:
I should have slept in yesterday (it was Monday when he answered this).
I wish that I could play a musical instrument.
The first thing I do when I get to work is check email.