What metrics would you use for Airbnb’s Experiences feature?

Answer Structure

  1. Ask clarifying questions to confirm your assumptions
    (e.g., What does the product do? Who is the product for?)
  2. State or verify with the interviewer what the goal of the feature is
    (e.g., to increase conversion, revenue, profits, sales, engagement, retention, or other?).
  3. Use the user journey map to drive your analysis of what to measure.
    (e.g., Think of what to measure when the user opens the app or opens a feature and interacts with it.)
  4. At each stage, identify behaviours that can be quantified with a metric(s).
  5. Provide criteria to prioritize the metrics.
  6. Summarize your analysis.

Answer Example

clarify the function of feature

INTERVIEWEE: Okay, first I want to confirm that my understanding of Airbnb Experiences is correct. Is Experiences a new feature that lets travelers book a wide range of activities as part of their travel plans

INTERVIEWER: That’s right.

clarify the goal

INTERVIEWEE: And, what is the overall goal of making this feature?

INTERVIEWER: Well, I would like you to tell me what you think the goal is?

INTERVIEWEE: Experiences is a new offering, which aligns with Airbnb’s mission to make travelers feel like they “belong anywhere.” So the goal of this feature seems to be to diversify Airbnb’s revenue stream with a complementary offer that appeals to its base. This additional revenue can come directly from people booking activities, or indirectly from people prolonging their stays especially if they book multi-day experiences.

INTERVIEWER: Yes, I agree.

INTERVIEWEE: The success of the Experiences feature in creating additional revenue depends on how well it appeals to two parties: the travelers that book Experiences activities and the providers of the activities. So, I would separate metrics into two sets. One set will measure how well the feature appeals to the demand side, the travelers. And, the other set will measure how well the feature appeals to the supply side, the providers. I would like to start with the demand.

INTERVIEWER: That sounds good. Let’s start with the demand side, and if there is time, we will go through the supply side too.

clarify metrics - user journey map

INTERVIEWEE: Ok. On the demand side, a basic metric to measure success is the number of bookings of activities. But, besides this general metric, I would be interested in evaluating this feature on how well it drives the user towards that conversion. This is important to course correct anything about the feature that is not working well. So, I would like to create a journey map of the user, identify which behaviours indicate success or failure towards conversion, and define metrics to measure them numerically. I will then prioritize the metrics based on two attributes: how core they are to evaluating success and how actionable they are. By actionable, I mean, if the metric provides insights to improve something about the product. After prioritizing the metrics, I will wrap up with my recommendation.

INTERVIEWER: That sounds good. Please proceed.

INTERVIEWEE: Okay, I have tried the Experiences feature, but may I try it again before I start describing the user journey?

INTERVIEWER: Sure.

(The interviewee tries the Airbnb Experiences feature on her smartphone.)

INTERVIEWEE: Okay, here is how I would describe the user journey in the case of the website:

In this journey, the user goes through an awareness, engagement, and conversion stage. And, there are two stages for post-experience: retention and referral. At each stage, the user takes actions that contribute to the goal of increasing revenue through booking of activities. I will list those actions and define metrics that can measure that level of contribution.

Two high-level metrics that measure success are:

  • Monthly bookings of Experiences activities
  • Revenue from monthly bookings

Although these metrics are not actionable, they summarize key business results at a point in time.

clarify metrics at each stage

Now, I will go through the user journey. Could I use the whiteboard to work on my analysis?

INTERVIEWER: Sure.

(The interviewee draws the user journey map, and writes down metrics to measure through each journey stage.)

I would track all of these metrics on a monthly basis to compare month-to-month progress. Per month is a good time frame since any changes we make to the product would take at least a month to make a difference.

prioritize the metrics

So, I will prioritize the metrics based on these two attributes with the help of a chart.

Summarize your analysis

So in summary, the metrics I would prioritize that fall in quadrant I and II are #1, #2, #4, #8, #9, #12, #13, and #14.

As a product manager, I would track all of these metrics. However, if I were to present to the CEO, I would pick the ones that will give the CEO a bird’s-eye view of the business. These metrics are:

The # bookings of activities per month (#8), which can be translated into revenue using an average dollar value per booking; and, The total # of activity bookings over the total # of room bookings per month (#12). Would you like me to continue with my analysis on the supply side?

INTERVIEWER: No, thank you. I think this suffices.

Source : https://stellarpeers.com/what-metrics-would-you-use-for-airbnbs-experiences-feature/