Vanity Metrics—the “feel-good” metrics that neither reflect nor necessarily correlate well to the key drivers of a business—continue to be used widely to market and report on growth and success. Examples of vanity metrics include:
- app downloads or installs,
- messages sent over platform,
- registered users,
- page views,
- number of tweets ,
- number of events processed,
- number of followers
- daily, weekly or monthly active users
It takes infrastructure, energy and resources to be able to deliver billions of messages a day or process billions of events a day to makes lives of customers easy and to provide value. Marketers love amplifying the achievements by announcing growth numbers:
140 billion apps cumulatively downloaded from Apple’s App Store
Facebook averaged 1.18 billion daily active users in September 2016
Pyze is processing more than 1 Billion events daily
Google delivered over 3 billion app downloads driven by ads
Eric Ries of The Lean Startup is says vanity metrics are not actionable and entrepreneurs should invest energy in collecting and analyzing metrics that help them make decisions. Apple does not use “Cumulative downloads” for managing their business, but has used it to report growth of the app store. Neither do Facebook, Google, or Pyze use the metrics listed above to report growth for manage their business.
So, some metrics are great for chest-thumping, reporting and marketing, but not for hinging your business on.
The Vanity Metrics
Let’s look at two Vanity Metrics – App downloads or Installs; and Daily, Weekly & Monthly Active Users – and see how these and its derivations are of value to product managers, growth marketers and data scientists.
App downloads or installs
App downloads or installs do not reflect success and are not directly related to revenue and hence its labeled as a “vanity metric”.
High Acquisition costs
Installs can be easily engineered by paying for app downloads indirectly via social marketing, search engine marketing, app store search ads. Although a large number of installs is beneficial to any app publisher, if the cost of app user acquisition via expensive advertising campaigns is significantly higher than the average revenue per user (ARPU), then gaining more users could eventually lead to losses.
Paying for downloads
App downloads or installs can also be manipulated by paying for downloads directly via fiverr and numerous other services. It is important to note that app stores detect script-based installs that usually show up as downloaded from “desktop” versus device, and paying for downloads has zero positive affect on app store rankings.
App stores look at engagement and uninstalls for rankings. Most purchased downloads eventually have a negative affect on rankings.
Daily, Weekly & Monthly Active Users
Daily, Weekly & Monthly Active Users are used for tracking and reporting growth by most app companies. Many SaaS businesses charge based on monthly active users. But a case can be made why they are vanity metrics.
Daily Active Users, Weekly Active Users and Monthly Active Users are not directly actionable and cannot be used to drive change.
Miscalculated and inflated
- The active user counts are often miscalculated to inflate the numbers. For example,
- MAU is not an aggregation of a month’s DAUs and
- WAU is not an aggregation of a week’s DAUs.
Large false positives
- Even if a user of an app launches the app for a second or two, they are counted towards the active user counts.
- A user who has forgotten his or her password and does not bother to recover the password, is counted towards the active user counts.
- An app launched and closed immediately as a result of a click on a deep-, app- or universal-link is counted towards the active user counts.
DAU does not correlate with Revenue
Users, based on in-app purchases are in games typically grouped as:
- Minnows (i.e. users who spend the least amount),
- Dolphins (i.e. users who spend a moderate amount),
- Whales (users who spend the most) and
For a specific game that used in-app purchases, we found:
- 4% Minnows,
- 2% Dolphins,
- 6% Whales,
- the rest 88% Freeloaders.
Others have reported similar numbers: only 5.2% of users are spenders.
Clearly, a large number of DAUs does not mean large number of paying customers.
Derived Vanity Metrics
Any metric based on DAU vanity metric is also a vanity metric. Microsoft closed the acquisition of Linkedin this week. Last published LinkedIn numbers are 933 million in revenue and 450 Million registered users, implying an ARPU of $2.07
- Revenue per active user or Average revenue per daily active user (ARPUDAU), includes users who use the app for a second or cannot get past the login screen.
In the LinkedIn example above, the ARPUDAU was $8.3
- Stickiness Factor is a ratio of DAU and MAU and can be calculated incorrectly. See here.
Going Beyond the Vanity Metrics
Vanity metrics, Installs and DAU/MAU/WAU may not be directly actionable but we continue to use them for reporting and find value.
You need installs
An Install is an important step in the funnel. It is preceded by Acquisition and Awareness Marketing (Channels: Search Engine, Social and App Store Search Ads) and followed by Retention (Channels: Push Notifications, MMS and SMS) and Engagement Marketing (In-App Messages). Without an Installed user base, you have no users to retain and engage.
Stickiness vs. Stickiness Factor
Stickiness is a ratio of DAU and MAU and indicates the depth of engagement. The closer the ratio of DAU and MAU to 1, better the stickiness and it means that more of your users are returning to the site every day. See why 7 Day Stickiness Factor and 30 Day Stickiness Factor are much better metrics instead of DAU/MAU.
If you compare this week’s installs with last week’s by overlaying them on top of each other, you get an idea of how your installs are doing week over week.
A significantly large number of users drop off after install the app and using the app for the first time. App publishers should personalize the on-boarding process for users individually based on what you already know about the users and the users’ device. We have written about personalizing the on-boarding process for mobile apps and for web apps.
Activations is a much better alternative to Installs and it is actionable. It is an relevant growth metric. It is important to keep track of important milestones in the lifecycle and activations help app publishers do exactly that. Examples to track include:
- App is launched or resumed Nth time,
- App used for M Days in last N day period
- App used K Unique Days
- App usage exceeds X Minutes
- App usage Exceeds X Minutes over Q Previous Days
- App user took 50 photos in a photography app
- App user shared 2 photos in a social photography app
- App user invited others to join the app
- App user provided email address and signed up for the weekly brief email
Loyalty is a much better alternative to daily active users as it indicates how loyal users are. Examples include:
- Number of times user uses app in last 7 days
- Number of times user used app in last 30 days
- Number of session app user had longer than 15 minutes for a creative app
- Number of searches app user performed in a search app in last 7 days
A number of metrics get written-off and ignored as “Vanity Metrics.” The key is to be able to differentiate vanity metrics from actionable metrics. Some metrics are relevant for reporting and marketing and others can be used for growing your business.
Unfortunately, a majority of analytics packages provide reports only on vanity metrics and calculate most metrics inaccurately or without giving much thought.