While developing Intelligence Explorer (video), we met with a number of product managers and data scientists at various app publishers to get feedback.  We already knew what the product managers, monetization folks and data scientists were trying to accomplish in a broader sense with their app.  We were mostly interested in how they  currently performed the task, and how we could drastically improve their workflow in terms of time they spent waiting for analysis.  Our primary goal was to make what we had envisioned, accessible to app publishers who either could not afford data scientists or had data scientists busy performing more strategic tasks.

We talked to many major app publishers over the course of a couple weeks both, before we wrote any code, and then again when we conducted usability tests. We received consistent feedback, but one major app developer’s comments helped shaped our product considerably.  Here are some notes from those conversations. #TBT

Notes

  1. Data scientists at this app publisher were in high-demand and very resource-constrained.
    1. Data scientists at this company were shared resources across multiple apps and they divided their time based on app popularity and overall strategy behind the app.
    2. Each of the data scientists had a favorite tool and programming language.
    3. We were surprised at the number of tools they used but the most popular ones were R,  scikit-learnWeka, Spark ML, some homegrown ones, Mahout, Matlab, and many others.
    4. Data scientist skills sets varied based on their background.
      1. Some had very strong statistical and mathematical skills,
      2. Others had strong visualization skills and
      3. Yet others had strong machine learning and algorithms skills.
    5. Data Engineers were not a shared resource but were dedicated to teams.  These were engineers who understood requirements and were predominantly users of open source machine learning algorithms.
  2. Product managers for apps coupled with a data scientist spent 2-3 hours daily analyzing data for
    1. engagement,
    2. core game actions and
    3. revenue.
  3. It took a long time for product managers to pull data out of the system and required help of a database administrator.
  4. Some product mangers wrote queries themselves and often to expedite data export.
  5. The exported data was provided to data scientists for analysis.
    1. Numerous times this had to be repeated as the analysis did not yield usable patterns.
    2. Numerous times the analysis, although accurate, arrived too late to take an timely action.
  6. Business folks at this app publisher thought of data more quantitatively, e.g. taking an action for gamers with lower revenue but high engagement, consistently week over week.
  7. Data scientists were in interested in watching how cluster centers are shifting for each dimension, but were not too concerned by day-to-day variations in cluster centers.

Learnings

  • Data scientists are resource constrained and can only focus on a select few strategic issues for some apps at a time.
  • They  use general purpose tools and often the analysis is too late to be actionable.
  • Product managers for apps coupled with a data scientist spent 2-3 hours daily analyzing data.
  • Mobile developers are spending less time on their apps and more on analyzing data.
  • App publishers struggle with expensive tools to do mundane tasks while relying on experts.

Pyze Intelligence Explorer

Intelligence Explorer is the key feature that came out as a result of these conversations and learnings.

It enables app publishers to explore the entire user base through automated segmentation and real-time explorations across key behavioral attributes like engagement, loyalty, revenue, attrition-risk, purchases, revenue, cohorts, and numerous more… in seconds.

 

Posted by Dickey Singh

Dickey Singh is the CEO and co-founder at Pyze and has over two decades of experience in mobile, Big Data and SaaS. He started Pyze to help app publishers engage, retain and grow their mobile users using automation. https://twitter.com/DickeySingh Get Pyze: https://pyze.com