For any organization, 20% of the AI/ML use cases drive 80% of the business value.
How do we identify this 20%?
Always start with specific business outcomes. Forget about machine learning at the beginning and let such a solution (if any) emerge naturally out of a systematic discovery process.
In 2021, my team and I deployed a machine learning solution to production with the goal of increasing gross revenue above a baseline number. The initial release took place in a relatively small market to prove the business value, learn the operational challenges, fix critical bugs, and validate several hypotheses.
In that initially small market, we increased gross revenue from a $67,000/month baseline to $124,000/month post-release. As we scale, the projected new revenue exceeds $1 million/month. And this is just 1 of our production ML solutions.
This ROI success story is extremely rare today, and we want to show you the process we followed to achieve this business outcome.
1) Identify the company’s key strategic initiatives
Every organization has quarterly and yearly goals defined by the executive leadership team. Goals typically include gross revenue growth, improved profitability, increased customer retention, customer acquisition, etc. KPIs beyond revenue and profit are company-specific.
Find out the top 3-5 KPI objectives and assess the opportunity to improve each one individually. Given the on-demand staffing industry where we connect people and work at scale, my team and I started with gross revenue growth:
“How to increase on-demand gross revenue in the US?”
Note how we frame the KPI choice as a concise and specific business question.
2) Break up the business question into components
The revenue growth question is too complex to tackle directly. The best practice is to break up the question into its components, quantify them, and identify the areas of highest impact to focus on first. Domain knowledge of the business is vital for proper disaggregation.
Revenue naturally breaks up into supply and demand. The first decision is which side of the marketplace to focus on. To answer this, we ask a key question:
“Are we meeting all existing customer demand?”
In the on-demand staffing industry, we quantify this question through a KPI called fill rate. Upon investigation, my team and I found fill rate was well below 100%. We decided it made the most sense to focus on filling the existing demand. Therefore, we focused on supply (workers) instead of demand (customers).
Within worker supply, the 2 components we could focus on are recruiting & onboarding new workers, or increasing the utilization of existing workers (hours worked per week). We treat capacity per worker as a constant, fixed at 40 hours per week.
3) Develop hypotheses and validate them through data analysis
Before proceeding, we ask another key question:
“What are the worker utilization statistics across US segments?”
If median worker utilization is already close to 100%, it might be a better idea to recruit new workers or increase retention. If it’s well below 100%, we have an opportunity to increase worker engagement to drive increased utilization.
Note that we are addressing gross revenue growth indirectly by focusing on the worker utilization KPI. This makes our efforts more targeted and simpler.
Validating our hypotheses required a combination of SQL data analysis (especially OLTP queries) and subject matter expert (SME) interviews. Ultimately, the OLTP schema is the business truth.
We created a list of initial hypotheses and created corresponding epics/stories to be completed over the next 2-week sprint. Upon completion, we validated:
- Median worker utilization was well below 100%
- 80% of the revenue is generated by 20% of the workers (Pareto principle)
After synthesizing all results and aligning with business leaders/stakeholders, we concluded that increasing median associate utilization was the most viable way to drive revenue growth.
4) Determine whether the problem is a candidate for a machine learning solution
The disaggregation diagram above is simplified, but the logic trees went a lot deeper. At those deep levels of details, we collaborated with business and technology teams to brainstorm possible solutions to increase median associate utilization.
After going through steps 4) to 18), we ultimately built a fully serverless machine learning solution in AWS to text the “job of the day” to workers using a recommendations engine:
Feel free to go back to the beginning of this article to review the business results from this solution. It may all sound easy, but it was a herculean effort spanning several months and teams. On the verge of failure multiple times, we continued pushing through the walls until we succeeded in providing an ROI to the company.
I still find it amazing how we started with a question (“How to increase on-demand gross revenue in the US?”) that had nothing to do with AI/ML, and systematically converged on a production solution generating monthly cash flows.
This was a simplified view of our AI/ML adoption methodology to show you the approach we take for any machine learning solution. Did you find it valuable or insightful? Let us know in the comments!
Feel free to message me if you’d like to learn more about the entire process beyond the 4 initial steps covered here.
Subscribe to our weekly LinkedIn newsletter: Machine Learning In Production
Reach out if you need help:
- Maximizing the business value of your data to improve core business KPIs
- Deploying & monetizing your ML models in production
- Building Well-Architected production ML software solutions
- Implementing cloud-native MLOps
- Training your teams to systematically take models from research to production
- Identifying new DS/ML opportunities in your company or taking existing projects to the next level
- Anything else we can help you with
Would you like me to speak at your event? Email me at firstname.lastname@example.org
Follow our blog: https://gradientgroup.ai/blog/