What To Do When Your Team Hits A Wall During An AI/ML Project

From a leadership perspective, what do you do when someone in your team gets stuck during the implementation of an AI/ML project?

As we have learned, AI/ML is using data to answer business questions. There is a lot that can happen between gathering that data and making a prediction that answers a business question.

The first step in AI adoption is proving the value of machine learning in your company. If you use lean methodology (which we highly recommend), you will establish initial value hypotheses, test them, gather feedback, and iterate based on what you learned (check out my previous blog post titled, “3 Steps In Your AI Journey” to learn the sequential steps of AI adoption).

Sometimes it may not be obvious where to go next, either from a strategy or technical perspective, and there are many reasons why someone could get stuck.

The most common place where AI/ML engineers could hit a wall is during the diagnosis and improvement of AI systems exhibiting suboptimal performance or strange behavior.

AI is different from traditional software development. Strange behavior or odd predictions made by machine learning models may not be due to ‘broken’ code. Often times an ML algorithm will seem to perform well on the surface, but there are subtle issues behind the scenes that impair its performance.

One of the most common ways in which AI engineers approach these types of issues is to improve the machine learning algorithm itself (in technical terms, this is referred to as ‘hyperparameter tuning’).

However, most of the time these tweaks only produce a marginal increase in the performance of the algorithm. ML engineers could spend days or weeks on this and experience great frustration because nothing seems to help. Managers and business leaders could also experience stress and frustration due to the apparent lack of progress.

How do you avoid wasting time? Where should machine learning engineers look first when attempting to fix strange behavior or improve suboptimal performance? They should first look at the data used to train the machine learning algorithm.

The performance of ML algorithms depends mostly on the quality and quantity of the data used to train them. If there are problems in the data, there will be problems in the performance of the algorithm (check out my blog post titled, “AI Fundamentals For Business Leaders” for more information).

Potential problems in your data (both structured and unstructured) include bias, incorrect numeric values, missing values, corrupted files, improper preparation, incorrect transformations, and more. You want to make sure that your team has thoroughly assessed, cleaned, and prepared the data to make sure it’s not the source of the problem (most of the time it is). This also involves examining every step in your data engineering pipeline.

For the specific case of problems in image and video data for object detection use cases, check out my blog post titled, “Things To Watch Out For In Your Labeled Image Data”.

Once this has been fixed/verified, the performance usually increases significantly and/or the strange behavior goes away. There is a chance that the data is not the (only) source of the suboptimal performance, and that is when your team should look at more technical aspects of AI/ML development.

AI technical details are beyond the scope of this article, but you could simply ask them a few questions to get them thinking about potential performance leaks:

“Is the chosen ML model architecture the best for our use case?”

“Are you initializing the weights correctly?”

“Have you found the optimal learning rate?”

“Are you using the correct activation functions?”

“How are you using batch normalization?”

“Are you exporting the models correctly?”

These are some common ways in which machine learning engineers could get stuck during the implementation of an AI project. Aside from coaching from a technical perspective, there are other internal traits that will compel them to break through quickly, or never even hit the wall in the first place (check out my blog post titled, “Top 3 Traits To Look For When Hiring AI Talent” for more information).

If you need help to accelerate your company’s machine learning efforts, or if you need help getting started with enterprise AI adoption, send me a LinkedIn message or email me at info@carloslaraai.com and I will be happy to help you.

Subscribe to this blog to get the latest tactics and strategies to thrive in this new era of AI and machine learning.

Subscribe to my YouTube channel for business AI video tutorials and technical hands-on tutorials.

Client case studies and testimonials: https://gradientgroup.ai/enterprise-case-studies/

Follow me on LinkedIn for more content: linkedin.com/in/CarlosLaraAI

2 thoughts on “What To Do When Your Team Hits A Wall During An AI/ML Project

Leave a Reply

%d bloggers like this: