ML illusion

With over 12 years in the tech industry, I’ve worked across ML, data science, and analytics, navigating e-commerce industry. My journey spans managing teams, launching large-scale projects, and teaching machine learning and statistics to hundreds of aspiring professionals who later become leading specialists in the industry.

Today, as the CEO of a retail marketplace (the story about how I become a CEO in retail being a tech specialist needs another article for sure), I view these technologies through a broader lens, focusing on real-world business impact and cost-effectiveness. While machine learning can be transformative, I’ve observed a recurring pattern: in many practical applications, simple heuristics or rule-based systems often achieve comparable or even better results — at a fraction of the cost and complexity.

In this article, I’ll draw on my experiences to explore why ML often fails to live up to its promises, why simple heuristics frequently outperform in production, and how businesses can take a more practical approach to solving problems. This perspective comes from my experience in the e-commerce sector, where speed and transparency are very important. In this fast-paced industry, mistakes don’t just result in financial losses — they undermine customer trust, a far more difficult asset to rebuild.

1. ML Rarely Reaches Production

Studies and industry reports highlight:

• Only 22% of ML models ever reach production, according to a 2020 report by Gartner.
• Of those that do, 50–60% fail to deliver meaningful ROI, often because they are poorly aligned with business needs or require complex maintenance.
• A Forrester survey found that 47% of companies struggled to operationalize ML projects due to infrastructure and organizational barriers.

These numbers align with my personal experience. In one large-scale e-commerce project I oversaw, we spent several months building a complex demand forecasting model. After significant investment in engineering and cloud resources, we realized that a simple heuristic based on historical averages performed just as well. The ML model, however, came with recurring infrastructure costs and required ongoing retraining.

It shows a reality: for many businesses, machine learning projects are more of a liability than an asset.

2. Simple, Fast, and Effective Heuristics

In many practical scenarios, heuristics — straightforward rule-based approaches — outperform ML due to their simplicity and efficiency:

• Quick to Implement: Heuristics can be developed and deployed in days or weeks, compared to the months often required for machine learning models.
• Cost-Effective: No need for expensive infrastructure or specialized engineering resources.
• Easier to Interpret: Managers can easily understand and trust heuristics, unlike ML models, which often act as “black boxes.”

Several industry examples comparing ML approach and the possible heuristic:

1. Product Recommendations

• ML Approach: Uses complex algorithms like collaborative filtering or deep learning. These systems are costly to build and maintain.
• Heuristic: Show “top-selling products” or “recently viewed items.” Fast, cheap, and often good enough to meet business goals.

2. Dynamic Pricing

• ML Approach: Implements real-time price optimization using demand forecasting. While effective, it can confuse customers with unpredictable changes.
• Heuristic: Offer clear, rule-based discounts like “10% off for orders over $50.” Simple, reliable, and trusted by customers.

3. Churn Prediction

• ML Approach: Requires complex customer behavior models and ongoing data maintenance.
• Heuristic: Identify inactive users based on straightforward rules, e.g., “no purchases in the last 30 days,” and act on these insights.

4. Fraud Detection

• ML Approach: Uses anomaly detection models with high computational costs and potential for errors.
• Heuristic: Flag transactions over a certain threshold, such as “purchases over $500 from new accounts.” Low-cost and effective for many use cases.

5. Inventory Management

• ML Approach: Employs detailed demand forecasting, which is expensive and time-intensive.
• Heuristic: Apply simple rules like “increase stock by 20% during peak seasons.” Easy to execute and delivers reliable results for predictable demand.

3. The Cost of Machine Learning

The true cost of implementing machine learning is often underestimated. It’s not just about hiring an ML specialist, but also about:

• Infrastructure: Cloud computing costs for training and serving ML models can quickly escalate to $10,000–$50,000 per month for mid-sized projects.
• Data engineering: Cleaning, processing, and storing large datasets requires additional resources and specialized personnel.
• Ongoing maintenance: Models degrade over time due to data drift, requiring constant retraining and monitoring.

Compare this to heuristic approaches:

• A heuristic-based solution often requires just a few SQL queries or basic scripting, which can be handled by an analyst or a software engineer earning a fraction of an ML specialist’s salary.
• Maintenance costs are minimal, as rules typically don’t degrade over time unless the business logic changes.

4. When Does ML Make Sense?

This is not to say that ML is useless — it has its place in scenarios where:

• The problem is highly complex and data-rich: For instance, fraud detection systems, where subtle patterns in transaction data are difficult to capture with heuristics.
• Scale and personalization are critical: Companies like Netflix and Amazon leverage ML to deliver highly personalized recommendations to millions of users simultaneously.

Even in these cases, businesses must carefully weigh the benefits against the costs and consider hybrid solutions, where heuristics and ML complement each other.

5. Rethinking the Role of ML Specialists

One of the key challenges in implementing machine learning in real-world business environments is the gap between ML specialists and the business itself. ML experts are often highly focused on the technical aspects of their work, such as optimizing models, fine-tuning algorithms, or experimenting with the latest tools and techniques. While this technical focus is essential, it frequently comes at the cost of understanding the business context and priorities.

Many ML specialists approach their work with a non-business motivation — they aim to innovate, publish papers, or solve technical challenges, often without fully considering how their solutions align with the company’s strategic goals or deliver value. This disconnect can lead to solutions that are overly complex, costly, or misaligned with business needs.

For example, an ML team might spend months developing a sophisticated demand forecasting model, only to realize later that a simple heuristic could achieve similar accuracy while being faster and easier to implement. The motivation to create technically impressive solutions can sometimes overshadow the need for practical, impactful results.

In contrast, data analysts often bridge the gap between technical work and business objectives. They focus on extracting actionable insights using simpler tools and prioritizing communication with stakeholders. These professionals not only work with data but also possess strong soft skills, enabling them to explain complex ideas in a way that decision-makers can easily understand.

6. Practical Recommendations for Businesses

If you’re a manager considering whether to invest in ML, ask yourself these questions:

1. Can the problem be solved with heuristics? Always start with the simplest approach and scale up only if necessary.
2. What is the expected ROI? Ensure that the potential gains justify the investment in time, money, and resources.
3. Do we have the infrastructure and expertise? Without the right foundation, even the best ML models will fail.
4. Test your ML approach properly before expanding. It’s essential to test the ML model in a controlled environment and compare its performance to a simpler, well-established heuristic. For example, comparing a complex ensemble ML model (combining different algorithms) with a simple heuristic like recommending popular items can reveal whether the added complexity is worth it.

By prioritizing simplicity and aligning data initiatives with business goals, companies can avoid the common pitfalls of “ML for ML’s sake.”

Conclusion: The Pragmatic Path Forward

Machine learning is undoubtedly a powerful tool, but it is not a one-size-fits-all solution. For most businesses, especially in e-commecrce, those operating on tight budgets or with limited data, simple heuristics can achieve 80% of the results at 20% of the cost.

Instead of chasing the ML hype, businesses should focus on pragmatic, cost-effective solutions — and remember that sometimes, the smartest answer is also the simplest.