Business Problems Can't (Only) Be Solved With Algorithms

Machine learning and deep learning solutions have already established themselves as business tools that organisations use to improve customer experience, increase return on investment and gain a competitive advantage in business operations, among others. Both artificial intelligence models are based on complex algorithms that, on too many occasions, are granted more power than they actually have. We discuss the role of algorithms in business problems resolution.

The euphoria for machine learning and deep learning in the business world continues to grow. A research conducted by MarketsandMarkets predicts that, by 2022, the machine learning market will have grown by 44.1% in 6 years; from $1.03 billion in 2016 to $8.81 billion in 2022. The same study points out that data generation and technological advances are already among the main factors driving the market. In addition, machine learning technologies such as Azure Machine Learning are becoming increasingly prevalent in enterprises.

We explore the difference between machine learning and deep learning in this article: 'What Is the Difference Between Machine Learning and Deep Learning?'

Both are artificial intelligence technologies based on complex mathematical algorithms that enable machines to learn from data in a similar way to how humans do. Algorithms are used for a wide range of business operations and activities.

Nowadays, algorithms are practically everywhere and the quest to develop or apply a better algorithm than the competition is as widespread as the desire to decipher all the secrets of Instagram's new algorithm.

What is an algorithm?

According to Google, which has quite a bit of knowledge about algorithms, an algorithm is an "ordered set of systematic operations that allows us to make a calculation and find the solution to a problem". In practice, an algorithm is nothing more than a mathematical equation or a set of several mathematical equations applied to technological tools so that they do exactly what we want them to do.

Eduardo Peña, professor at the Faculty of Computer Science at the Complutense University of Madrid, explains it this way: "Ultimately, the job of computer programmers is to translate the world's problems into a language that machines can understand".

The algorithm is not the solution: The story of Vincent Warmerdam

In the business environment, algorithms are constantly used for the optimisation of operations and functionalities. They are data-driven and rely, more than it may seem, on the human mind. Data scientists and engineers develop algorithms with the intention of solving problems and perfecting operations carried out by machines, technological tools, platforms, etc. However, on many occasions, no matter how prodigious the algorithm, business problems are not solved or the expected results are not achieved. 

But why is this?

Let's start at the beginning. Despite being represented as such by society, an algorithm is not some kind of magic wand with superpowers or an evil entity programmed to sneak into our brains and reveal all our secrets. Algorithms are certainly capable of solving complex problems and doing things that in other times would have seemed extraordinary, but they do not do it on their own.

Sticking with mathematics, an algorithm is only a formula. To solve a mathematical problem, the first step is to understand the problem and then find out which formula should be applied. Applying the wrong formula will obviously not solve the problem. This does not mean that the formula is wrong —the formula in itself is correct— it is just not well applied.

The same goes for algorithms. Vincent Warmerdam, co-founder of PyData and specialist in algorithms and machine learning, addresses this problem and talks about his experience applying algorithms to solve business problems in his lecture 'The profession of solving (the wrong problem)'. In it, Warmerdam expresses the problem of applying algorithms through several personal stories that helped him to realise that, indeed, the algorithm is not the solution. What really solves business problems is everything that surrounds the algorithm: databases, data quality, data analysis, A/B testing, problem statement and, most importantly, "natural intelligence", as he calls it.

Warmerdam's story is based on a memory from his high school days. The teacher asked the students to apply statistics to a real database. At the time, Warmerdam was working in a theatre where a possible extension was being evaluated. Warmerdam asked his superior for the theatre's annual attendance growth figures and he quickly discovered that attendance growth had been steadily decreasing year after year. Vincent came to a very clear conclusion: the theatre should not go ahead with the expansion project, as attendance was gradually falling with each passing year. Warmerdam's discovery astonished his professor, who rewarded him with an A. His superior at work was also impressed and congratulated him on his discovery. Problem solved, right?

The following weeks Warmerdam continued to work at the theatre and during his shit he noticed how hot it was in the hall because it was packed. All the seats were taken and there were even people standing. The next day, the scene was the same and Warmerdam observed that every day that week, the hall was full to capacity. 

Then he understood. He had not solved the problem. He had applied the wrong formula. The theatre's attendance growth was steadily decreasing year after year because there was no more space and therefore no more people could fit. During the first years of activity, the growth was booming until the hall was filled almost every day and the space started to become too small. From then on, theatre attendance stopped growing. Not because people stopped coming, but because there was no more space. Warmerdam had not approached the problem correctly, had applied the wrong formula and therefore had not solved the problem, despite being congratulated by his superiors. 

After that first encounter with the algorithm, Warmerdam continued to insist and managed to build a successful career working as an expert in machine learning and algorithms. His long experience in the field has served to confirm that what happened to him in high school with statistics, happens all the time in the business world with algorithms. 

The problem with the algorithm

Warmerdam is convinced that algorithms do not solve problems and that, in fact, no matter how good an algorithm is, if poorly applied, it can make the problem worse. What is more concerning is that he himself, on more occasions than he would like to admit, has celebrated with his colleagues the resolution of a business problem after the development of an algorithm, only to realise days or weeks later that the problem had not been solved at all and that they were celebrating a fake victory proclaimed by themselves.

This is the problem with algorithms. The world is hell-bent on believing that an algorithm can solve anything. Vincent himself paraphrases several of his colleagues who, when faced with any problem, the first thing they say is something like: "Do you need to solve this? We're going to create a super-algorithm!" Immediately after learning that a client needs to solve a problem. Without even understanding the problem, analysing the data or making sure of its veracity. 

Another example of failure caused by blind faith in the algorithm is the famous 'Flash Crack' incident. On 6 May 2010, stock market algorithms caused the stock market to plummet by 1,000 points —practically 9% of the shares— for no apparent reason. After a few minutes everything returned to normal and the points returned to their natural state. However, to this day, no one can explain why this happened or what happened. The creators of the algorithm themselves were unable to determine why the algorithm had acted as it did, which showed that none of them really understood the whole process or what was behind the algorithm, confirming Warmerdam's suspicions that artificial intelligence is incapable of being intelligent without natural intelligence or, in other words, human intelligence. 

In this respect, while machine learning, deep learning and algorithms have been a huge breakthrough in the business world, it is essential that entrepreneurs and data scientists realise that algorithms alone do not solve problems. Applying the right formulas to the wrong problem can lead to a false sense of victory that, in the long run, always ends in defeat.