Do you know how old algorithms really are? Or where they came from? What is an algorithm? Are there different types of algorithms…? Muhammad al-Khwarizimi was a 9th century Persian astronomer and mathematician. His textbook On Indian Numerals was translated into Latin four centuries later. It is considered the most important source for the Indo-Arabic numeral system and written arithmetic. His surname even sounds phonetically like an algorithm “Algorismi”. Ada Lovelace was the first person to write down an algorithm intended for a computer in 1843. She is considered to be the first female programmer.
In the latest of her Duet interviews, Dr Caldarola, author of Big Data and Law, and IT-Expert Prof. Dr Katharina Zweig talk about algorithms and heuristics and their daily use.
Let’s start our duet by explaining what an algorithm is.
Prof. Dr Katharina Zweig: We have heard a lot about algorithms in the last decades. However, it is a specific term, with a slightly different meaning from how it is often used: an algorithm is a sequence of instructions that can lead to a proven solution of a mathematical problem. For example, there are multiple ideas about how to sort things on a computer: a bank might want to sort all of their borrowers by the level of their credit. Given any sequence of instructions to supposedly solve this problem, this needs to be proven mathematically to be called an algorithm. In the media, however, any type of software is nowadays called an algorithm. This is a bit unfortunate, as AI does in almost all cases not use real algorithms, but only so-called heuristics. A heuristic is a sequence of instructions that is likely to yield a solution, maybe even a good one. But in contrast to an algorithm, we do not have a mathematical guarantee that the solution will be optimal for a given problem.
Is an algorithm more objective than the human brain and, if so, why?
Again, we need to differentiate between real algorithms and those that are actually heuristics. Using the definition above, an algorithm is objective: it is proven to find the best solution to a given problem. For example, when you use a navigation system, the algorithm behind it will find the absolutely shortest path between your location and some other location. However, if you want to find the fastest solution given real traffic, no algorithm can guarantee the fastest route. Instead, the system will use AI to predict which street will be congested at which time point. There are heuristics for these predictions and we can measure in hindsight, how well they performed. But there is no objective guarantee that the heuristic will find the fastest way. Thus, algorithms are always to be preferred, but in most cases, no algorithm exists and we need to resort to heuristics which are not objective.
How much data do you need to develop an algorithm? Does the quality of the algorithm depend on the quantity and quality of the data used? What quantity and quality of data is required?
The interesting thing is that the development of an algorithm in the strict definition mentioned above does not need any data. It needs mathematical creativity, pens, a black board, time, frustration tolerance, and a bit of luck. Finding the best sequence of instructions to solve a given problem is an abstract, theoretical process. But now comes the twist: many interesting problems cannot be solved efficiently by an algorithm. There are two reasons: either because we haven’t found one yet, or what we have found is too slow. Sometimes we have an algorithm, but it would literally take longer than 1000 years to wait for the result.
Thus, nowadays, computer science answers some questions by designing heuristics that try to find statistically robust patterns in large amounts of data – independent of the concrete problem to solve. This approach is called machine learning. Machine learning consists of many different heuristics, based on some human intuition of how learning by seeing multiple examples works. For example, the machine could be shown information on a bank’s borrowers and on whether they paid back their loan or not. The machine could then be tasked to find the best patterns to identify those that are not creditworthy. These ideas about learning can be simple mathematical equations like a line through some data, or the famous neural networks. Neural networks are just huge sets of mathematical equations with an even larger set of parameters or weights, that model which of the presented information is most important. All these ideas of learning have in common that they read data from the past to find patterns to make decisions for the future. But they identify and store these patterns in different structures. In the above examples information is stored in the weights in the respective mathematical equations.
The problem to solve is then to find the best patterns in the data to make future decisions. For example, find the best line through the data or the best weights in the neural network such that they can identify creditworthy future applicants. However, only the simplest of these problems can be solved by an algorithm, namely finding the best possible line. However, the line is very often a very poor representation of the data. Thus, it cannot be used in reality. All other ideas about learning create mathematical problems for which we do not have any algorithms, only heuristics. In short, we only have algorithms with proven optimal solutions for the simplest learning tasks. For more complex learning tasks, we only have heuristics that might not result in the best model to represent the data.
It is believed that enough data will make it very likely that the heuristics can come close to a very good solution. However, in most cases we do not know exactly how much data we need.
Are AI systems developed with raw data or do they require processed data? If the latter is the case, what ‘mold’ is required and how is it ‘made’?
My favourite quote by Geoffrey Bowker answers that question:
My favourite quote:
“Raw data is both an oxymoron and a bad idea”.
Geoffrey Bowker
The moment someone decides to store some information is the moment the data is “cooked” according to some flavour of choice. What we select and what remains undocumented is a choice. There is no such thing as raw data, all data is cooked.
AI systems work with probabilities and correlations, while our life, science and law follow causality. How is this compatible?
In my view, machine learning is interesting as a tool to find hypotheses about how the world works. However, the patterns found may be incidental. Hundreds of years of epistemology, the science of science or how we can deduce facts from observations, have shown us that identifying correlations is not enough. Thus, if an AI system actually does show superior decision-making it is necessary to identify why that is the case. For this, we have methods from the natural sciences: We need to isolate possible causal factors and find out whether they are indeed causes or not. I see no way to shortcut this process. The question of what is a fact cannot be left to the methods of machine learning alone.
AI systems are touted for their efficiency, measurability, standards, automatism and much more. Is that at the expense of humanity because, for instance, people are seen as “poor” decision-makers or because people are predictable based on past behaviour?…
The first question is: Under which circumstances are humans poor decision makers? Of course, psychologists and behavioural economists have shown ample examples in recent years where humans have made biased decisions. Sometimes these can also be called irrational decisions. The book Noise by Kahnemann, Sibony and Sunstein summarises many of these studies in which experts are asked to come up with decisions, but it turns out that they do not agree in their findings very well. This can be costly, e.g., if the risk of some insurance case is wrongly calculated. The authors also provide good rules to reduce the noise. So, can AI systems help to make better decisions than humans? I do not know many studies in which this has actually been proven. I am open to the idea, but I would like to see studies. Furthermore, the question is what would we conclude from such a finding? The thing is that machine learning is based on identifying statistical correlations, not necessarily causal relations. Thus, we do not know whether the finding can be generalised and applied to future data or not. Can a machine replace a human decision under this condition? The answer to this question is what I consider in my new book. My general answer is a machine can only safely replace humans in decision making if the machine comes to an answer that can be verified independently. For all kinds of judgments, like the suitability of a job candidate, it cannot.
We talk about errors in an AI system as a matter of course. But when is the result of such a system actually “good”? The word “good” certainly differs according to various cultures, ethics, morals, etc. How does that go hand in hand with the worldwide use of an algorithm?
A decision is good if it is correct. If we cannot define what a correct decision is then we should not use AI systems. The decision of a machine is correct if an independent verification process comes to the same conclusion. If there is no such verification process, we should not use AI systems which somebody just claims that they are good to go. Unless in situations that are not very harmful, such as product recommendation systems.
How does a consumer of a product or service that contains an AI system get the necessary evidence or the training data behind it to be able to show that the algorithm contains an error? Does the average consumer have the necessary competence for this and isn’t the AI system rather “opaque”?
If there is a verification process, even if it is expensive to use it, the person can at least identify incorrect decisions. For example, if a machine predicted a Corona infection based on the coughing sound of a patient, a PCR could be used to verify that result. For creditworthiness, it is already much more complicated: Of course, it can be checked in 10 years whether persons deemed creditworthy by the system, actually did pay their loans back or not. But we cannot know whether the persons deemed unworthy of credit would have paid back their loans or not. Thus, for risk predictions, we already have a problem of knowing whether the system is accurate or not.
I assume from your answer that it takes a lot of knowledge and experience to detect an error in an AI system. And I also assume that it takes even more expertise to find the cause for it after the fact. If an employee in a company has detected an error in an AI system, it will certainly be difficult to find a colleague who is competent in terms of IT (for the AI system) and the specialist area (e.g., the area of application taxes) who can verify the opinion. It is even more difficult to persuade colleagues to stop using the system or even “repair” it, because for the company this means effort, time, loss of sales, possibly loss of reputation and so on. Is that an unsolvable dilemma?
An AI system is a process used by someone. If that process is faulty and the consequences of this faulty process are high, society needs to find ways to stop them. I do not think that there is something inherently different with respect to whether AI is used or not. However, it is difficult to find out whether they are faulty or not, as already discussed. Thus, those systems for which we have a hard time proving their faultiness, should not be used for decisions with a high social impact.
Prof. Dr Zweig, thank you for sharing your insights on algorithms, heuristics and their impact on daily life.
Thank you for having me.