Almost everyone nowadays has seen Artificial Intelligence (AI) being utilized in a plethora of unique ways, such as generating hilarious Dad jokes and creating realistic images of fictional animals. The latter is a prototypical example of a deep learning model. A deep learning model is just an umbrella term for all types of AI models that can recognize complex patterns or trends in pictures, texts, sounds, etc. Of course, deep learning models are extremely useful, from automating important yet mundane tasks to providing accurate insights regarding a company’s financial performance. But they have one issue: unexplainable inaccurate predictions. 

To further understand this issue with deep learning models, we must first take a look at their interesting computational mechanisms. Deep learning models are powered by neural networks, which are very similar to how our brains work: by having neurons pass information to other neurons. Similarly, neural networks take in information and pass it along layers of different neurons. Neural networks even have their own version of neurons, which are powered by inequalities. At each neuron, a neural network computer algorithm uses a specific inequality to determine which neuron the information is passed onto next. Every neuron has an inequality, with specific coefficients defining that inequality. The coefficients are all added into a vector or list, constituting the neural network’s system of weights. These weights are adjusted after a process called training. 

Training essentially compares the computer’s outputs to the expected output of an algorithm using already-known data. This is usually powered by Gradient Descent, a type of technique to minimize the difference between a computer’s outputs and the expected output. It is often compared to a ball rolling into a valley: The computer has a cost function that models the difference between a computer’s outputs and the expected output. Gradient Descent thus helps minimize the cost function by leading a figurative “ball” into the lowest “valley” of the cost function’s graph. Sometimes this graph has over 1 million dimensions, so our 3D example with the ball and valley does not quite make sense. The principles, however, still apply. By using calculus, one may find a direction of steepest descent to find a minimum of the cost function in one million dimensions.

How does this all apply to the issue of a deep learning model? Well, the explanation of its mechanisms shows that its input and output are both restricted using specified distributions. So, all that’s going on in the middle of the neural network is vastly unknown. The process of finding out what’s going on in the middle stages of these networks (called visual circuits) is complicated and has taken several years to discover. These unknown visual circuits inside a neural network cause issues. In an adversarial example, oftentimes, deep learning models classify a red light with the words “green light” written on it as being an actual green light. Adversarial examples like this pose serious safety issues in modern society.

Eventually, a high schooler named Achyuta Rajaram developed an AI algorithm to automatically detect these visual circuits. This algorithm was paired with another newly developed algorithm called CatFish. CatFish works by putting two images side by side and telling a deep-learning algorithm to identify both images. By doing this, Rajaram’s algorithm can detect the purposes of each visual circuit inside this deep learning algorithm. By doing this, one can add or delete visual circuits to fix these aforementioned adversarial examples, immensely increasing the safety and potency of a deep learning algorithm. 

As it is quite counter-intuitive, understanding how AI “thinks” is a difficult proposition. However, these insights are crucial in achieving the next levels of AI innovation, whether it be in self-driving cars or more accurate AI enhancement software.