How Neural Networks are More Powerful than We Think
A written transcript of a TED talk I completed.
What does music, X-rays, language translation, and eye cancer have in common?
Well, they all relate to human senses. Our hearing, breathing, speaking, and seeing.
And that is correct.
But what’s something else they have in common?
Apparently, this. This is a neural network.
A neural network is one of the most common and versatile types of models used in machine learning tasks, where humans train it to solve problems. The beneficial thing about machine learning is that, unlike in software engineering tasks, models in ML code their own algorithms to solve problems instead of relying on humans to, which saves lots of time and effort.
The varied topics above, music, X-rays, language translation, and eye cancer, would also have machine learning in common because ML models have the potential to solve each one of these tasks. For example, Google translate is already created and widely in use.
ML models can solve other tasks as well. They can perform classification, such as classifying between and identifying types of animals; they can do recognition, such as finding if there is a pair of glasses in an image, like in the picture below:
This may not seem very important, but it can be. This same model can be trained to identify objects in the walking path of a visually impaired person, preventing them from tripping or hurting themselves if necessary.
Today we will focus on a specific type of model, the neural network. Understanding its inner workings is key to finding new solutions to problems. But you may be wondering, how? How can neural networks really be a solution for so many different complex tasks and problems?
To understand how it works, picture a graph below:
It looks something like a sine curve, and each dash represents a data point, an x and y pair. Neural networks are basically functions that approximate data values, trying to model a trend. Basically, they have inputs and then output 1 output.
In the first picture above is the structure of a neural network. Each node/circle is a neuron (hence called a neural network, as it basically models how our human brains work), and going left to right there are layers or columns of neurons that are all connected and work together to read input data, or x, learning features and relationships between them and finally generate a prediction or output (y). The cool thing about neural networks is that they allow multiple inputs, or x values. For example, the amount of CO2 emission of a car can be influenced by engine size, number of cylinders in a car, and amount of gasoline burned, all real-life factors that can impact an output. This situation makes neural networks model real life data more accurately.
So basically, the process of using this neural network is giving it input data, training it to learn the data, and finally outputting something. To get this neural network to output something, you first need to give it input data in the form of a dataset (audio, images, videos, etc) so this network can train on it and learn and look at patterns in data to get output. This is sort of like studying for a math test- doing practice problems will give you more examples to help you learn specific patterns to get more accurate answers on the test.
N(x1,x2…) = w1*x1 + w2*x2 +… + b
Here is the equation for a neural network: N stands for neural network, w is weight, b is bias, and x represents the different inputs or factors. Weights and bias, which are both numerical values, are arbitrarily changed or tuned by the specific neural network model based on patterns they see throughout the data as the function studies it. The weights sort of determine the importance of a specific input and decides how much influence the input will have on the output. Take notice here that there is a different specific weight for each type of input or x. Because in real life situations, there will be multiple causes that cause an effect and some factors are more impactful than others, this arrangement ensures that the neural network models the real world even more accurately. Biases, which are constant, are like the b term or y-intercept in the linear equation y=mx+b: its task is to shift the position of the curve left or right to better fit the data points.
But one problem is, the natural and inherent matrix multiplications that exist within a neural network ensures that neural networks fit linear functions. However, most real world relationships are nonlinear, such as exponential or logistic. So, people include a ReLu, or a rectified linear unit, to use with neural networks to get rid of linearity. ReLus can help neural networks model any relationship that is nonlinear like quadratic relationships. A relu applied to each neuron basically takes little pieces of basic linear/piecewise functions, adds it up, and approximates any curves such as the sin function we saw earlier. In sum, ReLus use simple linear functions and combine them to make complex ones, making the model more accurate and fitting for real world relationships.
Another interesting method neural networks use to tune weights and biases to better approximate a relationship or function is by back propagation, where the neurons basically turn back to previous neurons to fine tune all their weights to decrease error rates.
So what? Why is this all important? Well, neural networks are proven to be universal function approximators, as said in the words of youtuber Emergent Garden. That means that neural networks can essentially model any function. Since a function basically can have any input or output, these networks can train and solve any problem hypothetically or ideally, which is huge. That’s what makes them so versatile and powerful and super useful for humans. From cancer, to language translation, lung disease detection, aid the hearing impaired, and more. For example, for language translation, the input could be English and output Spanish, input is lots of cancer images and neural networks learn each specific feature to classify the type of cancer or detect cancer, etc. Because images, text, audio, etc. can all be encoded as numbers, a neural network can understand them. Though there are some other types of solutions to these above problems or challenges, neural networks can serve as an alternative, more efficient and reliable tech solution as they can reduce human efforts (they can basically train themselves). Humans do have to code the model structure and optimize and tweak areas of model to get higher accuracy, but once a model is put into use, it requires much less human effort as it relies on itself. And it is cheap as well.
With enough data given to these networks, they can learn to identify patterns such as certain words/phrases, certain sounds, shapes of tumors, etc. which is crucial to solve humanity’s problems in healthcare, language, and in aiding human senses and disabilities and can potentially solve any problem in the world.
However, there are some caveats to the design of neural networks.
For example, if there are too many outliers in a dataset, it can impact the accuracy of neural networks because the model may get confused and value them too critically. Another limit is the size — we can add more neurons to increase complexity of the network but there is going to be a size and memory limit due to the limited amount of GPUs of a Jupyter notebook, which is usually the environment people train models on. Lastly, a neural network’s accuracy/performance is highly dependent on the amount and quality of data. If we only make it learn to identify a cat by giving it 3 picture of cats, it’s not going to learn that effectively. However, if we feed it 1,000 images, then the neurons would have enough varied examples to generalize the shape and features of a cat. Similarly, if our goal is to have a network identify skin cancer, we need to give it pictures of a variety of skin colors, or else the architecture will be inaccurate for minority groups as it would not have enough examples.
But despite their setbacks, neural networks are one of the most versatile and powerful inventions in the ai field, producing numerous solutions to problems around the world. Neural networks are present already in a variety of ways. Google translate, personalized Spotify music recommendations, you name it. And recently, an AI company from the UK, Sonantic, used a neural network to restore the voice of actor Val Kilmer, who was inflicted with throat cancer many years ago. The Top Gun actor had provided voice recordings from before he had cancer, and he can now speak through his old voice, helping him communicate with others around him much easier.
So, how can we humans use an AI solution as a product? After a neural network is trained to high accuracy/performance, the next is to transform it into an application product, such as a web or mobile app. For example, a skin cancer detection model can be embedded into a web app, where people can upload skin images for early, cheap, and efficient detection. This product can be distributed to rural and low-income areas that may not have the most efficient or convenient access to healthcare.
This is also personal to me. My family all come from different parts of China, be it a mountainous area, a flat plain, in the city, etc. Each place has a unique dialect of Chinese, and it is said that there are over 300 distinct dialects in China. They are so different that if a Beijing local who speaks Mandarin went to a Guizhou region village, the Beijing person would not understand a lot of the words and slangs used (but most village locals understand Mandarin as it is the most common language in movies and the media). Since there is Google translate for different languages, it would be interesting if thre was one for dialects so the diverse villages can connect with each other more and form a bigger community.
Another thing is that my 80 year old grandma had lost her hearing years ago. Even with expensive hearing aids, she still has trouble hearing many sounds around her. A neural network can be used as another mechanism to hopefully help her and other hearing impaired people identify sounds and facilitate their navigation.
Although there is a potential AI solution for many problems and tasks, right now, neural networks aren’t the cure to all yet. Such AI products have not been widely used as a commercial product yet. There is still much research to do about their effectiveness, and what happens if they make major errors.
However, if developed enough over time, they can be a real-life solution. They have the potential to solve many of humanity’s most critical problems and in future society. Neural networks, as you can see, are more powerful and versatile than we think, so we’ve got to realize that we have the will and power to impact our own future with neural networks.
So let’s strive to explore more with neural networks and other models in AI and ML.
Sources cited:
https://towardsdatascience.com/whats-the-role-of-weights-and-bias-in-a-neural-network-4cf7e9888a0f
https://www.newsweek.com/top-gun-maverick-val-kilmer-voice-throat-cancer-tom-iceman-kazansky-1710799
