Modeled after the human brain, a neural network, unsurprisingly, has many neurons. Each one of which receives some sort of input, and depending on that input, either fires and sends input to the next neuron or doesn’t. Then that neuron does the same, etc

In a neural network used for Machine Learning each of the neurons has some function, where it manipulates the information that it was given by the previous neuron and then passes it to the next. The first layer of neurons is the initial input, anything from pixel data from a photo, demographic data, etc. That is then passed to each layer until the computer gives you your output, whether the photo is determined to be a Dog or a Cat etc

We train a model by giving it many photos of Dogs and Cats, for example, and then telling it if its guess was correct or not. If it got it correct vs. incorrect there is a function that will go and update the parameters for each of the neurons to get closer to the desired output. After training your model on many many photos and iteratively updating the parameters for each of the neurons in the network the model can “learn” to always tell the difference between dogs and cats. The wonderful thing though is that it can now do that for any photo of a dog or a cat, not just the ones that we used to train it. Furthermore, in theory, this same neural network approach can be used to do anything, from language analysis to audio transcriptions etc. However, the more complex of a task the more data, time, money, neurons, etcetera will be required to train the model. A model that is simple and can recognize the difference between dogs and cats is not that complex and can be made by an amateur. The GPT3 engine used by OpenAi to generate text sentences almost indistinguishable from human writing has 2 billion neurons (the human brain as 82 billion) and cost millions of dollars to develop.

The biggest challenge that is faced when training a neural network is to avoid “overtraining the model.”

If we were using a neural network to approximate the function given by these data points this is perhaps what two outputs from two different models could look like. (In the same way that we would want to approximate this function we would want to approximate a function that tells us the difference between dogs and cats in photos). As you can see the one approximation looks smoother than the other, like a parabola, and should it be given new data outside of the range provided initially it is likely that it would continue to be a good approximation and perform well. i.e if we were to show it a photo of a dog that it had never seen before it is likely that it would guess correctly.

The overfit model, on the right however, is very different. While this function is a good approximation for the points given (the photos used to train it) the instant that we leave the range of the training data the function tapers off quickly. While this neural network does a good job identifying images that it has seen previously it would do terribly when looking at images that it has not seen before. It has “memorized” the answer rather than learning general principles and patterns and so it does not perform well in situations that it has not seen before.

Briefly, before I continue, here are two differences between this AI model and that of a human brain.

1. The AI Model’s “desire” is to get the correct output for this specific task. That is its feedback loop. People have the desire to feel good, get dopamine. That is our feedback loop. If the AI got the incorrect answer it changes its behavior (parameters) in an attempt to get better results. When we make a poor decision we change our behavior in the hopes of having more happiness.
2. The AI Model cannot choose, nor has any desire to, seek out additional or different information, experiences, or content based on previous training data. Its only “experiences” are the training data that we give it. People on the other hand make conscious choices to seek out future new or different experiences based on previous experience. We are self-determining. We have agency.

While I was looking at this graph, I realized that an overfit model is the same as human addiction. Human addiction is where someone is cheating. They have found a model that performs exceptionally well (gets them dopamine), but only in a very limited and specific context. Analogous to the overfit model, the moment that the addict leaves the small sliver of the human experience, where their model performs “well,” they get horrible results. Unlike the AI however, people can choose what they seek, and so it is tempting for them to return to the small sliver where they know that they can get good results and ignore the rest.

This is of course a range. I would suggest that the more overfit a model is, the more tempting the sliver becomes because of the poorer performance of the model elsewhere and the better it performs in the given range. An addict has a highly polar model. After this extreme example of a highly narrow and overtrained model there is a gradient. We all have varying models, some better trained in some areas than others; some trained with more data than others.

Ideally, you would embrace the full range of the human experience. Your model performs well and can get the result that you want (happiness/joy) for the whole swath of human experiences, not just a narrow sliver in the way that an addict does. The only difference between the two is how much truth, how much of the world, you are willing to accept, embrace, and grapple with. Training a more advanced model is more expensive. It takes more time, energy, diligence etc to train a model that performs well no matter how complex a task or how different an experience it is given. The same is true for people. We have the option to choose to stay in the range of experiences that we have already experienced, or venture outward and collect more data, knowing that doing so will be more costly and require more energy but will ultimately give us more of the human experience and a model that will perform better.

A model that performs well in the eternities, and in any of an infinite number of human experiences, that has learned general principles and patterns, is infinitely more valuable than a model that performs extremely well in one infinitesimally small context. Perhaps even the general model, given enough time, can become perfect. The perfect model doesn’t merely spit out the correct answer to the arithmetic of life, meaning that all “perfect people” are carbon copies of each other, however. The perfect model is one that maximizes joy and agency. Perhaps one person likes apples and another likes bananas. There is no moral qualm with this difference in taste. The perfect model allows a person to be creative and seek after what they desire, while still being aware of others and helping them to do the same.

From The God Who Weeps:

“Love reveals truth. It does not create the impression of truth; love does not merely endow something with a subjective truth–love is the only position or emotional disposition from which we become fully aware of the already present reality of the other person as more than a mere object among other objects in a crowded universe. Love alone reveals the full reality and value of the other person.”

People have the greatest capacity to bring joy or sadness to each of us. They are the “act”ors on the world stage where we live. It is unsurprising then that learning how to see other people, and learning how to be happy and cooperate with others, would be important. Empathy and love are of paramount importance when collecting data to train a model that understands the whole human experience. Love is an emotional form of knowing.

The moral person is one who knows how to make themselves, and by necessity others, happy. It is a person who has a model that accounts for and includes others.

The plight of the addict, and of all of us, is choosing to venture out and continually refine ourselves and in so doing avoid choosing to live in a smaller, more condensed version, of the world.