First Steps

So we want to create an amazing chatbot, meaning a software program that takes an arbitrary prompt as input and generates an amazing response as output.

This necessarily requires an amazing set of rules which the chatbot follows to determine its response to any given prompt.

Rule-based AI

The first thing we might think of doing is to find such an amazing set of rules using what we know about language, grammar, and logic, and simply write it up in the form of code. Historically, this sort of approach has been known as Rule-Based AI.†Key terms will be highlighted like this when they are first introduced.[note]

Unfortunately, it doesn’t get us very far, simply because we do not know a set of rules that is even remotely close to being powerful enough to generate good responses to arbitrary prompts. After all, logical and grammatical rules are restrictive rather than prescriptive — they enable us to check whether a given statement is logical or grammatical, but they don’t tell us what statements to make, which is what we need to tell our chatbot.

Machine learning

But humans are pretty good at responding to prompts. So how do we do it if we don't know the rules?

Our thinking and actions originate from our brains, so the answer must be that even though we cannot formulate the rules underlying our responses in an explicit, mathematical fashion, the neural circuitry in our brains does reflect a set of (implicit) rules learned over the course of our lives.

Let’s take inspiration from this in designing our chatbot — instead of finding a set of almighty explicit rules and turning them into a piece of software, let’s create a malleable piece of software that can learn the rules for itself over time, just like we did. This approach is known as Machine Learning.

Imitation, supervision, reinforcement

Of course, in some sense, we have merely kicked the can down the road so far — instead of needing to find a set of rules linking responses to prompts, we now need to find a set of rules that will allow our program to learn the rules linking responses to prompts. Let’s see if we can make some headway on this by taking further inspiration from our own learning.

Generally speaking, we are far from understanding how exactly humans learn language, or, for that matter, anything else, meaning we are also (very) far from having an algorithmic understanding of all the rules involved in updating the neural circuitry used in responding to prompts. So we won’t be able to have our chatbot use the same mechanisms our brains use to learn, simply because we don’t know them.

But let’s see if we can derive some rough inspiration from looking at language learning on a much coarser level. Here are some things we know for sure:

1. As newborn babies, we don’t know language.
2. Over the course of several years, we learn to produce language, first by directly mimicking individual words and phrases we have been exposed to and then by producing our own, original speech. This phase is where the true black magic of language learning happens. We don't know how it goes down, but it seems to start with imitation.
3. Throughout life, our language and prompt-answering skills are honed via feedback from our environment:
   1. Sometimes, this takes the form of supervised instruction. For example, teachers explicitly show us how to write essays, parents might tell us how exactly to respond to a specific situation, and so on.
   2. Other times, the feedback is raw. For example, we keep honing our communication skills based on social cues and reactions from our peers signalling success vs failure of our latest utterance. As a result, successful tendencies get reinforced and unsuccessful ones get suppressed.

Fair enough. Let’s see if we can apply this same (very rough) recipe to our cause: