What are the key ideas that Artificial Intelligence is built upon?
The field of AI is built on a foundation of many brilliant ideas from Computer Science, Data Science, Neuroscience, and Machine Learning.
These ideas are typically expressed in the form of complex mathematical theorems, computer algorithms, and scientific principles.
However, the key insight to each of these complex ideas is often very simple and easy to understand — if it’s explained to you in simple terms.
So, to help you better understand these foundational ideas and get you excited about going deeper into AI, I’ve created this five-part series of articles.
In this series of articles, we’ll cover each of the key ideas listed below. However, each idea will be explained in very simple and easy-to-understand terms. As a quick guide, I’ve written the shortest and simplest possible summary of each idea, so you know what each is and why it’s important in AI.
In this first article, we’ll learn about a series of ideas from CS that were highly influential for AI.
- Lambda Calculus – any program can be represented as variables, functions, and function calls
- Gödel’s Incompleteness Theorems – math cannot be both consistent and complete
- Turing Completeness – all general-purpose computers can simulate any other computer
- The Church-Turing Thesis – a general-purpose computer can perform any computable task
- The Chomsky Hierarchy – there are four levels of languages of increasing complexity
Data Science (Coming May 1)
In the second article, we’ll learn about the most important ideas from data science that influence AI:
- Bayes Theorem – a method to update your predictions based on historical and new evidence
- The Law of Large Numbers – the more data you collect, the better your estimates will be
- The Central Limit Theorem – the averages of samples from any distribution is normally distributed
- The Pareto Principle – in some systems, 20% of inputs produce 80% of the outputs; and vice versa
- The Bootstrap – if you repeatedly sample with replacement, you can better estimate statistics
- The Grammar of Graphics – a language for describing virtually any data visualization in code
Neuroscience (Coming June 1)
In the third article, we’ll learn about the most important ideas from neuroscience that influenced AI:
- Hebbian Learning – connections in the brain that fire together wire together
- Reinforcement Learning – achieve goals by interacting with an environment and receiving rewards
- Bayesian Brain Hypothesis – the brain is a giant probabilistic inference machine
- Predictive Coding – the brain makes predictions and uses sensory data to update those predictions
- Attention Mechanisms – cognitive processes involve focusing on certain input data over others
- The Free Energy Principle – all living systems work to minimize entropy via active inference
Machine Learning (Coming July 1)
In the final article, we’ll learn about the most important ideas from machine learning that influenced AI:
- Gradient Decent – you can optimize a function by taking small steps in the downward direction
- The Turing Test – a thought experiment for judging if a machine is intelligent or not
- Backpropagation – train a neural network by passing error signals back through it’s layers
- The Kernel Trick – you can project high-dimensional data into a space that is linearly separable
- The Universal Approximation Theorem – a deep neural network can approximate any function
Once you understand all of these key ideas, you will have a much deeper understanding of AI. In addition, I hope you will see that these complex ideas are actually quite simple — provided they are explained in ways that are easy to understand.
To learn more, check out the next article in this series on The Ideas that Build AI from Computer Science.
