Inside the AI Engine

Using AI can feel surprisingly simple. All you need to do is type a request, select an option, or upload something, or simply use a tool with AI working in the background. Behind that interaction, however, is a much larger system involving data, models, computing power, and physical infrastructure.

You do not need to be a computer scientist to understand these building blocks. Think about a car. You likely understand that an engine makes it go, fuel keeps it running, and roads and other infrastructure allow it to travel, without knowing how to build or repair a car yourself. We can approach AI in a similar way. Understanding a few key components can make what happens in the background much less mysterious.

If AI were that car:

The car analogy gives us a simple way to see how these pieces fit together but let’s take a closer look at each part of an AI system including what it is, what it does, and how it contributes to the whole.

Data is the fuel behind AI. That’s because data provides the examples that machine learning systems use to identify patterns and relationships.

Rather than programming an AI system with a rule for every possible situation it might encounter, machine learning allows a model to learn patterns from examples. Those examples are data. As a model works through them, it identifies patterns and relationships that help shape what it is able to do.

So, where does all of that data come from? Well, it depends on what the AI is designed to do.

A language model may learn from large collections of text. An image recognition system may be trained using labelled images. A recommendation system might learn from patterns in previous patterns in what someone has clicked, watched, or interacted with before. A weather prediction system might use years of observations such as temperature, precipitation, and atmospheric conditions. In other words, there isn't one kind of “AI data.” Different AI systems learn from different kinds of information depending on what they are designed to do.

• Text → Language AI
• Labelled images → Image Recognition
• Previous interactions → Recommendation System
• Historical weather observations → Weather Prediction

One important thing to keep in mind is that more data does not automatically mean better AI. The kinds of examples an AI learns from, and the quality of those examples, can influence what it learns and how well it performs. In Chapter 2, we’ll take a much closer look at data quality, and what can happen when it is incomplete, inaccurate, or biased.

Data gives AI something to learn from, but that’s only part of the picture. AI systems also need the computing power to work through the data. The computing power, which can also be understood as processing capacity, is often referred to simply as compute. Think of it like working through a large stack of math problems. One person could solve the problems one at a time, but a large team could divide the work and solve many at once. Computing power works in a similar way. Greater processing capacity allows computers to perform more calculations, often much faster.

Two important types of processors are CPUs (central processing units) and GPUs (graphics processing units). CPUs carry out many of the general instructions needed to make a computer work. GPUs are particularly good at performing many calculations at the same time, making them well suited to the mathematical work involved in AI. As processors such as GPUs have become more powerful, so has the amount of computing power available for AI. This has been one of the major drivers of AI’s rapid progress, allowing AI systems to work with more information and take on increasingly complex tasks.

Canada’s Growing AI Compute Capacity

So, where is all of this computing actually happening? Often, the answer is “in the cloud.” But despite its name, the cloud is very much connected to the physical world. It relies on computers and other equipment located in data centres around the world.

Data centres are physical facilities filled with computers and other equipment used to process and store information. Inside, processors provide computing power, while storage systems hold data and network connections which allow information to move from one place to another, and all of that equipment needs electricity to operate and generates heat that needs to be managed. So, data centres rely on cooling systems, network connections, security, and other equipment to keep everything working reliably.

But, how does all of this connect to the AI tool you might use on a laptop, tablet, or phone? When you use a cloud-based AI tool, your device doesn't do all of the computing work itself. Instead, it can send a request over a network to computing resources in a data centre. Those resources process the request and send the result back to your device.

Ultimately, what appears to be a simple interaction on your screen depends on a much larger physical system working behind the scenes.