See how a neural network learns, from one neuron to a small transformer. You train a network that routes Larkfield's support tickets, find out why its training goes wrong and fix it, and then open a real pretrained language model to see what is inside.
Read every lesson as a guest, or sign in to enroll and track your progress.
Enroll to save the lessons that you complete. It is free.
By the end of this course, you will be able to:
Deep learning powers the image, speech and language tools that you use every day. From the outside, it can look like magic. From the inside, it is a few simple ideas repeated many times: weighted sums, a measure of error, and small steps that make the error smaller.
In this course, you build those ideas up one at a time. You start from the logistic regression that you already know and turn it into a neural network. You see how the network learns, why its training sometimes fails, and how to find the cause from evidence instead of guessing.
Larkfield’s help desk gives you one task for the whole course: read a ticket and send it to the right team. You solve it with a small network, then with embeddings and attention, and finally with a tiny transformer that you train yourself. You measure each one against the simple linear model of the machine learning course, which stays the better router on these tickets. The tiny transformer is the weakest of them, because a few thousand tickets are too few to learn a language from nothing. At the end, you open a real pretrained language model, the answer to that problem. You read its parts and measure what it costs to run.
Diagrams, interactive explorers and annotated outputs carry the main ideas, so the text stays short. Every example runs on an ordinary computer without a graphics card.
You completed Python for Practical AI Engineering, Essential Mathematics and Statistics for AI, and Machine Learning, From Problem to Reliable Model, or you have the same skills. You need a free Google or Kaggle account for the notebooks, or Python 3.12 or later on your computer. Everything runs on an ordinary computer without a graphics card, and all tools are free.
Courses to take first: Python for Practical AI Engineering, Essential Mathematics and Statistics for AI, Machine Learning: From Problem to Reliable Model.
People who finished Machine Learning, From Problem to Reliable Model, or know the same basics, and want to understand how neural networks and transformers really work. Fresh graduates and professionals moving into AI roles.
Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses
Learn what a neural network is, starting from the logistic regression of the machine learning course. One neuron is a weighted sum, a layer is many neurons, and activations let a network draw curved boundaries that one straight line cannot.
Meet tensors, the arrays that hold every number a network uses, and learn to read their shapes. Set up PyTorch, check that it works, and run Larkfield's tickets through a first, untrained network.
See how a network learns. Measure how wrong it is with a loss, walk downhill with gradient descent, pass the blame backwards with backpropagation, and put the steps together into a training loop that you check on held-out data.
Find out why training goes wrong and how to fix it. Read the curves for a bad learning rate, control overfitting, and debug a broken experiment step by step with evidence.
Turn words into vectors with tokens and embeddings, see where a bag of words fails because order and context matter, and calculate attention by hand on tiny matrices.
Assemble a transformer block from attention, a feed-forward layer, residual connections and normalization. Add positions and masks, train a tiny transformer on the tickets, and compare the two ways language models learn from raw text.
Open a small pretrained language model. Separate training from inference, measure what generation costs in memory and time, and record an experiment so that another person can repeat it.
Go deeper with the mathematics behind the course's ideas. Turn text into vectors with counts, TF-IDF and embeddings, see how word2vec and GloVe learn word meaning from raw text, write attention in matrix form, and follow the original 2017 transformer step by step. Optional: the certificate does not require these lessons.
Four end-to-end case studies on open data. Classify clothing images, test a neural network against gradient boosting on a table, fine-tune a small transformer, and train a tiny language model. Complete at least one for the certificate; each one is a portfolio piece.
Complete at least one case study for the certificate. The others are optional. What each one covers
Use every module in one practice project. You collect evidence, and check it against the course outcomes.
Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.
Train and diagnose a small network on Fashion-MNIST end to end: a baseline, training curves, a confusion matrix, the errors it makes, and a saved, reproducible model.
Deep network or gradient boosting?2 hr
Compare a neural network with gradient boosting on an open tabular dataset under the same split and metric, and decide honestly whether the network is worth its cost.
Fine-tune a small transformer2 hr
Adapt a small pretrained transformer to a text classification task on open data, compare it with a model trained from scratch, and measure what the pretraining bought.
Train a tiny language model2 hr
Train a character-level transformer on public-domain text with next-token prediction, watch its samples improve during training, and measure loss on held-out text.
Other people teach this subject well too. We chose these to go with this course. They open on other sites.See all community picks
Course · 1 h 16 minDeepLearning.AI · Josh Starmer
Why we recommend itJosh Starmer explains attention one step at a time, with simple pictures. Then you write it in PyTorch. Take it when you know basic Python and what a matrix is.
ChannelYouTube · Luis Serrano
Why we recommend itLuis Serrano explains machine learning, the mathematics behind it and how transformers work, with simple pictures and friendly examples. He is the author of Grokking Machine Learning. A good partner to our mathematics and machine learning courses.
Loading the reviews…
Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.
