Machine Learning Curriculum
Machine Learning is a branch of Artificial Intelligence dedicated at making machines learn from observational data without being explicitly programmed.
Machine learning and AI are not the same. Machine learning is an instrument in the AI symphony — a component of AI. So what is Machine Learning — or ML — exactly? It’s the ability for an algorithm to learn from prior data in order to produce a behavior. ML is teaching machines to make decisions in situations they have never seen.
This curriculum is made to guide you to learn machine learning, recommend tools, and help you to embrace ML lifestyle by suggesting media to follow. I update it regularly to maintain freshness and get rid of outdated content and deprecated tools.
Machine Learning in General
Study this section to understand fundamental concepts and develop intuitions before going any deeper.
A computer program is said to learn from experience
E
with respect to some class of tasksT
and performance measureP
if its performance at tasks inT
, as measured byP
, improves with experienceE
.
- Elements of AI A bunch of easy courses teaching AI and machine learning
- COMS W4995 Applied Machine Learning Spring 2020 videos and slides of an applied ML course from Columbia University.
- Opinionated Machine Learning Course by Fast.ai
- Machine Learning Crash Course with TensorFlow APIs Google's fast-paced, practical introduction to machine learning
- Intro to Machine Learning - Udacity hands on scikit-learn (python) programming learning on core ML concepts
- Machine Learning: Supervised, Unsupervised & Reinforcement - Udacity the 2 instructors are hilarious
- Machine Learning Mastery carefully laid out step-by-step guide to some particular algorithms
- Andrew Ng's Specialization on Coursera recommended for people who want to know the details of ML algorithms under the hood, understand enough maths to be dangerous and do coding assignments in python
- ML Recipes - YouTube Playlist a really nicely designed concrete actionable content for ML introduction
- Machine Learning is Fun Part 1 simple approach to machine learning for non-maths people
- Machine Learning with Python - YouTube Playlist
- Machine Learning Yearning by Andrew Ng
- Landing A Data Job: The Course is an opinionated and practical guideline for people who want to focus on landing a job. For example, they suggest that knowing how decision tree works is already good enough, you don't need to know how all the models work, which is true! But this advice is mostly for tabular data.
- Serverless Machine Learning Build your own Machine Learning Serverless Prediction Service
- Effective MLOps: Model Development A free course by Weights & Biases with certification
- Mathematics for Machine Learning and Data Science Specialization math course by Coursera
Books
- Machine Learning with PyTorch and Scikit-Learn by Sebastian Raschka
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition by Aurélien Géron is the best selling book because it's great.
- https://github.com/fastai/fastbook The fastai book, published as Jupyter Notebooks
- https://www.deeplearningbook.org/ math heavy book by Ian Goodfellow and Yoshua Bengio and Aaron Courville
- https://christophm.github.io/interpretable-ml-book/ A Guide for Making Black Box Models Explainable
- https://themlbook.com/ The Hundred-Page Machine learning book by Andriy Burkov
Reinforcement Learning
Building a machine that senses the environment and then chooses the best policy (action) to do at any given state to maximize its expected long-term scalar reward is the goal of reinforcement learning.
- OpenAI Spinning Up This is an educational resource produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).
- Basic Reinforcement Learning An introduction series to Reinforcement Learning (RL) with comprehensive step-by-step tutorials.
- Advanced Topics: RL 2015 (COMPM050/COMPGI13) by David Silver (the guy behind AlphaGo)
- An Introduction Book by Richard S. Sutton and Andrew G. Barto
- Deep Reinforcement Learning: Pong from Pixels
- Lecture 10: Reinforcement Learning - YouTube
- A Survey Paper
- Deep Reinforcement Learning: A Tutorial - OpenAI
- CS 294: Deep Reinforcement Learning, Spring 2017
Deep Learning
Deep learning is a branch of machine learning where deep artificial neural networks (DNN) — algorithms inspired by the way neurons work in the brain — find patterns in raw data by combining multiple layers of artificial neurons. As the layers increase, so does the neural network’s ability to learn increasingly abstract concepts.
The simplest kind of DNN is a Multilayer Perceptron (MLP).
- THE LITTLE BOOK OF DEEP LEARNING This book is a short introduction to deep learning for readers with a STEM background, originally designed to be read on a phone screen. It is distributed under a non-commercial Creative Commons license and was downloaded close to 250'000 times in the month following its public release.
- Full Stack Deep Learning Learn Production-Level Deep Learning from Top Practitioners
- DeepLearning.ai a bunch of courses taught by Andrew Ng at Coursera; It's the sequel of Machine Learning course at Coursera.
- Intro to Deep Learning with PyTorch a course by Facebook AI on Udacity
- A friendly introduction to Deep Learning and Neural Networks
- A Neural Network Playground Tinker with a simple neural network designed to help you visualize the learning process
- Deep Learning Demystified - Youtube explain inspiration of deep learning from real neurons to artificial neural networks
- Learn TensorFlow and deep learning, without a Ph.D. This 3-hour course (video + slides) offers developers a quick introduction to deep-learning fundamentals, with some TensorFlow thrown into the bargain.
- A Guide to Deep Learning by YN^2 a curated maths guide to Deep Learning
- Practical Deep Learning For Coders Course at Fast.ai taught by Jeremy Howard (Kaggle's #1 competitor 2 years running, and founder of Enlitic)
- Deep learning - Udacity recommended for visual learner who knows some ML, this course provides high level ideas of deep learning, dense intuitive details put in a short amount of time, you will use TensorFlow inside the course
- Deep Learning Summer School, Montreal 2015
- Neural networks class - YouTube Playlist
- http://neuralnetworksanddeeplearning.com/index.html a hands-on online book for deep learning maths intuition, I can say that after you finish this, you will be able to explain deep learning in a fine detail.
- The Neural Network Zoo a bunch of neural network models that you should know about (I know about half of them so don't worry that you don't know many because most of them are not popular or useful in the present)
- Intro to TensorFlow for Deep Learning taught at Udacity
- Primers • AI Here’s a hand-picked selection of articles on AI fundamentals/concepts that cover the entire process of building neural nets to training them to evaluating results. There is also a very detailed Transformer architecture explanation.
- Hugging Face Diffusion Models Course Learn the theory, train the model from scratch, and use it to generate images and audio.
- Deep Learning Fundamentals by Lightning.AI with Sebastian Raschka
Convolutional Neural Networks
DNNs that work with grid data like sound waveforms, images and videos better than ordinary DNNs. They are based on the assumptions that nearby input units are more related than the distant units. They also utilize translation invariance. For example, given an image, it might be useful to detect the same kind of edges everywhere on the image. They are sometimes called convnets or CNNs.
- How Convolutional Neural Networks work - Youtube technical explanation including pooling operations, ReLU, fully connected layer, optimization using gradient descent
- Neural Network that Changes Everything - Computerphile
- A Beginner's Guide To Understanding Convolutional Neural Networks
- Deep Learning for Computer Vision (Andrej Karparthy, OpenAI) this is my most favorite video of convolutional net. Andrej explains convnet in detail answering all the curious questions that one might have. For example, most articles only talk about convolution in grayscale image, but he describe convolution in images with color channels as well. He also talks about the concerns and the assumptions that convnets make. This is a great lecture!
- Understanding Neural Networks Through Deep Visualization explains how to visualize a convnet using various techniques
Recurrent Neural Networks
DNNs that have states. They also understand sequences that vary in length. They are sometimes called RNNs.
- http://karpathy.github.io/2015/05/21/rnn-effectiveness/
- http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/
Best Practices
- Rules of Machine Learning: Best Practices for ML Engineering it's a document by Martin Zinkevich about Google's best practices in machine learning.
- A Recipe for Training Neural Networks Andrej's blogpost about how to train yourself to be a paranoid machine learning engineer.
- How to debug neural networks. Manual.
- An overview of gradient descent optimization algorithms
- Deep Learning Tuning Playbook A playbook for systematically maximizing the performance of deep learning models.
Tools
Libraries and frameworks that are useful for practical machine learning
Frameworks
Machine learning building blocks
- scikit-learn general machine learning library, high level abstraction, geared towards beginners
- TensorFlow; Awesome TensorFlow; computation graph framework built by Google, has nice visualization board, probably the most popular framework nowadays for doing Deep Learning
- Keras: Deep Learning for humans Keras is a deep learning API written in Python, running on top of TensorFlow. It's still king of high level abstraction for deep learning. Update: Keras is now available for TensorFlow, JAX and PyTorch!
- PyTorch Tensors and Dynamic neural networks in Python with strong GPU acceleration. It's commonly used by cutting-edge researchers including OpenAI.
- Lightning The Deep Learning framework to train, deploy, and ship AI products Lightning fast. (Used to be called PyTorch Lightning)
- JAX is Autograd and XLA, brought together for high-performance machine learning research.
- OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.
- Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity.
- Chainer A flexible framework of neural networks for deep learning
- Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning. There is a specific focus on reinforcement learning with several contextual bandit algorithms implemented and the online nature lending to the problem well.
- H2O is an in-memory platform for distributed, scalable machine learning.
- spektral Graph Neural Networks with Keras and Tensorflow 2.
- Ivy is both an ML transpiler and a framework, currently supporting JAX, TensorFlow, PyTorch and Numpy. Ivy unifies all ML frameworks 💥 enabling you not only to write code that can be used with any of these frameworks as the backend, but also to convert 🔄 any function, model or library written in any of them to your preferred framework!
No coding
- Ludwig Ludwig is a toolbox that allows users to train and test deep learning models without the need to write code. It is built on top of TensorFlow.
Gradient Boosting
Models that are used heavily in competitions because of their outstanding generalization performance.
- https://github.com/dmlc/xgboost eXtreme Gradient Boosting
- https://github.com/microsoft/LightGBM lightweight alternative compared to xgboost
- https://github.com/catboost/catboost