CS229 Machine Learning – Lecture 1: Course Introduction & Overview

Professor Andrew Ng opens CS229 by positioning machine learning as a transformative technology comparable to electricity 100 years ago. He argues that demand for ML skills is growing faster than supply, creating enormous opportunities across industries—from tech giants to healthcare, transportation, logistics, and law. He encourages students to see ML as a tool for meaningful impact, not just financial gain.

• AI/ML is compared to electricity as a general-purpose transformative technology.
• Demand for ML skills is growing rapidly; even non-tech industries (manufacturing, healthcare, law) are adopting ML.
• CS229 has produced multiple generations of ML practitioners who have built major products and startups.
• The speaker identifies personal meaning in using ML to improve healthcare, education, and democracy.

Prof. Ng covers the practical details of CS229. The class meets Mondays and Wednesdays for lectures and has optional Friday discussion sections. All lectures and sections are recorded and available online the same day. The class uses Piazza for discussion and Gradescope for grading. A key logistical change this year is replacing the timed midterm with a take-home midterm, and moving programming assignments from MATLAB/Octave to Python with NumPy.

• Lectures: Monday & Wednesday; Discussion sections: Friday (optional, recorded).
• ~800 students enrolled; lectures broadcast via SCPD.
• Piazza is the recommended platform for questions—typically faster responses than email.
• Take-home midterm replaces timed midterm this year.
• Programming shifts from MATLAB/Octave to Python/NumPy.
• ~60 office-hour slots per week are offered to reduce crowding.
• Four homework assignments and a final project are required.

The course assumes familiarity with basic computer science (Big-O, data structures), probability and statistics (random variables, expectation, variance), and linear algebra (matrices, vectors, matrix multiplication, eigenvectors). Review sessions in the first few weeks will cover prerequisite material. Students are encouraged to form study groups, but homework solutions must be written up individually without copying shared notes.

• Prerequisites: CS fundamentals, probability/statistics, linear algebra, Python.
• First three Friday sections review linear algebra, probability/statistics, and Python/NumPy.
• Students may discuss homework with peers but must write solutions independently.
• CS229 completion is recognized by employers; academic integrity is emphasized to protect that value.

A central component of CS229 is an open-ended machine learning project completed in small groups. Students are encouraged to pick an application area they find exciting and build a working ML system for it. Previous projects span domains from cancer diagnosis to art generation to engineering applications. Groups of one to three are standard; groups of four face a stricter grading rubric.

• Projects done in groups of 1–3 (recommended); groups of 4 held to a higher standard.
• Students should browse previous years' projects on cs229.stanford.edu for inspiration.
• Project proposal due a few weeks into the quarter; final project due at end of quarter.
• TAs are matched to groups by application-area expertise (vision, NLP, robotics, biology, etc.).

Two classical definitions are presented. Arthur Samuel (1959) defined ML as giving computers the ability to learn without being explicitly programmed; his checkers-playing program famously surpassed its own creator. Tom Mitchell defined a Well-Posed Learning Problem more formally: a program learns from experience E with respect to task T measured by performance P, if its performance on T improves with experience E.

• Arthur Samuel's definition: 'the field of study that gives computers the ability to learn without being explicitly programmed.'
• Mitchell's definition ties together Experience (E), Task (T), and Performance measure (P).
• Samuel's checkers program learned through self-play and outperformed Samuel himself—an early proof-of-concept.
• Today, narrow ML tasks (speech recognition, Go) routinely surpass human-level performance.

Supervised learning is the most widely used category of ML. The learner is given labeled training data—pairs of inputs X and outputs Y—and must learn a mapping from X to Y so that it can predict Y for new, unseen inputs. Two main problem types are regression (continuous Y) and classification (discrete Y). Inputs can be multi-dimensional feature vectors; the Support Vector Machine (SVM) can work with effectively infinite-dimensional feature spaces via the kernel trick.

• In supervised learning, both inputs X and labels Y are provided during training.
• Regression: Y is continuous (e.g., house price prediction from square footage).
• Classification: Y is discrete (e.g., tumor malignant vs. benign; multi-class cancer types).
• Real applications use many features (e.g., clump thickness, cell uniformity) that cannot be plotted.
• SVMs with kernels allow working with infinite-dimensional feature representations.
• ALVINN (Carnegie Mellon) used supervised learning with a neural network to steer a car by learning from a human driver's inputs and steering decisions.

A significant portion of CS229 focuses on systematic approaches to building and debugging ML systems. Prof. Ng observes that two teams using the same algorithm can have drastically different outcomes based on their engineering decisions. Rather than relying on 'black magic' or tribal knowledge, the course aims to teach principled strategies—such as diagnosing whether to collect more data, change the model, or adjust hyperparameters—analogous to using a code profiler before optimizing software.

• Skilled ML practitioners are strategic about which improvements to pursue.
• Key decisions include: collecting more/different data, choosing algorithms, allocating compute (GPUs), and architecture choices.
• Running an ML algorithm and debugging its failures requires systematic methods, not intuition alone.
• Prof. Ng authored a free book ('Machine Learning Yearning') to codify these engineering principles.

Deep learning, a subset of ML based on neural networks, is currently advancing very rapidly and is one of the five main topics of CS229. The course covers enough to understand how to train neural networks, while the dedicated course CS230 covers deep learning more narrowly and in greater depth. CS229 situates deep learning within a broader toolkit of ML algorithms.

• Deep learning = a subset of ML using (deep) neural networks.
• CS229 covers deep learning basics; CS230 focuses exclusively on deep learning.
• Deep learning has driven much of the recent wave of ML progress across many domains.

Unsupervised learning uses datasets with no labels—only inputs X—and seeks to discover interesting structure in the data automatically. The most common technique is clustering (e.g., K-means). Applications include grouping news articles, analyzing genetic microarray data, organizing computing clusters, social network community detection, market segmentation, and audio source separation (cocktail party problem via Independent Components Analysis).

• No labels Y are provided; the algorithm finds structure in X alone.
• K-means clustering discovers natural groupings in data.
• Google News clustering groups thousands of articles about the same event.
• The Cocktail Party Problem: separate overlapping audio sources using ICA (Independent Components Analysis).
• Unsupervised learning powers market segmentation, genomic analysis, and word analogy learning from raw text.
• Most recent economic value from ML comes from supervised learning, but unsupervised learning remains important for research and special use cases.

Reinforcement learning (RL) addresses problems where the correct action is unknown in advance. Instead of labeled examples, the algorithm receives reward (positive or negative) signals based on its behavior and learns to maximize cumulative reward over time. RL is analogous to training a dog with praise and correction. Major applications include autonomous helicopter control, robotic locomotion, and game-playing (Atari, AlphaGo).

• No labeled correct actions; the agent learns from reward/penalty signals.
• Analogous to training a dog: reward good behavior, penalize bad behavior.
• Applied to Stanford's autonomous helicopter: reward smooth flight, penalize crashes.
• AlphaGo (Go) and Atari game agents are landmark RL successes.
• RL is increasingly applied to robotics and logistics optimization, not just games.

This opening lecture establishes both the practical and intellectual scope of CS229, grounding it in the extraordinary current demand for ML expertise and the breadth of its applications.

• CS229 covers five major areas: supervised learning, ML strategy/learning theory, deep learning, unsupervised learning, and reinforcement learning.
• Supervised learning is the most widely used and economically impactful form of ML today.
• Systematic, principled engineering—not intuition—is what separates effective ML practitioners.
• ML opportunities exist far beyond tech: healthcare, law, logistics, education, and more are all active frontiers.
• Students should start forming study groups and project teams immediately, and engage actively on Piazza.
• The course is continuously updated to reflect the rapidly accelerating pace of ML progress.