Classification
Decision Tree, Random Forest, confusion matrix — build a spam filter that learns by example.
Gmail's spam filter processes 15 billion emails per day. It doesn't use a list of bad words — it uses a Random Forest (and more). It learns from millions of examples: this email = spam, this = not.
Classification is ML's answer to yes/no questions: spam or not, fraud or not, cancer or not. The stakes are real — a false negative (missed fraud) costs money; a false positive (blocked real email) costs trust.
📈 Key Metrics (5–25 min)
| Metric | Formula | When it matters |
|---|---|---|
| Accuracy | (TP+TN) / total | Balanced classes |
| Precision | TP / (TP+FP) | Cost of false alarms (spam filter) |
| Recall | TP / (TP+FN) | Cost of misses (cancer detection) |
| F1 | 2×P×R / (P+R) | Imbalanced classes |
One decision tree overfits. Random Forest grows 100 trees on random data subsets, then votes. Like asking 100 doctors — majority rules. Much harder to fool. This is ensemble learning.
💻 Live Demo — Spam Filter (25–45 min)
>import numpy as np from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import classification_report, confusion_matrix # Features: [word_count, exclamation_marks, links, caps_ratio, has_unsubscribe] np.random.seed(0) n_spam = 300 n_ham = 700 spam_X = np.column_stack([ np.random.randint(20, 200, n_spam), # word_count np.random.randint(3, 20, n_spam), # exclamation_marks np.random.randint(2, 15, n_spam), # links np.random.uniform(.3, .9, n_spam), # caps_ratio np.random.randint(0, 2, n_spam) # has_unsubscribe ]) ham_X = np.column_stack([ np.random.randint(50, 500, n_ham), np.random.randint(0, 3, n_ham), np.random.randint(0, 3, n_ham), np.random.uniform(.0, .2, n_ham), np.zeros(n_ham, dtype=int) ]) X = np.vstack([spam_X, ham_X]) y = np.array([1]*n_spam + [0]*n_ham) # 1=spam, 0=ham X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) model = RandomForestClassifier(n_estimators=100, random_state=42) model.fit(X_train, y_train) y_pred = model.predict(X_test) print(classification_report(y_test, y_pred, target_names=["ham", "spam"])) print("Confusion matrix:") print(confusion_matrix(y_test, y_pred)) # Feature importance: which signal matters most? features = ["word_count", "exclamations", "links", "caps_ratio", "unsubscribe"] for f, imp in zip(features, model.feature_importances_): print(f" {f:20s}: {imp:.3f}" )
📋 You Code It (45–55 min)
Change n_estimators to 10, 50, 200. Print F1 score for each. Does more trees always help? Also try a single DecisionTreeClassifier — how does its accuracy compare to the forest?
⚡ Day 6 Cheat Sheet
RandomForestClassifier(n_estimators=100)model.fit(X_train, y_train)model.predict(X_test)model.predict_proba(X)classification_report(y, yhat)confusion_matrix(y, yhat)model.feature_importances_cross_val_score(model, X, y)