Machine Learning Cheatsheet 2026

Linear Regression

When to use: Continuous target variable with approximately linear relationship to features. Good baseline for any regression task. Works best when features are not highly correlated (multicollinearity).

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

# Coefficients reveal feature importance
print(model.coef_)       # feature weights
print(model.intercept_)  # bias term

Regularized variants: Use Ridge (L2) when features are correlated, Lasso (L1) when you want feature selection, ElasticNet for both.

Logistic Regression

When to use: Binary or multiclass classification. Strong baseline for any classification task. Excellent when you need probability estimates and interpretable coefficients. Works well with high-dimensional sparse data (text classification).

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(C=1.0, max_iter=1000)
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
y_prob = model.predict_proba(X_test)  # probability estimates

# C controls regularization strength (smaller = stronger)

Decision Tree

When to use: When interpretability is critical. Handles mixed feature types and missing values natively. No feature scaling needed. Use as a building block for understanding your data before trying ensembles.

from sklearn.tree import DecisionTreeClassifier, export_text

model = DecisionTreeClassifier(max_depth=5, min_samples_leaf=10)
model.fit(X_train, y_train)

# Print the tree rules
print(export_text(model, feature_names=feature_names))

Key limitation: Prone to overfitting without depth constraints. Always set max_depth or min_samples_leaf.

Random Forest

When to use: Excellent general-purpose algorithm for tabular data. Robust to overfitting, handles nonlinear relationships, provides feature importance scores. Great when you need a strong model without much tuning. Parallelizable.

from sklearn.ensemble import RandomForestClassifier
import numpy as np

model = RandomForestClassifier(
    n_estimators=200,
    max_depth=15,
    min_samples_leaf=5,
    max_features='sqrt',
    n_jobs=-1,
    random_state=42
)
model.fit(X_train, y_train)

# Feature importance
importances = model.feature_importances_
sorted_idx = np.argsort(importances)[::-1]
for i in sorted_idx[:10]:
    print(f"{feature_names[i]}: {importances[i]:.4f}")

SVM

When to use: Small to medium datasets with clear margin of separation. Effective in high-dimensional spaces (text classification). Kernel trick enables non-linear decision boundaries. Less effective on noisy data with overlapping classes or datasets with more than 100K samples (slow training).

from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler

# SVM requires feature scaling!
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = SVC(C=1.0, kernel='rbf', probability=True)
model.fit(X_train_scaled, y_train)
y_prob = model.predict_proba(X_test_scaled)

K-Nearest Neighbors

When to use: Small datasets, simple decision boundaries, or as a baseline. Non-parametric with no training phase. Struggles with high-dimensional data (curse of dimensionality) and large datasets (slow predictions). Requires feature scaling.

from sklearn.neighbors import KNeighborsClassifier

model = KNeighborsClassifier(n_neighbors=5, weights='distance')
model.fit(X_train_scaled, y_train)  # must scale features first

# Finding optimal k: try odd values to avoid ties
# Typical range: 3 to sqrt(n)

Gradient Boosting (XGBoost / LightGBM / CatBoost)

When to use: The go-to algorithm for tabular data competitions and production systems. Sequentially builds trees that correct previous errors. Consistently wins Kaggle competitions. XGBoost, LightGBM, and CatBoost are the three major implementations. LightGBM is fastest, CatBoost handles categoricals best, XGBoost is most widely deployed.

# Option A: sklearn API
from sklearn.ensemble import GradientBoostingClassifier

model = GradientBoostingClassifier(
    n_estimators=200, learning_rate=0.1, max_depth=5
)

# Option B: XGBoost (recommended)
import xgboost as xgb

model = xgb.XGBClassifier(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=6,
    subsample=0.8,
    colsample_bytree=0.8,
    eval_metric='logloss',
    early_stopping_rounds=50,
    n_jobs=-1
)
model.fit(X_train, y_train, eval_set=[(X_val, y_val)])

# Option C: LightGBM (fastest)
import lightgbm as lgb

model = lgb.LGBMClassifier(
    n_estimators=1000,
    learning_rate=0.03,
    num_leaves=31,
    subsample=0.8
)

Neural Networks (MLP / Deep Learning)

When to use: Large datasets (100K+ samples), image recognition, natural language processing, speech, and any task where feature engineering is difficult. Requires significant data and compute. For tabular data, gradient boosting usually wins unless you have millions of rows. Use sklearn's MLPClassifier for quick experiments; switch to PyTorch/TensorFlow for production deep learning.

from sklearn.neural_network import MLPClassifier

model = MLPClassifier(
    hidden_layer_sizes=(128, 64, 32),
    activation='relu',
    solver='adam',
    learning_rate_init=0.001,
    early_stopping=True,
    validation_fraction=0.1,
    max_iter=500,
    random_state=42
)
model.fit(X_train_scaled, y_train)  # must scale features first

# For deep learning: use PyTorch or TensorFlow
# import torch.nn as nn
# model = nn.Sequential(nn.Linear(d, 128), nn.ReLU(), ...)

K-Means

When to use: Clusters are roughly spherical and similarly sized. You know (or can estimate) the number of clusters. Fast and scalable to large datasets. Sensitive to initialization (use k-means++) and outliers.

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# Find optimal k using elbow method or silhouette score
scores = []
for k in range(2, 11):
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    labels = km.fit_predict(X_scaled)
    scores.append(silhouette_score(X_scaled, labels))

# Best k = max silhouette score
best_k = scores.index(max(scores)) + 2
model = KMeans(n_clusters=best_k, random_state=42)
clusters = model.fit_predict(X_scaled)

DBSCAN

When to use: Clusters have arbitrary shapes, unknown number of clusters, or data contains outliers/noise. Does not require specifying k in advance. Automatically detects noise points. Struggles with varying density clusters (use HDBSCAN instead).

from sklearn.cluster import DBSCAN
from sklearn.neighbors import NearestNeighbors
import numpy as np

# Find optimal eps using k-distance graph
nn = NearestNeighbors(n_neighbors=5)
nn.fit(X_scaled)
distances, _ = nn.kneighbors(X_scaled)
k_distances = np.sort(distances[:, -1])  # find the "elbow"

model = DBSCAN(eps=0.5, min_samples=5)
labels = model.fit_predict(X_scaled)

n_clusters = len(set(labels) - {-1})
n_noise = (labels == -1).sum()
print(f"Clusters: {n_clusters}, Noise points: {n_noise}")

PCA

When to use: Reduce feature count while preserving maximum variance. Preprocessing step before other algorithms. Visualization (project to 2D/3D). Dealing with multicollinearity. Fast and deterministic. Does not work well when the important structure is non-linear.

from sklearn.decomposition import PCA
import numpy as np

# Keep 95% of variance
pca = PCA(n_components=0.95)
X_reduced = pca.fit_transform(X_scaled)
print(f"Reduced from {X_scaled.shape[1]} to {X_reduced.shape[1]} features")

# Explained variance per component
print(np.cumsum(pca.explained_variance_ratio_))

# For visualization only
pca_2d = PCA(n_components=2)
X_2d = pca_2d.fit_transform(X_scaled)

t-SNE

When to use: Visualizing high-dimensional data in 2D/3D. Preserves local neighborhood structure. Excellent for exploring cluster structure visually. Do not use for preprocessing before ML models (non-deterministic, no inverse transform, does not preserve global distances). Slow on large datasets.

from sklearn.manifold import TSNE

# Apply PCA first for speed if d > 50
pca = PCA(n_components=50)
X_pca = pca.fit_transform(X_scaled)

tsne = TSNE(n_components=2, perplexity=30, random_state=42)
X_2d = tsne.fit_transform(X_pca)

# perplexity: try values between 5-50
# Higher = more global structure, lower = more local detail

UMAP

When to use: Modern alternative to t-SNE that is faster, preserves more global structure, and supports transforming new data points. Works for both visualization and as a preprocessing step. Better at preserving the relative distances between clusters. Requires the umap-learn package.

import umap

reducer = umap.UMAP(
    n_components=2,
    n_neighbors=15,
    min_dist=0.1,
    metric='euclidean',
    random_state=42
)
X_2d = reducer.fit_transform(X_scaled)

# Unlike t-SNE, UMAP can transform new data
X_new_2d = reducer.transform(X_new_scaled)

# n_neighbors: controls local vs global (higher = more global)
# min_dist: how tightly points cluster (lower = tighter)

Which machine learning algorithm should I use?

It depends on your data type and goal. For tabular data classification, start with Gradient Boosting (XGBoost/LightGBM). For regression, try Linear Regression first, then Random Forest. For clustering, use K-Means for spherical clusters or DBSCAN for arbitrary shapes. For images, use Convolutional Neural Networks. For text, use transformer-based models or TF-IDF with Logistic Regression. Use the Algorithm Selector Tool above to get personalized recommendations.

What is the difference between precision and recall?

Precision measures how many of your positive predictions were correct (TP / (TP + FP)). Recall measures how many actual positives you found (TP / (TP + FN)). Use precision when false positives are costly (spam detection). Use recall when false negatives are costly (disease screening). F1 score is the harmonic mean of both, giving a balanced single number.

How do I prevent overfitting in machine learning?

Use cross-validation to detect overfitting early. Apply regularization (L1/L2). Reduce model complexity (fewer features, shallower trees). Increase training data or use data augmentation. Use dropout for neural networks. Apply early stopping during training. Ensure your validation set is representative of production data.

What is data leakage in machine learning?

Data leakage occurs when information from outside the training dataset is used to create the model. Common causes: using test data during preprocessing (fit_transform on the whole dataset instead of fit on train only), including features derived from the target variable, and using future data to predict past events. Always split data before any preprocessing steps and use sklearn Pipelines.

When should I use Random Forest vs Gradient Boosting?

Random Forest trains trees independently in parallel, making it faster and more robust to hyperparameter choices. Gradient Boosting trains trees sequentially, correcting previous errors, and usually achieves higher accuracy but is slower and more prone to overfitting. Start with Random Forest for baselines and switch to Gradient Boosting (XGBoost, LightGBM) when you need maximum performance.