Cheatsheet

# ===== When to reach for which model =====
# - Tabular, < 100k rows:           Logistic Regression, Random Forest, XGBoost
# - Tabular, > 100k rows:           XGBoost / LightGBM / CatBoost  (still the SOTA for tables)
# - Text classification, < 50k:     TF-IDF + Logistic Regression
# - Text classification, large:     fine-tune a small transformer (DistilBERT, MiniLM)
# - Image classification:           transfer-learn from a pretrained ResNet/ViT
# - Time series (univariate):       Holt-Winters, ARIMA, Prophet
# - Time series (multivariate):     XGBoost on lags, or a small RNN
# - Recommendations:                Two-tower or ALS over implicit interactions
# - Anomaly detection:              Isolation Forest, autoencoders for non-tabular
# - Tiny labelled dataset:          Few-shot LLM, k-NN, or rule-based heuristics

# ===== Which metric to optimise =====
# - Balanced binary classes:        Accuracy, ROC-AUC
# - Imbalanced classes:             PR-AUC, F1 on minority class, recall@k
# - Multiclass:                     Macro-F1, weighted log-loss
# - Regression:                     MAE if outliers matter, RMSE otherwise
# - Ranking / search / recsys:      NDCG@k, MAP@k, MRR
# - Calibration matters?            Add Brier score / reliability diagrams

# ===== Train/test split rules =====
# - i.i.d. data:                    Random split + KFold cross validation
# - Imbalanced classes:             Stratified split, stratified KFold
# - Grouped data (users, devices):  GroupKFold so a user is not in both train and test
# - Time-ordered data:              TimeSeriesSplit, NEVER random — leak guaranteed otherwise
# - Held-out test set:              Lock it before any tuning. Re-use = silent over-fit.

# ===== Feature scaling cheat =====
# - Linear, SVM, KNN, neural nets:  scale (StandardScaler or MinMaxScaler)
# - Tree-based (RF / boosting):     no scaling needed
# - One-hot vs target encoding:     one-hot small-card; target encoding high-card (careful: target leak)

# ===== Common pitfalls =====
# - Data leak:                      using future values, target column in features, fit_transform on full set
# - Class imbalance:                accuracy looks great while you predict majority class
# - Improper CV:                    random KFold over user-grouped or time data
# - Overfitting on the test set:    repeated peeks tune you to that split, not to reality
# - Feature drift:                  monitor input distributions vs. training distribution in prod
# - No baseline:                    always report a dumb baseline (majority class, mean, last value)

# ===== Sklearn skeleton you can paste =====
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.model_selection import StratifiedKFold, cross_val_score

pre = ColumnTransformer([
    ('num', Pipeline([('imp', SimpleImputer(strategy='median')),
                      ('sc',  StandardScaler())]), num_cols),
    ('cat', Pipeline([('imp', SimpleImputer(strategy='most_frequent')),
                      ('ohe', OneHotEncoder(handle_unknown='ignore'))]), cat_cols),
])

model = Pipeline([('prep', pre), ('clf', xgb.XGBClassifier())])
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(model, X, y, cv=cv, scoring='roc_auc', n_jobs=-1)
print(scores.mean(), '+/-', scores.std())

# fit predict score | StandardScaler OneHotEncoder | Pipeline GridSearchCV | train_test_split