iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

End-to-end Workflow

By · Reviewed & last updated · Part of Machine Learning Tutorial

Most ML projects follow the same shape: frame the problem → gather + split data → train a baseline → feature-engineer → tune → evaluate → ship + monitor. Iterate.

A condensed scikit-learn workflow

EXAMPLE
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

# 1) data + split
X, y = load_breast_cancer(return_X_y=True)
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)

# 2) pipeline = preprocessing + model
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf',    LogisticRegression(max_iter=500)),
])

# 3) baseline cross-val
print('CV F1:', cross_val_score(pipe, Xtr, ytr, cv=5, scoring='f1_macro').mean())

# 4) hyperparameter search
grid = GridSearchCV(pipe,
    param_grid={'clf__C': [0.01, 0.1, 1, 10]},
    cv=5, scoring='f1_macro').fit(Xtr, ytr)
print('Best:', grid.best_params_, grid.best_score_)

# 5) test-set evaluation (ONCE — never tune on this)
pred = grid.predict(Xte)
print(classification_report(yte, pred))

Why it matters

Hold the test set untouched until you’ve finished tuning. Tuning on the test set means you’re measuring how well you memorised it, not how well your model generalises.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
# 1) Frame the problem and a metric
# 2) Get + split data (train / val / test)
# 3) Baseline model
# 4) Feature engineering
# 5) Train + cross-validate
# 6) Tune
# 7) Error analysis
# 8) Ship + monitor
Try it Yourself »

Discussion

Loading…