Python Classification: Build a Classifier with scikit-learn

Your email inbox sorts messages into "spam" or "not spam." A bank flags transactions as "fraudulent" or "legitimate." A doctor classifies a tumor as "benign" or "malignant." These are all classification problems — the model predicts which category something belongs to.

Classification is different from regression. Regression predicts a number (like a price). Classification predicts a label (like "spam" or "not spam"). But the workflow is almost identical: prepare data, split, train, predict, evaluate.

In this tutorial, you'll build classifiers with three popular algorithms — Logistic Regression, Decision Trees, and K-Nearest Neighbors. You'll also learn how to measure classification performance beyond simple accuracy.

The key difference is in the target variable. If the target is a continuous number (price, temperature, salary), it's regression. If the target is a discrete category (spam/not-spam, cat/dog, healthy/sick), it's classification.

Despite the name, logistic regression is a classification algorithm. It works by finding a line (or hyperplane) that separates the classes, then applies a sigmoid function to squish the output between 0 and 1. If the output is above 0.5, it predicts class 1. Otherwise, class 0.

Sometimes you don't just want the label — you want to know how confident the model is. The predict_proba() method returns the probability for each class. A prediction with 99% probability is more trustworthy than one with 51%.

A decision tree works like a game of 20 Questions. It asks a series of yes/no questions about the features ("Is feature 2 greater than 0.5?") and follows different branches based on the answers until it reaches a final prediction.

KNN is the simplest classifier to understand. When a new data point arrives, KNN looks at the K closest points in the training data and takes a majority vote. If 4 out of 5 neighbors are "spam," the new point is classified as "spam."

Accuracy tells you the overall percentage of correct predictions, but it hides important details. A confusion matrix breaks predictions into four categories: true positives, true negatives, false positives, and false negatives.

Think of a medical test. A false negative means telling a sick person they're healthy (dangerous!). A false positive means telling a healthy person they're sick (stressful but less dangerous). The confusion matrix lets you see both types of errors.

There's no single "best" algorithm. The right choice depends on your data and your problem. A quick way to compare is to train several models on the same data and check their test accuracy.