Data Analysis for Engineers/Module 5

Module 5 · Section 7 of 12

Lesson 5.6 - Classification Basics

Target: ~10 min read - 20 min hands-on

Overview

Classification predicts a category instead of a number. Logistic regression, despite the name, is a classification method - it models the probability of belonging to a class, with a decision boundary (typically at probability 0.5) separating predicted classes. We'll classify water samples as safe/unsafe from measured parameters.

Why This Matters (Engineering Context)

Environmental and process-safety standards define numeric thresholds (dissolved oxygen, BOD, particulate, pressure) that produce a pass/fail. A logistic model can learn a combined, probabilistic version of those rules from historical monitoring data - useful across environmental, chemical, and safety engineering.

Code-Along

from sklearn.linear_model import LogisticRegression

np.random.seed(70)
n_samples = 200

dissolved_oxygen = np.random.uniform(2, 9, n_samples)     # mg/L, higher = better
bod = np.random.uniform(1, 12, n_samples)                  # mg/L, lower = better

# "true" rule with noise: safe when a weighted score clears a threshold
safety_score = 1.2 * dissolved_oxygen - 1.0 * bod + np.random.normal(0, 1.5, n_samples)
is_safe = (safety_score > 2).astype(int)                   # bool -> 0/1 label

water_df = pd.DataFrame({"dissolved_oxygen": dissolved_oxygen, "bod": bod, "is_safe": is_safe})
print(water_df["is_safe"].value_counts())                  # class balance

X = water_df[["dissolved_oxygen", "bod"]].values
y_class = water_df["is_safe"].values
# Logistic regression: despite the name it CLASSIFIES - it models P(class = 1)
clf = LogisticRegression().fit(X, y_class)

print(f"\nCoefficients: DO={clf.coef_[0][0]:.3f}, BOD={clf.coef_[0][1]:.3f}")
print(f"Intercept: {clf.intercept_[0]:.3f}")
print(f"Training accuracy: {clf.score(X, y_class):.3f}")    # .score() = accuracy for a classifier

# --- Draw the decision boundary by classifying every point on a fine grid ---
fig, ax = plt.subplots(figsize=(7, 6))
xx, yy = np.meshgrid(np.linspace(2, 9, 200), np.linspace(1, 12, 200))   # grid of (DO, BOD) points
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)        # ravel -> flatten, np.c_ -> stack cols
ax.contourf(xx, yy, Z, alpha=0.2, cmap="RdYlGn")                        # shaded predicted regions
ax.scatter(water_df["dissolved_oxygen"], water_df["bod"], c=water_df["is_safe"],
           cmap="RdYlGn", edgecolor="black", alpha=0.8)
ax.set_xlabel("Dissolved Oxygen (mg/L)"); ax.set_ylabel("BOD (mg/L)")
ax.set_title("Water Safety Classification - Decision Boundary")
plt.tight_layout(); plt.show()

Run it: the decision boundary runs roughly diagonally, with the "safe" region toward high dissolved oxygen and low BOD. Training accuracy lands comfortably above 80% but not perfect, since noise was added to the safety score.

Practice Exercises

  1. Use clf.predict_proba() to find the predicted probability of safety for a sample with DO=6.5, BOD=4.0.
  2. Add a third feature that's pure noise and check whether it changes the model's coefficients or accuracy meaningfully.
  3. Find a DO/BOD combination sitting almost exactly on the decision boundary (probability ~ 0.5) using predict_proba().
# Try the practice exercises here

Knowledge Check

  1. What does logistic regression predict - a category directly, or something else?
  2. What probability threshold is conventionally used to assign a class label?
  3. Why might a logistic model be more informative than a single hard-coded threshold?
Answer key
  1. A probability of belonging to a class; the label comes from thresholding it
  2. 0.5
  3. It gives a continuous probability (useful for risk-based decisions) and combines multiple factors into one estimate

50 / 63 sections · Course home · Join the coaching cohort