Data Analysis for Engineers/Module 5

Module 5 · Section 5 of 12

Lesson 5.4 - Linear Regression

Target: ~10 min read - 25 min hands-on

Overview

Linear regression fits the best straight-line (or hyperplane) relationship between inputs and an output. sklearn.linear_model.LinearRegression handles the fitting; R^2 tells you what fraction of the output's variance the model explains; RMSE tells you the typical prediction error in the output's own units. We'll build a simple (one predictor) and a multiple (two predictor) regression predicting facility energy demand.

Why This Matters (Engineering Context)

Predicting energy demand from temperature and an activity proxy is a standard planning tool for utilities, campuses, and process plants alike - it informs capacity planning and flags anomalous consumption that doesn't fit the expected pattern.

Code-Along

from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error

# --- Simple regression: one predictor ---
# sklearn wants X as a 2-D array (rows x features); [["avg_temp_c"]] keeps it 2-D
X_simple = demand_df[["avg_temp_c"]].values
y = demand_df["demand_mw"].values                    # target as a 1-D array

model_simple = LinearRegression().fit(X_simple, y)   # .fit() learns the coefficients
y_pred_simple = model_simple.predict(X_simple)       # .predict() applies them
r2_simple = r2_score(y, y_pred_simple)               # fraction of variance explained (0..1)
rmse_simple = mean_squared_error(y, y_pred_simple) ** 0.5   # sqrt(MSE) = typical error, in MW
print("Simple regression (demand ~ temperature):")
print(f"  Coefficient: {model_simple.coef_[0]:.2f} MW per degree C")   # .coef_ = slope(s)
print(f"  Intercept:   {model_simple.intercept_:.1f} MW")
print(f"  R^2:  {r2_simple:.3f}    RMSE: {rmse_simple:.1f} MW")

# --- Multiple regression: two predictors (one row of X per month, two columns) ---
X_multi = demand_df[["avg_temp_c", "activity_index"]].values
model_multi = LinearRegression().fit(X_multi, y)
y_pred_multi = model_multi.predict(X_multi)
r2_multi = r2_score(y, y_pred_multi)
rmse_multi = mean_squared_error(y, y_pred_multi) ** 0.5
print("\nMultiple regression (demand ~ temperature + activity index):")
print(f"  Coefficients: temp={model_multi.coef_[0]:.2f}, activity={model_multi.coef_[1]:.2f}")   # one per feature
print(f"  Intercept:    {model_multi.intercept_:.1f} MW")
print(f"  R^2:  {r2_multi:.3f}    RMSE: {rmse_multi:.1f} MW")

# predicted vs actual: points hug the diagonal when the model is good
fig, ax = plt.subplots(figsize=(6, 6))
ax.scatter(y, y_pred_multi, alpha=0.6, color="steelblue")
lims = [min(y.min(), y_pred_multi.min()), max(y.max(), y_pred_multi.max())]
ax.plot(lims, lims, "r--", label="Perfect prediction")
ax.set_xlabel("Actual Demand (MW)"); ax.set_ylabel("Predicted Demand (MW)")
ax.set_title(f"Multiple Regression: Predicted vs Actual (R^2={r2_multi:.3f})"); ax.legend()
plt.tight_layout(); plt.show()

Run it: since demand_mw was built from avg_temp_c and activity_index with known coefficients (45 MW/C and 6.0 MW per activity point) plus noise, the multiple regression's fitted coefficients land close to those values, and R^2 is noticeably higher than the temperature-only model - the second predictor captures real explanatory power the first was missing.

Practice Exercises

  1. Compare the simple model's R^2 to the multiple model's - how much does adding activity_index improve the fit?
  2. Use the fitted multiple regression to predict demand for a month with avg_temp_c=31.5 and activity_index=115.
  3. Standardize both predictors ((x - mean) / std) before fitting and compare the coefficients - why are standardized coefficients more directly comparable?
# Try the practice exercises here

Knowledge Check

  1. What does R^2 measure?
  2. What are the units of RMSE, relative to the output variable?
  3. Why did adding activity_index as a second predictor improve model fit here?
Answer key
  1. The fraction of variance in the output explained by the model (0 to 1)
  2. The same units as the output (MW) - a typical prediction error
  3. Demand was generated using both temperature and activity as real drivers

48 / 63 sections · Course home · Join the coaching cohort