Module 5 · Section 8 of 12

Lesson 5.7 - Decision Trees & Random Forests

Target: ~10 min read - 20 min hands-on

Overview

A decision tree splits data using a sequence of yes/no questions on feature values, producing a model that's easy to visualize and explain. A random forest trains many trees on random subsets and averages them - usually more accurate and less prone to overfitting. Both provide feature importance scores.

Why This Matters (Engineering Context)

Predicting equipment failure from sensor readings (vibration, temperature, running hours) is a common predictive-maintenance application in manufacturing and power generation - feature importance tells the maintenance team which sensor to watch most closely.

Code-Along

from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.ensemble import RandomForestClassifier

np.random.seed(80)
n_machines = 300
vibration_mm_s = np.random.uniform(0.5, 12, n_machines)
bearing_temp_c = np.random.uniform(40, 95, n_machines)
running_hours = np.random.uniform(100, 8000, n_machines)

# "true" failure rule: a weighted sum of the three sensors, plus noise, over a threshold
failure_score = (
    0.35 * (vibration_mm_s - 6)
    + 0.06 * (bearing_temp_c - 65)
    + 0.0006 * (running_hours - 4000)
    + np.random.normal(0, 1.0, n_machines)
)
failed = (failure_score > 0.5).astype(int)

equip_df = pd.DataFrame({"vibration_mm_s": vibration_mm_s, "bearing_temp_c": bearing_temp_c,
                         "running_hours": running_hours, "failed": failed})
print(equip_df["failed"].value_counts())

X = equip_df[["vibration_mm_s", "bearing_temp_c", "running_hours"]].values
y_fail = equip_df["failed"].values

# A single decision tree, depth-limited to keep it readable
tree = DecisionTreeClassifier(max_depth=3, random_state=1).fit(X, y_fail)
print(f"\nDecision tree training accuracy: {tree.score(X, y_fail):.3f}")

# A random forest = many trees on random subsets, votes averaged
forest = RandomForestClassifier(n_estimators=200, max_depth=5, random_state=1).fit(X, y_fail)
print(f"Random forest training accuracy: {forest.score(X, y_fail):.3f}")

# feature_importances_: how much each feature contributed to the splits (sums to 1)
feature_names = ["vibration_mm_s", "bearing_temp_c", "running_hours"]
for name, imp in sorted(zip(feature_names, forest.feature_importances_), key=lambda p: -p[1]):
    print(f"  {name:<18} importance = {imp:.3f}")

# plot_tree draws the actual yes/no decision structure of the single tree
fig, ax = plt.subplots(figsize=(12, 6))
plot_tree(tree, feature_names=feature_names, class_names=["OK", "Failed"], filled=True, fontsize=8, ax=ax)
plt.title("Decision Tree (max_depth=3) - Equipment Failure")
plt.tight_layout(); plt.show()

Run it: vibration_mm_s has the largest coefficient in the true rule and comes out as the top-ranked feature. Note that running_hours can outrank bearing_temp_c in importance even though its raw coefficient is smaller - feature importance reflects how much a feature's actual range of values contributes to splitting the data, not just its coefficient. Always sanity-check "importance" against the features' real-world scales.

Practice Exercises

Increase the single tree's max_depth to 6 and compare training accuracy - does it keep improving? What does that say about overfitting risk for deep single trees?
Retrain the forest with n_estimators=10 instead of 200 - does the feature-importance ranking change meaningfully?
Use the fitted forest to predict failure probability (predict_proba) for a new machine with vibration=9.0, temp=80, hours=6000.

# Try the practice exercises here

Knowledge Check

What does a random forest do differently from a single decision tree?
What does "feature importance" tell you about a trained model?
Why might a maintenance team care about feature importance beyond overall accuracy?

Answer key

It trains many trees on random subsets of data and features, then averages - improving accuracy and reducing overfitting
Which input features contributed most to the model's predictions
It tells them which sensor to monitor and invest in, not just that a prediction was made

51 / 63 sections · Course home · Join the coaching cohort