Data Analysis for Engineers/Module 2

Module 2 · Section 9 of 12

Lesson 2.8 - Time Series Basics

Target: ~9 min read - 20 min hands-on

Overview

Setting a DatetimeIndex unlocks pandas' time-series toolbox: .resample() aggregates data into new time buckets (daily to monthly), and rolling windows (.rolling()) compute a moving statistic - useful for smoothing noisy daily consumption into a clearer trend line.

Why This Matters (Engineering Context)

Smoothing daily energy or process data with a rolling average is standard practice before trend or anomaly analysis - raw daily spikes (a weekend, a shutdown) can obscure the underlying pattern.

Code-Along

# --- Set a DatetimeIndex so pandas' time-series tools become available ---

# Filter to one site, make "date" the index (set_index), and put it in
# chronological order (sort_index). Both resample() and rolling() below need
# a sorted DatetimeIndex.
site_ts = energy_df[energy_df["site"] == "Charlie DC"].set_index("date").sort_index()

# --- resample(): re-bucket a time series into a different frequency ---

# "MS" = Month Start: group all daily rows into calendar months and sum
# energy_kwh within each -> one total per month.
monthly_total = site_ts["energy_kwh"].resample("MS").sum()
print("Monthly totals (first 6 months):")
print(monthly_total.head(6))

# "W" = weekly: same idea, averaging ambient temperature per week.
weekly_ambient = site_ts["ambient_c"].resample("W").mean()
print("\nWeekly mean ambient temp (first 4 weeks):")
print(weekly_ambient.head(4))

# --- rolling(): a moving window that slides one row at a time ---

# window=7: each output value is the mean of that day plus the 6 days before it.
# min_periods=1: the first few days (with fewer than 7 prior) still get a value
# instead of NaN. The effect is to smooth out the weekday/weekend sawtooth.
site_ts["energy_7d_avg"] = site_ts["energy_kwh"].rolling(window=7, min_periods=1).mean()
site_ts[["energy_kwh", "energy_7d_avg"]].head(10)

Run it: monthly_total collapses ~30 daily rows into one per month; energy_7d_avg should visibly smooth the weekday/weekend sawtooth in the raw daily column - you'll chart this directly in Module 4.

Practice Exercises

  1. Resample site_ts["energy_kwh"] to yearly totals (freq="YS") and print the result.
  2. Compute a 30-day rolling maximum of energy, to spot the worst single day within any month-long window.
  3. Compare the monthly totals from .resample("MS").sum() against grouping by dt.to_period("M") and summing (Lesson 2.7) - do they agree for Charlie DC?
# Try the practice exercises here

Knowledge Check

  1. What must a DataFrame have before you can call .resample() on it?
  2. What does a 7-day rolling average do to a noisy daily series?
  3. Which resample frequency string produces monthly totals starting on the 1st?
Answer key
  1. A DatetimeIndex
  2. Smooths short-term day-to-day noise, revealing the underlying trend
  3. "MS" (Month Start)

19 / 63 sections · Course home · Join the coaching cohort