Skip to content

2.1 Surface Air Temperature from Weather Stations

Learning Objectives

After completing this lecture, students should be able to:

  • read weather station data from a CSV file using Python
  • visualize surface air temperature as a time series
  • interpret timestamps correctly as datetime objects
  • identify and handle missing values in observational datasets
  • construct a reusable Python script for climate data visualization

Concept

Weather stations provide continuous measurements of surface air temperature. These observations form the basis of many climate analyses, including studies of seasonal variability, long-term trends, and extreme events.

However, observational datasets often require preprocessing before they can be analyzed. Typical issues include:

  • timestamps stored as text
  • missing values
  • inconsistent column names

In this lecture, we learn a basic workflow for working with observational climate data:

  1. read the dataset
  2. inspect the dataset
  3. visualize the time series
  4. interpret timestamps correctly
  5. handle missing values

This workflow forms the foundation for later analyses throughout the course.

Sample Code

Dataset (CSV files)

Surface air temperature (°F) at:

Original datasets can also be obtained from the Utah Climate Center.

Students are encouraged to download additional weather station data from the Utah Climate Center and apply the same workflow to those datasets.

Python Script

Optional: Running on a Linux server

When running Python on a remote Linux server (e.g., through SSH), plt.show() may not display a figure because the server does not have an interactive graphical display. In this case, use Matplotlib's non-interactive Agg backend by adding the following lines before importing matplotlib.pyplot:

import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

The Agg backend renders figures directly to image files instead of opening an interactive display window. Since the template already saves the figure as a PNG file, simply comment out or remove

plt.show()

when using the Agg backend.

Python Script

Exercise

In this exercise we will gradually improve the script to create a clean time-series visualization.

Step 1. Prepare the Files and Run the Initial Script

Download the following files and place them in the same directory:

  • 2_1-read_temp_initial.py
  • greenfarm_temp.csv

For example:

Chapter_2/
├── 2_1-read_temp_initial.py
└── greenfarm_temp.csv

Run the script:

python 2_1-read_temp_initial.py

The script contains the following code to read the CSV file:

print('----- Read CSV file -----')
fin = pd.read_csv('greenfarm_temp.csv')
print(fin)
print('-------------------------')

Confirm that:

  • the CSV file is found successfully
  • the dataset is printed to the screen
  • no file-path error messages appear
  • a basic figure is produced

If you receive a FileNotFoundError, verify that greenfarm_temp.csv is located in the same directory as 2_1-read_temp_initial.py.

Step 2. Improve the Plot Format

Modify the script to improve the appearance of the figure.

First create a new version of the script:

$ cp 2_1-read_temp_initial.py 2_1-read_temp_step2.py

Revise the x-axis label, y-axis label, and figure title appropriately.

You can also experiment with different plot styles.

Example modification:

ax.plot(date, temp, 'o', markersize=2, color='C1')

ax.set_xlabel('Date')
ax.set_ylabel('Air Temperature [°F]')
ax.set_title('Greenville Farm')

Common Plot Styles

You may try several plotting styles to see how the visualization changes.

Line plot (default)

ax.plot(date, temp)

Marker plot

ax.plot(date, temp, 'o', markersize=2)

Each observation is shown as a point.

Line with markers

ax.plot(date, temp, '-o', markersize=2)

The line connects the data points and markers show each observation.

Dashed line

ax.plot(date, temp, '--')

Change color

ax.plot(date, temp, '-o', color='C1')

Matplotlib automatically provides several default colors (C0, C1, C2, etc.).

Try several combinations and decide which visualization is easiest to interpret.

Step 3. Interpret the Timestamp Correctly

In the previous steps, the time column may still be treated as text.
To properly analyze climate time series, the timestamp should be interpreted as a datetime object.

Create a new version of the script:

$ cp 2_1-read_temp_step2.py 2_1-read_temp_step3.py

Step 3.1 Modify the CSV Reading Section

The original dataset uses long variable names. Renaming the columns simplifies the code and makes the variables easier to use. Update the CSV reading section, then convert the timestamp explicitly.

fin = pd.read_csv(
    'greenfarm_temp.csv',
    delimiter=',', header=0,
    names=['date','temp']
)

fin['date'] = pd.to_datetime(fin['date'])
print(fin.dtypes)

Also modify the date and temp variables to match the column names defined above.

date = fin['date']
temp = xr.DataArray(fin['temp'])

The output should show something similar to:

date    datetime64[ns]
temp           float64

Step 3.2 Format the Time Axis

Add the following import and commands to improve the time axis.
Place these commands after setting the figure title.

import matplotlib.dates as mdates

ax.xaxis.set_major_locator(mdates.YearLocator())
ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y-%m-%d'))
ax.xaxis.set_minor_locator(mdates.MonthLocator())

These commands control the spacing and format of the date labels.

Step 3.3 Improve Label Layout

When many dates appear on the axis, the labels may overlap.
Use the following command to automatically rotate the labels.
Also include grid lines to improve readability.

fig.autofmt_xdate()
ax.grid(True)

Step 4. Handle Common Challenges When Reading CSV Files

Next, try reading a different weather station file: Venice_temp.csv.

Create a new version of the script based on Step 3.

$ cp 2_1-read_temp_step3.py 2_1-read_temp_step4.py

Then update the CSV filename and change the figure title.

fin = pd.read_csv(
    'Venice_temp.csv',
    delimiter=',', header=0, names=['date','temp']
)

ax.set_title('Venice Weather Station')

Run the code.

$ python 2_1-read_temp_step4.py

You may encounter an error similar to the following:

UnicodeDecodeError: 'utf-8' codec cannot decode byte 0xa1 in position 34: invalid start byte

Step 4.1 Fix the Encoding Issue

This is a common problem when reading data files created on different operating systems.

Different systems (Windows, macOS, Linux/Unix) may use different character encoding standards when saving text files. When Python reads a file using the default encoding (utf-8), it may fail if the file was created with a different encoding.

One solution is to specify the encoding when reading the CSV file.

Modify the code as follows:

fin = pd.read_csv(
    'Venice_temp.csv',
    encoding='ISO-8859-1',
    delimiter=',', header=0, names=['date','temp']
)

After adding the encoding option, run the script again.

The file should now load successfully.

Step 4.2 Handle Missing Values

After fixing the encoding issue, the script may run successfully, but the resulting plot may still look unusual.

This occurs because the dataset contains missing values.

If you examine the original CSV file, you will see entries such as:

2012-11-01 23:59:59,M

The character "M" indicates a missing observation.

Python does not automatically recognize "M" as missing data, so it must be specified explicitly.

Add the option na_values=['M'] when reading the file:

fin = pd.read_csv(
    'Venice_temp.csv',
    na_values=['M'],
    encoding='ISO-8859-1',
    delimiter=',', header=0, names=['date','temp']
)

This command converts "M" entries into NaN (Not a Number) values, which Python can handle correctly during analysis and plotting.

Common Data Flags in Weather Station Records

Weather station datasets often use special characters to represent data conditions.

Some common examples include:

  • "M" : Missing value
  • "T" : Trace value (a quantity too small to measure accurately, often used for precipitation)

Handling these flags correctly is an important step in preparing observational climate data for analysis.

Step 5. Final Template Script

After resolving the issues related to plotting, timestamps, encoding, and missing values, we can prepare a clean template script that can be reused for other weather station datasets.

This script includes the following improvements:

  • a configurable input filename (file_csv)
  • automatic handling of missing values ("M" and "T")
  • explicit datetime conversion
  • improved figure formatting
  • automatic output filename based on the Python script name
  • a quick check of missing-value counts

You can download the final template script here:

Why Use a Template?

Creating reusable templates is an important skill in scientific programming.

Instead of rewriting code each time you analyze a new dataset, you can:

  • reuse a tested script
  • change only the input file
  • maintain consistent analysis procedures

This approach improves efficiency, reproducibility, and code readability in climate data analysis.

Key Points

  • Weather station observations are a fundamental source of climate data.
  • Observational datasets often require preprocessing before analysis.
  • Timestamps must be interpreted correctly for time-series analysis.
  • Missing values are common and must be handled carefully.
  • Building reusable scripts improves efficiency and reproducibility in climate data workflows.

Homework

Required

Submit the final figure produced in today's exercise.

Alternatively, you may download a weather station dataset from the Utah Climate Center and submit a time-series plot of air temperature or another variable (see Optional/Advanced below).

Optional / Advanced

Download a weather station dataset from the Utah Climate Center:

https://climate.usu.edu/

Tasks:

  1. Download a dataset from the Utah Climate Center.
  2. Modify your script so that it can read the downloaded dataset.
  3. Create a time-series plot of surface air temperature.
  4. Save the figure generated by your script.

Briefly explain:

  • how you modified the script
  • whether the dataset contains missing values
  • how the missing values were handled.