Standard Library Tour I
Python comes with "batteries included": modules for files, command-line arguments, regular expressions, maths, dates, compression, timing and testing, all available without installing anything.
You will learn
- Working with the operating system and files:
os,shutil,glob,pathlib. - Command-line tools:
sys.argv,argparse,sys.stderr,sys.exit. - Text patterns with
re, and numbers withmath,randomandstatistics. - Dates with
datetime, compression withzlib, timing withtimeit. - Quality control with
doctestandunittest, pluscsvandcollections.
Operating system interface
The os module talks to the operating system: the current folder, environment variables, and file operations. Always write import os, not from os import *: os.open() would hide the built-in open(), which works quite differently.
For everyday file and folder management, shutil has higher-level operations:
💡 Tip: to run other programs, use the subprocess module (subprocess.run(["git", "status"])) rather than os.system(). It is safer and gives you the output and the exit code.
pathlib: the modern way to handle paths
pathlib.Path represents paths as objects. The / operator joins path parts, and many file operations are methods on the path.
File wildcards
glob finds files whose names match a wildcard pattern: * means any characters, ? means one character, and ** means any number of subfolders (with recursive=True).
Command-line arguments
You met sys.argv in Chapter 1. For real tools, argparse parses the arguments for you. It generates --help, checks types, and reports errors.
Running python3 top.py --help prints the usage and a description of every option, generated automatically.
Error output and program termination
sys has three standard streams: sys.stdin (input), sys.stdout (normal output) and sys.stderr (errors and warnings). Writing errors to stderr keeps them visible even when the normal output is redirected to a file (Appendix B).
String pattern matching
The re module provides regular expressions: a small language for describing text patterns. Always write patterns as raw strings (r"…"), so that backslashes reach re unchanged.
| Pattern | Matches |
|---|---|
\d / \w / \s | a digit / a word character (letter, digit, _) / whitespace |
. | any character except a newline |
* + ? {2,4} | 0 or more / 1 or more / 0 or 1 / 2 to 4 times |
^ $ \b | start / end / a word boundary |
[a-z], [^,] | one character from a set / any character not in the set |
( … ), (?P<name> … ) | a capture group / a named group |
⚠️ Pitfall: re.match only matches at the start of the string. re.search looks anywhere, and re.fullmatch requires the whole string to match.
Mathematics
❌ Never use random for passwords or tokens. Its numbers are predictable. Use the secrets module instead: secrets.token_urlsafe(16).
Internet access
urllib.request fetches data from URLs, and smtplib sends email. (Most projects use the third-party requests or httpx package for HTTP; see Chapter 15.)
Dates and times
datetime supports date and time arithmetic and formatting. A date is a calendar day, a datetime is a date plus a time, and a timedelta is a duration.
💡 Tip: store dates as ISO strings ("2026-10-01") in JSON. They sort correctly as text, and fromisoformat() reads them back. For time zones, use zoneinfo.ZoneInfo("Europe/Berlin").
Data compression
zlib, gzip, bz2, lzma, zipfile and tarfile cover the common archive and compression formats.
Performance measurement
timeit times small snippets accurately by running them many times. Here it compares the classic temporary-variable swap with tuple swapping:
From the terminal: python3 -m timeit "sum(range(1000))". For whole programs, the cProfile module shows where the time goes. (The older pure-Python profile module is deprecated in 3.15.)
Quality control
Good software comes with tests. Python ships two testing tools.
doctest: examples in docstrings that are checked
doctest finds REPL-style examples in your docstrings and checks that they still produce the output shown. This book's own examples are verified the same way.
unittest: full test suites
unittest is the xUnit-style framework, like JUnit. Write a class that subclasses TestCase, add methods whose names start with test, and use assert* methods.
☕ Java: unittest is JUnit. Most Python projects use the third-party pytest instead (installed with pip, Chapter 15), which lets you write plain assert statements in plain functions: def test_avg(): assert average([2, 4]) == 3.
Batteries included
A few more modules you'll use constantly:
| Module | For |
|---|---|
json, csv | common data formats |
sqlite3 | a full SQL database in a single file, with no server |
collections | Counter, defaultdict, deque, namedtuple |
pathlib, shutil, tempfile | files and folders |
email, xml.etree, html | messages and markup |
logging, threading, decimal | Chapter 14 |
(io.StringIO wraps a string so that it behaves like a file. Notice that csv handled the comma inside the quoted field, which split(",") can't do.)
More examples
1. Find every Python file larger than N bytes
2. Validating e-mail-like strings with re
3. Days until a deadline
4. A tiny CLI with sub-options
5. Expenses per month with Counter
Recap
os/shutil/glob/pathlibhandle files. Preferpathlibandsubprocessin new code.argparsebuilds real CLIs. Errors go tosys.stderr, andsys.exit(code)sets the exit code.- Use
rewith raw strings (match= at the start,search= anywhere,fullmatch= the whole string). Usemath,random(not for secrets) andstatisticsfor numbers. datetimedoes date arithmetic and ISO formats.zlibcompresses, andtimeit/cProfilemeasure speed.doctestchecks examples andunittestis xUnit.csv,sqlite3,Counteranddefaultdictcome for free.
Exercises
Exercise 1 🟢 easy build it
Use pathlib to print the name, the extension, and the folder name of Path("/home/ana/receipts/oct.pdf").
Show solution
A path is an object with attributes, so there is no string slicing to do.
Exercise 2 🟢 easy predict the output
What does re.findall(r"\d+", "3 apples, 12 pears and 0 kiwis") return?
Show solution
['3', '12', '0']. \d+ matches runs of one or more digits, and findall returns them as strings.
Exercise 3 🟢 easy build it
Print the date that is 100 days after 2026-10-30, in ISO format.
Show solution
date + timedelta gives a new date, and it handles month lengths and leap years for you.
Exercise 4 🟢 easy refactor it
Replace this manual counting with Counter:
Show solution
Counter is a dict subclass made for counting, and it has most_common().
Exercise 5 🟡 medium fix the bug
This regex never matches "Total: 12.50". Why? Fix it.
Show solution
re.match only matches at position 0, and the string starts with "Total". re.search scans the whole string.
Exercise 6 🟡 medium build it · playground
Create playground/cli.py with argparse: a positional amount (float), --category (default misc), and --date (default today, parsed with date.fromisoformat as the type). Print what would be added.
Show solution
type= can be any function that takes a string: date.fromisoformat both validates and converts. Bad input gives a friendly argparse error.
Exercise 7 🟡 medium build it
Write a unittest test case for format_money from Chapter 5. Test a positive amount, a negative amount, and a custom currency.
Show solution
Each test checks one behaviour, so a failure tells you exactly what broke. (argv and exit are set here only so the snippet can run inside other code.)
Exercise 8 🟡 medium explain why
Why should error messages go to sys.stderr, and why should a failing script call sys.exit(1)?
Show solution
When output is redirected (tool > out.txt) or piped into another program, stderr still reaches the user's screen and doesn't pollute the data. A non-zero exit code lets shells, scripts and CI systems detect the failure (if tool; then …). Appendix B covers both.
Exercise 9 🔴 hard build it
Parse log lines like "2026-10-01 09:15 ERROR disk full" with a regex that uses named groups. Count the errors per day with Counter, and ignore lines that don't match.
Show solution
The walrus operator runs the match once, and keeps the result for the filter and the value. Named groups make the code self-documenting.
Exercise 10 🔴 hard mini-project · playground
Create playground/monthly.py. It reads an expenses CSV with the csv module (header date,category,amount,note, where notes may contain commas), groups the totals by month and category with defaultdict, and prints a report sorted by month. Include a doctest for a helper month_of(iso_date).
Show solution
csv.DictReader handles quoted commas and gives each row as a dict. A nested defaultdict avoids all the "is this key there yet?" checks. Opening CSV files with newline="" is what the csv docs recommend.