Terminal, Paths & Text Encoding
Python programs live in the terminal: they read arguments, environment variables, files and streams, and they hand back exit codes. This refresher covers the shell and text-encoding basics that the chapters assume.
You will refresh
- Shell commands, the working directory, and relative vs absolute paths.
- Environment variables and
PATH: how a command likepython3is found. - stdin, stdout and stderr, redirection, pipes, and exit codes.
- Quoting, executables and the shebang.
- Bytes vs text, UTF-8, and line endings.
1. The shell and commands
A shell (zsh, bash, PowerShell) reads a line, splits it into words, and runs the first word as a program with the rest as its arguments. The $ in this book is the prompt; you don't type it.
In Python: those words arrive in sys.argv (Chapter 1) and are parsed by argparse (Chapter 13).
Quick check: in python3 tool.py -v a.txt, what is sys.argv?
['tool.py', '-v', 'a.txt']. The interpreter name itself is not included.
2. The working directory
Every process has a current working directory (cwd). pwd prints it, cd changes it, and ls (dir on Windows) lists it. Relative file names are resolved against the cwd, not against the script's folder.
The second command failed: after cd .. we are back in the starting folder, where there is no data/. The same relative path means different files depending on the cwd.
In Python: os.getcwd(); and to find files next to your script, use Path(__file__).parent / "data.json".
Quick check: a script works with python3 tool.py but fails with python3 sub/tool.py. Why?
It opens a relative path, and the cwd differs. Build the path from Path(__file__).parent instead.
3. Relative vs absolute paths
An absolute path starts at the root (/Users/ana/x.txt, C:\Users\ana\x.txt). A relative path starts from the cwd (data/x.txt). . means "this folder" and .. means "the parent". Windows uses \, everything else uses /. Python accepts / on Windows too.
In Python: pathlib and os.path (Chapter 13). Never glue paths together with +.
Quick check: what does Path("a/b.txt").parent give?
Path('a')
4. Environment variables
An environment variable is a named string that a process inherits from its parent: HOME, PATH, LANG, or your own, like API_KEY. Set it for one command with NAME=value command, or for the session with export NAME=value.
In Python: os.environ (Chapter 13), PYTHONSTARTUP (Chapter 1), PYTHONPATH (Chapter 8). Values are always strings.
Quick check: how do you read an optional variable with a default?
os.environ.get("NAME", "default"). os.environ["NAME"] raises KeyError when it's missing.
5. PATH: how commands are found
PATH is a list of folders, separated by : (; on Windows). When you type python3, the shell searches those folders in order and runs the first match. which python3 (where on Windows) shows which one wins. Activating a venv simply puts .venv/bin at the front of PATH.
In Python: Chapter 15 (venv activation), shutil.which.
Quick check: pip install x succeeded but import x fails. A likely cause?
pip and python come from different PATH entries, i.e. different Pythons. Use python -m pip.
6. stdin, stdout, stderr, redirection and pipes
Every process gets three streams: stdin (input), stdout (normal output) and stderr (errors). The shell can redirect them:
| Syntax | Meaning |
|---|---|
cmd > out.txt | stdout to a file (overwrite) |
cmd >> out.txt | stdout to a file (append) |
cmd 2> err.txt | stderr to a file |
cmd < in.txt | stdin from a file |
a | b | a's stdout becomes b's stdin (a pipe) |
In Python: print(..., file=sys.stderr) (Chapter 13), input() reads stdin (Chapter 9), and logging writes to stderr (Chapter 14).
Quick check: why did done still appear on screen?
Only stdout was redirected to the file. stderr still goes to the terminal.
7. Exit codes
When a program ends, it returns a number: 0 means success, anything else means failure. The shell stores it in $?. a && b runs b only if a succeeded, and a || b only if it failed.
In Python: sys.exit(code); an uncaught exception exits with 1 (Chapter 10); Chapter 16's main() returns its exit code.
Quick check: what exit code does sys.exit("bad config") give?
1. A string argument is printed to stderr, and the exit code is 1.
8. Quoting and spaces
The shell splits on spaces, so a value containing spaces must be quoted. Inside double quotes, $VAR is expanded; inside single quotes, nothing is.
(In the last line, $NAME inside double quotes was expanded by the shell before the command ran, when NAME wasn't set yet, so it became empty.)
In Python: --note "lunch with Ana" in Chapter 16. With subprocess.run, pass a list (["git", "commit", "-m", msg]) so that no quoting is needed.
Quick check: how many arguments does tool add "12.5" 'food & fun' pass?
Three: add, 12.5 and food & fun.
9. Executables, permissions and the shebang
On macOS and Linux, a file runs as a program only if it has the execute permission (chmod +x), and a script's first line (the shebang, #!/usr/bin/env python3) says which interpreter runs it. ./tool.py means "the file in this folder", because the cwd is usually not on PATH.
In Python: Chapter 1 (shebang), Chapter 15 ([project.scripts] makes real commands).
Quick check: ./tool.py gives "permission denied". Fix?
chmod +x tool.py, or run it as python3 tool.py.
10. Bytes vs text
Files, sockets and pipes carry bytes (numbers 0–255). Text is a sequence of characters. An encoding maps between them. In Python, these are two different types: bytes (b"...") and str. You encode a str into bytes, and decode bytes into a str.
In Python: binary mode 'rb'/'wb' (Chapter 9), struct and zlib (Chapters 13–14), and urlopen().read() returns bytes (Chapter 13).
Quick check: what type does open("x", "rb").read() return?
bytes. Text mode ("r") returns str.
11. Encodings and UTF-8
UTF-8 stores every Unicode character in 1–4 bytes, and plain ASCII in exactly 1. It is the standard almost everywhere, and Python's default for source files and, since 3.15, for open(). Decoding with the wrong encoding either fails or produces garbage ("mojibake", like café).
In Python: open(path, encoding="utf-8") (Chapter 9), source encoding (Chapter 1).
Quick check: you see é where é should be. What happened?
UTF-8 bytes were decoded as Latin-1 or cp1252. Open the file with encoding="utf-8".
12. Line endings
Unix ends lines with \n; Windows uses \r\n. In text mode, Python translates both to \n when reading, so your code only ever sees \n. str.splitlines() handles every style. The csv module wants newline="" so it can handle line endings itself.
In Python: file reading (Chapter 9), csv (Chapter 13).
Quick check: why strip lines with line.rstrip("\n") rather than line[:-1]?
The last line of a file may have no newline, and [:-1] would cut off a real character.
Recap: prerequisite feature → where you see it in Python
| Terminal / encoding concept | Python feature | Where in this book |
|---|---|---|
| command arguments | sys.argv, argparse | Ch. 1, 13, 16 |
| working directory | os.getcwd(), relative open() | Ch. 9, 13 |
| paths | pathlib.Path, os.path | Ch. 13 |
| environment variables | os.environ, PYTHONPATH | Ch. 1, 8, 13 |
PATH | venv activation, python -m pip | Ch. 15 |
| stdin/stdout/stderr | input(), print(file=sys.stderr), logging | Ch. 9, 13, 14 |
| exit codes | sys.exit(), main() return value | Ch. 13, 16 |
| shebang + chmod | executable scripts | Ch. 1 |
| bytes vs text | bytes, encode()/decode(), 'rb' | Ch. 9, 14 |
| UTF-8, line endings | encoding="utf-8", newline="" | Ch. 9, 13 |
Exercises
Exercise 1 🟢 easy predict the output
What does python3 -c "import sys; print(len(sys.argv))" a "b c" d print?
Show solution
4: '-c' plus three arguments, because "b c" is one quoted argument. (With -c, sys.argv[0] is '-c'.)
Exercise 2 🟢 easy build it
Write a one-line command that saves the output of python3 -c "print(2**20)" to big.txt, then shows the file.
Show solution
> redirects stdout into the file, and && runs cat only if the first command succeeded.
Exercise 3 🟢 easy explain why
Why should a script print its error messages to stderr rather than stdout?
Show solution
So that they still reach the user when stdout is redirected or piped, and don't get mixed into the data that another program reads.
Exercise 4 🟢 easy predict the output
What are len("ñ") and len("ñ".encode("utf-8"))?
Show solution
1 and 2. One character, which needs two bytes in UTF-8.
Exercise 5 🟡 medium build it
Write count_lines.py, which reads stdin and prints the number of non-empty lines. Test it with printf "a\n\nb\n" | python3 count_lines.py.
Show solution
sys.stdin is a text file object, so you loop over it line by line, just like an opened file.
Exercise 6 🟡 medium fix the bug
This always reads config.json from wherever the user happens to be. Make it read the file next to the script.
Show solution
__file__ is the script's own path. Its parent folder doesn't depend on the cwd.
Exercise 7 🟡 medium build it
Write a script that reads BUDGET from the environment, defaulting to 500, converts it to an int, and exits with code 2 and a stderr message if the value isn't a number.
Show solution
Environment values are always strings, so convert and validate them like any other user input.
Exercise 8 🟡 medium explain why
A colleague's file shows Café in your program. Explain the cause and the fix.
Show solution
The file is UTF-8 (é is the two bytes C3 A9), but it was decoded as Latin-1 or cp1252, which turns each byte into its own character (Ã, ©). Read it with encoding="utf-8". Python 3.15 also defaults to UTF-8, but older versions may not.
Exercise 9 🔴 hard predict the output
What is printed on the screen, and what ends up in out.txt? python3 -c "import sys; print('A'); print('B', file=sys.stderr); sys.exit(1)" > out.txt || echo failed
Show solution
The screen shows B (stderr) and then failed, because the exit code 1 triggered ||. out.txt contains A.
Exercise 10 🔴 hard mini-project
Write fix_encoding.py IN OUT: it reads a file as cp1252, writes it out as UTF-8 with Unix line endings, prints how many lines it converted to stderr, and exits with code 1 if the input file is missing.
Show solution
Decoding uses the file's real encoding, and writing chooses the encoding and line endings you want. Messages go to stderr, and exit codes report success or failure.