pathlib

For most of Python’s life, working with paths meant a scattering of os.path.join, os.path.basename, and os.path.splitext calls — string surgery, essentially, with all the off-by-a-slash bugs that invites. Then pathlib arrived and made a path a real object, with methods and an operator, and quietly retired all of that. New code should default to pathlib; you will still meet the old string-based API in legacy code, but reaching for it in a new pull request now reads as a small red flag.

Path() and the slash operator

A Path is built from a string and joined with / — yes, the division operator. It looks odd for about ten seconds, and then it becomes the most natural thing in the world:

from pathlib import Path

root = Path("/tmp/datarekha")

# Join with / — portable, and it becomes a backslash on Windows automatically.
log_path = root / "logs" / "app.log"
print(log_path)

# Mix in variables freely.
user_id = 42
user_dir = root / "users" / str(user_id) / "profile.json"
print(user_dir)

# Convert to a plain string only when an old API demands one.
print(str(log_path))

/tmp/datarekha/logs/app.log
/tmp/datarekha/users/42/profile.json
/tmp/datarekha/logs/app.log

Path("/a") / "b" / "c" is Path("/a/b/c") — the same result as os.path.join("/a", "b", "c"), with a fraction of the noise, and it reads like the path it builds.

The properties that replace ten os.path functions

A Path exposes its pieces as plain attributes, which is where most of the old os.path zoo disappears:

from pathlib import Path

p = Path("/var/log/datarekha/app.log")

print("name:  ", p.name)      # filename with extension
print("stem:  ", p.stem)      # filename without extension
print("suffix:", p.suffix)    # extension, including the dot
print("parent:", p.parent)    # the containing directory
print("parts: ", p.parts)     # every component, as a tuple

sibling = p.parent / "audit.log"   # a file next to this one
print("sibling:", sibling)

renamed = p.with_suffix(".log.gz")  # swap the extension
print("renamed:", renamed)

moved = p.with_name("debug.log")    # swap the whole filename
print("moved:  ", moved)

name:   app.log
stem:   app
suffix: .log
parent: /var/log/datarekha
parts:  ('/', 'var', 'log', 'datarekha', 'app.log')
sibling: /var/log/datarekha/audit.log
renamed: /var/log/datarekha/app.log.gz
moved:   /var/log/datarekha/debug.log

The split between .name, .stem, and .suffix is worth fixing in your mind — it is the breakdown the rest of the API leans on:

.with_suffix() and .with_name() are the two methods you reach for constantly in build scripts and data pipelines — the classic “for each .csv, write a .parquet next to it.”

Existence and type checks

A Path can also ask the filesystem about itself, and create directories:

from pathlib import Path

Path("/tmp/example.txt").write_text("hi", encoding="utf-8")
Path("/tmp/example_dir").mkdir(exist_ok=True)

p_file = Path("/tmp/example.txt")
p_dir = Path("/tmp/example_dir")
p_missing = Path("/tmp/does_not_exist.txt")

print(p_file.exists(), p_file.is_file(), p_file.is_dir())
print(p_dir.exists(), p_dir.is_file(), p_dir.is_dir())
print(p_missing.exists())

Path("/tmp/nested/deep/dir").mkdir(parents=True, exist_ok=True)
print(Path("/tmp/nested/deep/dir").exists())

True True False
True False True
False
True

Two flags on mkdir do a lot of work: parents=True creates any missing parent directories, and exist_ok=True makes the call idempotent (no error if the directory is already there). You will type .mkdir(parents=True, exist_ok=True) so often it becomes muscle memory.

read_text and write_text — the one-liners

For small text files — configs, a single CSV, a prompt template — you can skip the with open(...) dance entirely:

from pathlib import Path

config_path = Path("/tmp/config.yaml")

config_path.write_text("model: gpt-4\ntemperature: 0.2\n", encoding="utf-8")

contents = config_path.read_text(encoding="utf-8")
print(contents)

# Binary equivalents, when you need raw bytes.
config_path.write_bytes(b"\x00\x01\x02")
print(config_path.read_bytes())

model: gpt-4
temperature: 0.2
b'\x00\x01\x02'

For large files, stay with open() and a with block — .read_text() pulls the whole file into memory exactly like .read() does.

glob and rglob — finding files

This pair is the engine of every “process all the X files under here” script. .glob matches one directory level; .rglob recurses the whole tree:

from pathlib import Path

root = Path("/tmp/myproj")
(root / "src").mkdir(parents=True, exist_ok=True)
(root / "tests").mkdir(parents=True, exist_ok=True)
(root / "setup.py").write_text("# setup", encoding="utf-8")
(root / "src" / "main.py").write_text("# main", encoding="utf-8")
(root / "src" / "utils.py").write_text("# utils", encoding="utf-8")
(root / "src" / "data.json").write_text("{}", encoding="utf-8")
(root / "tests" / "test_main.py").write_text("# test", encoding="utf-8")

# .glob is one level deep (sorted only so the output is stable to show).
print("top-level .py (glob):")
for p in sorted(root.glob("*.py")):
    print(" ", p)

# .rglob recurses the whole tree.
print("all .py anywhere (rglob):")
for p in sorted(root.rglob("*.py")):
    print(" ", p)

top-level .py (glob):
  /tmp/myproj/setup.py
all .py anywhere (rglob):
  /tmp/myproj/setup.py
  /tmp/myproj/src/main.py
  /tmp/myproj/src/utils.py
  /tmp/myproj/tests/test_main.py

The difference is plain in the output: glob("*.py") saw only setup.py at the top level, while rglob("*.py") walked into src/ and tests/ and found everything. rglob("*.py") is exactly glob("**/*.py"), and — the part that matters at scale — it returns a generator, streaming paths as it walks rather than building one giant list first. (We sorted the results purely so the printed order is stable; the raw order is filesystem-dependent.)

Finding files by modification time

A frequent request is “find files changed recently”. A Path gives you .stat().st_mtime — the modification time as a Unix timestamp — so the filter is a one-liner. Here we set fixed timestamps and a fixed cutoff so the result is exact; in real code the cutoff would be time.time() - 7 * 86400 for “the last seven days”:

from pathlib import Path
import os

root = Path("/tmp/myproj")
(root / "src").mkdir(parents=True, exist_ok=True)
(root / "src" / "old.py").write_text("# old", encoding="utf-8")
(root / "src" / "new.py").write_text("# new", encoding="utf-8")

# Stamp explicit modification times (Unix seconds) — no wall clock involved.
os.utime(root / "src" / "old.py", (1_700_000_000, 1_700_000_000))
os.utime(root / "src" / "new.py", (1_900_000_000, 1_900_000_000))

cutoff = 1_800_000_000          # keep only files modified after this instant
recent = [p for p in sorted(root.glob("src/*.py"))
          if p.stat().st_mtime >= cutoff]

print("modified after cutoff:")
for p in recent:
    print(" ", p.name, "->", int(p.stat().st_mtime))

modified after cutoff:
  new.py -> 1900000000

Only new.py cleared the cutoff; old.py, stamped earlier, was filtered out.

resolve(), home(), and the cwd

Path("config.yaml").resolve()       # absolute path, following symlinks
Path("config.yaml").absolute()      # absolute path, NOT following symlinks
Path.cwd()                          # current working directory
Path.home()                         # the user's home directory

.resolve() is the safe default for “give me the one true path to this file” — exactly what you want to print in a log line so whoever is debugging can find the file on disk.

In one breath

• Build paths with Path(...) and join with the / operator — portable and readable.
• Decompose with .name, .stem, .suffix, .parent; rewrite with .with_suffix() / .with_name().
• Inspect with .exists(), .is_file(), .is_dir(); create with .mkdir(parents=True, exist_ok=True).
• .glob is one level, .rglob recurses (a streaming generator); .read_text()/.write_text() are the small-file shortcuts.
• .stat().st_mtime gives modification time for “find recent files” filters.

Practice

What’s next

pathlib finds files; next we look inside them. Regex is the language for finding and extracting patterns within text — the other half of nearly every data-cleaning job.