File I/O

open() is the one function for reading and writing files in Python, and its defaults are almost always wrong for code you intend to ship — wrong encoding, the wrong mode, the wrong assumption about how big the file is. The good news is that there are only a handful of knobs, and once you set them deliberately you stop chasing a whole category of “weird characters” and “the file disappeared” bugs. Let us turn each knob on purpose.

The modes

The second argument to open(path, mode, encoding=...) is a short string saying what you intend to do with the file:

'r'   read (the default)        — fails if the file does not exist
'w'   write, TRUNCATING first   — creates the file if missing
'a'   append                    — creates the file if missing
'x'   exclusive create          — fails if the file already exists
'b'   binary (combine: 'rb', 'wb')
'+'   read AND write            — rarely what you actually want

The single most expensive mistake hides in that list: opening with 'w' when you meant 'a', which silently empties an existing file before you write a byte. The grid below is worth burning into memory, because the dangerous cell — 'w' on a file that already exists — looks exactly as innocent as the others:

Always use with

A file is an operating-system resource, and forgetting to close it leaks a file handle — let that happen in a loop and your process eventually dies with OSError: too many open files. The with statement is Python’s context-manager syntax, and it removes the chance of forgetting entirely: when the indented block ends — normally or by exception — it calls the file’s close() for you.

# Don't do this — the handle leaks if anything throws between open and close.
f = open("/tmp/notes.txt", "w", encoding="utf-8")
f.write("hello\n")
f.close()

# Do this — close() runs automatically on the way out, success OR exception.
with open("/tmp/notes.txt", "w", encoding="utf-8") as f:
    f.write("hello\n")
    f.write("world\n")

# Read it back.
with open("/tmp/notes.txt", "r", encoding="utf-8") as f:
    contents = f.read()
print(contents)

hello
world

For file I/O, with is not a style preference — it is the correct shape, and anything else is a bug waiting to ship.

Specify the encoding. Always.

Left to itself, open() uses the operating system’s default text encoding — utf-8 on Linux and macOS, but historically cp1252 on Windows. That is how code which “works on my machine” detonates in production with a UnicodeDecodeError the first time it meets an emoji or an accented name. The cure is a single keyword argument, applied every time:

text = "Aarav, Priya, Sofía, café, naïve, 北京\n"

with open("/tmp/names.txt", "w", encoding="utf-8") as f:
    f.write(text)

with open("/tmp/names.txt", "r", encoding="utf-8") as f:
    print(f.read())

Aarav, Priya, Sofía, café, naïve, 北京

Because both the write and the read pinned encoding="utf-8", the accented characters and the Chinese text survive the round trip exactly — on every platform.

Reading: four ways, and the one that scales

f.read()           # the whole file as one string — memory grows with file size
f.read(1024)       # the next 1024 characters
f.readline()       # one line, including its trailing newline
for line in f:     # iterate lines LAZILY — does not load the file
    ...

The last form is the one to internalise, because a file object is an iterator over its own lines. Writing for line in f streams the file a line at a time, which is the difference between code that scales and code that falls over. Watch it on a log file big enough that loading it whole would be reckless:

# First, fabricate a "log" file with 50,000 lines.
with open("/tmp/app.log", "w", encoding="utf-8") as f:
    for i in range(50_000):
        level = "ERROR" if i % 137 == 0 else "INFO"
        f.write(f"2026-05-28 14:{i % 60:02d}:00 {level} request id={i}\n")

# Stream it — count every ERROR line without ever loading the whole file.
error_count = 0
with open("/tmp/app.log", "r", encoding="utf-8") as f:
    for line in f:                       # one line at a time, constant memory
        if " ERROR " in line:
            error_count += 1
            if error_count <= 3:
                print(line.rstrip())     # rstrip drops the trailing newline

print(f"... and {error_count - 3} more")
print(f"total ERROR lines: {error_count}")

2026-05-28 14:00:00 ERROR request id=0
2026-05-28 14:17:00 ERROR request id=137
2026-05-28 14:34:00 ERROR request id=274
... and 362 more
total ERROR lines: 365

Every 137th line was an ERROR, so 365 of the 50,000 lines matched — and the loop found them all while holding only one line in memory at a time. The exact same code works identically on a 50 MB log or a 50 GB log, because the file object streams and the operating system handles the buffering underneath.

Newlines, and why a file might contain \r\n

In text mode (the default), Python normalises line endings for you. A file last edited on Windows may store \r\n on disk, yet for line in f hands you each line ending in a single \n. When you need the raw bytes untouched — say you are hashing a file for an integrity check — open it in binary mode, and Python translates nothing:

with open("config.txt", "rb") as f:     # binary mode = no newline translation
    raw_bytes = f.read()

Writing, buffering, and durability

f.write() does not always reach the disk immediately — Python and the OS buffer writes for speed, and closing the file flushes that buffer. That is yet another reason with is the right shape: the flush happens automatically on exit. But for a long-running writer — a service appending to a log over hours — you may want each line on disk promptly. Opening a text file with buffering=1 selects line buffering, flushing after every newline:

with open("/tmp/service.log", "a", encoding="utf-8", buffering=1) as f:
    for i in range(5):
        f.write(f"event {i}\n")
        # buffering=1 -> flush after every newline, so no explicit f.flush() needed.

with open("/tmp/service.log", "r", encoding="utf-8") as f:
    print(f.read())

event 0
event 1
event 2
event 3
event 4

In one breath

• open(path, mode, encoding="utf-8") — pick the mode on purpose; 'w' truncates an existing file.
• Always wrap open() in a with, so the file is closed (and flushed) on every exit path.
• Always pass encoding="utf-8" — the platform default differs and breaks on non-ASCII text.
• for line in f streams a file lazily in constant memory; .read() loads it whole.
• Text mode normalises newlines to \n; binary mode ('rb') touches nothing.

Practice

What’s next

Real codebases do not pass paths around as bare strings — they use pathlib. One import, and path handling stops being a stringly-typed mess of os.path.join calls.