What is the GIL, and how do you run Python tests or I/O work in parallel?

The Global Interpreter Lock (GIL) is a mutex in CPython (the standard interpreter) that allows only one thread at a time to execute Python bytecode. It simplifies memory management (reference counting) and C extensions, but it means threads cannot run Python code on multiple CPU cores simultaneously. The practical rule: threads or asyncio for I/O-bound work, processes for CPU-bound work.

Why threads still help with I/O

Blocking I/O calls (sockets, files, time.sleep, most HTTP clients) release the GIL while waiting. So 20 threads each waiting on an HTTP response overlap their waits nicely:

from concurrent.futures import ThreadPoolExecutor
import requests

urls = [f"https://api.example.com/health/{svc}" for svc in ("orders", "users", "billing")]

def check(url: str) -> tuple[str, int]:
    return url, requests.get(url, timeout=5).status_code

with ThreadPoolExecutor(max_workers=10) as pool:
    for url, status in pool.map(check, urls):
        print(url, status)

A requests.Session is not documented as thread-safe, so share one per thread or use a separate session per worker if you reuse connections.

CPU-bound work needs processes

Hashing large files, parsing huge logs or generating big data sets in pure Python will not speed up with threads. Use ProcessPoolExecutor or multiprocessing, which run separate interpreters, each with its own GIL. The cost is process startup and pickling arguments and results between processes. Libraries like NumPy release the GIL inside their C code, so they can benefit from threads.

asyncio for high-concurrency I/O

import asyncio
import httpx

async def main() -> None:
    async with httpx.AsyncClient(timeout=5) as client:
        responses = await asyncio.gather(
            *(client.get(f"https://api.example.com/orders/{i}") for i in range(100)),
            return_exceptions=True,
        )
    failures = [r for r in responses if isinstance(r, Exception) or r.status_code >= 400]
    print(f"{len(failures)} failures")

asyncio.run(main())

One thread, cooperative multitasking, very low overhead per request. Good for load-ish smoke checks and bulk data setup. Limit concurrency with an asyncio.Semaphore so you do not overwhelm the system under test. Never call blocking functions such as time.sleep or requests.get inside a coroutine.

Running tests in parallel: pytest-xdist

pip install pytest-xdist
pytest -n auto              # one worker process per CPU core
pytest -n 4 --dist loadscope  # group tests by module/class on the same worker

xdist uses processes, so the GIL is irrelevant; the real issues are isolation:

• scope="session" fixtures run once per worker, not once per run. Expensive global setup (seeding a database) needs coordination, for example a file lock, or doing it before pytest starts.
• Shared external state (the same test user, the same rows) causes collisions. Make data unique per worker using the worker_id fixture or the PYTEST_XDIST_WORKER environment variable.
• Tests that depend on execution order will fail randomly.

For Playwright with pytest, xdist gives each worker its own browser; for Selenium, each worker creates its own driver.

The free-threaded build

PEP 703 introduced an optional free-threaded CPython build (no GIL). It was experimental in Python 3.13 and became officially supported, though still optional and not the default, in 3.14. Many C extensions need updates to support it, and single-threaded performance is somewhat lower. For test frameworks today, the standard build plus processes or asyncio remains the safe default; check your dependencies before relying on free-threading.

What interviewers look for

• Accurate GIL definition, scoped to CPython.
• I/O-bound versus CPU-bound and the matching tool.
• pytest-xdist and the isolation problems it introduces (session fixtures per worker, shared data).
• Up-to-date awareness of free-threaded Python, stated carefully.