Around 3 months ago (I started writing this a fortnight ago btw), I was asked a question in my PIL interview: "How is threading in python different from other languages, like C++?" I did not know, but I hazarded a guess about it being an interpreted language that I quickly abandoned.
I swore on that day that I would look into this (and more), and it took me 3 months to stumble upon it --- in a LinkedIn post. Later, I searched it up to finally answer it.
threading is not really simultaneous¶
Back when I just got admitted into my uni, I was playing around with threading to increase the speed of my scraping script. threading creates multiple python threads, which might seem like they can run in parallel (and faster), but that is not the case.
The reason for this is the Global Interpreter Lock (GIL).
Global Interpreter Lock¶
The Global Interpreter Lock (GIL) is a mutex lock used in CPython (the standard Python implementation). But why does Python use GIL in the first place?
Memory management in Python is usually done using reference counting, where every object keeps track of the references that point to it. If two threads modify these counts concurrently without any sort of protection, the reference counts can be corrupted, leading to memory leaks or dangling pointers.
One could fix this by either having locks for every object, or just have a singular lock across the entire interpreter, which saves on a lot of memory. And that lock is GIL.
This means that at any particular point in time, only one thread would run on the machine. And that begs the question:-
Why the hell did my scraping script get faster?¶
Despite the fact that only one thread can run at a time, threading can still speed up programs in a special case: when the program is I/O bound. These processes spend a major chunk of time waiting for external operations to complete, rather than computing. Since my script is just spamming the requests to the server, it inherently was I/O bound, thus gained from threading.
If my script were doing some heavy calculations instead, I would've barely seen any improvement, because then my script would've been CPU bound, and threads would just be competing for the GIL.
In more depth, a thread in my script sends the request and sits idle waiting for the server's response. During this waiting time, Python releases the GIL, letting another thread send its request. While the script isn't executing concurrently, it is waiting concurrently XD.
True parallelism in Python¶
That begs the question: What if I wanted stuff to be executed concurrently? Turns out the most prescribed way to do this is to use the multiprocessing module. The trick that this module uses is to create separate processes instead of threads. Thus each process gets its own Python interpreter and independent GIL, leveraging multiple CPU cores. Ofcourse, the trade-off here is that each process has its own memory space, which makes sharing data between processes more expensive.
There are other flavours of Python (like IronPython and Jython) that allow you to escape the GIL, by virtue of them running on top of .NET and JVM respectively.
While looking these up, I found that there exists a build of python that disables the GIL that has been introduced since 3.13. it's denoted with a t suffix at the end like in python3.13t. It removes the regular memory allocator (pymalloc) and instead uses a custom integration of mimalloc from Microsoft Research to provide for thread-safe memory allocation.
Quite a few directions I would love to head into after reading this, and hopefully it should help me get back to writing these TILs on a more regular basis :)