Needled by Threads · Sep 18, 9:38pm
Recently, the company that I work for made a bold decision in choosing Python to implement a new project. I am exceedingly pleased by this decision, because it means a whole new group of people are going to learn how much more productive that they can be when they use Python. However, I have heard the same questions and concerns repeatedly from some of the people who are learning Python. For the most part, they are easy to answer, but one of them has really bothered me.
People are often surprised to hear that Python has a global interpreter lock that makes it essentially impossible to span a single Python process across multiple processors—even with threads. This fact actually shocked a few people when I told them, and they were very concerned about how well Python would “scale” and how they could take advantage of multiple processors.
The BDFLs Concur!
Even though I have an opinion on the issue, I will go ahead and let a few great minds speak for me before I summarize. First, lets take a look at a recent email to the python-dev mailing list from Guido van Rossum, the BDFL for Python. This email was in response to an email about the possibility of removing the global interpreter lock from the CPython interpreter.
I personally don’t think the threaded programming model as found in Java works all that well; without locks you end up with concurrent modification errors, with locks you get deadlocks and livelocks. A typical programmer has a hard enough time keeping track of a bunch of variables being modified by a single thread; add multiple threads acting simultaneously on the same variables to the mix, and it’s a nightmare.If my hunch is right, I expect that instead of writing massively parallel applications, we will continue to write single-threaded applications that are tied together at the process level rather than at the thread level…. I expect that most problems (even most problems that we will be programming 10-20 years from now) get little benefit out of MP.
Lets also take a look at an email that went out a few months ago to the sqlite-users mailing list from D. Richard Hipp, the primary author of SQLite. This email was in response to a query about using an SQLite connection across multiple threads.
This seems like a good opportunity to repeat my oft-ignored advice to not use more than one thread in a single address space. If you need multiple threads, create multiple processes. This has nothing to do with SQLite – it is just good programming advice. I have worked on countless multi-threaded programs over the years, and I have yet to see a single one that didn’t contain subtle, hard to reproduce, and very hard to troubleshoot bugs related to threading issues.I am constantly amazed at the prevailing idea (exemplified by Java) that software should be strongly typed and should not use goto statement or pointers – all in the name of reducing bugs – but that it is OK to use multiple threads within the same address space. Strong typing helps prevent only bugs that are trivially easy to locate and fix. The use of goto statements and pointers likewise results in deterministic problems that are easy to test for and relatively easy to track down and correct. But threading bugs tend to manifest themselves as timing-dependent glitches and lock-ups that are hardware and platform dependent, that never happen the same way twice, and that only appear for customers after deployment and never in a testing environment.
These two very smart guys summarize exactly why I don’t fret about the global interpreter lock in Python. Sure, Python isn’t well-suited to threading or massively parallel applications. Big deal. Again, people focus on the wrong thing.
Python at Present
So if you shouldn’t use threads, what should you use? Well, Guido said it himself: “write single-threaded applications that are tied together at the process level rather than at the thread level.” Not only will this work very well, it will keep your code simpler, and let you break down problems into smaller chunks. Thats a big deal when working on projects of any size, and multiple process solutions have proven themselves to be effective for years.
In addition to this, there are some great ways to get creative in many places where threads are typically (ab)used. For example, threads are often used to handle blocking IO operations, especially in network applications. Python provides several solutions to this problem, all of which I like much better than threads. The asyncore module has shipped as a standard part of the Python distribution for years, and before that it was available as a part of the medusa project. Asyncore is a fabulous way to do asynchronous IO, and I have personally used it with great success with everything from sockets, to files, to spread mailboxes.
The twisted project has also been around for quite some time, and while I am not personally a big fan, many people find it to be a great way to write event-driven network applications in Python. There is even a new book on Twisted from O’Reilly written by Abe Fettig who is another really smart guy in the Python community.
Python in the Future
While Python is fine at present, some recent developments have made the future even brighter. Guido van Rossum and Phillip J. Eby submitted a Python Enhancement Proposal (PEP) earlier this year entitled Coroutines via Enhanced Generators. This PEP proposes making a few enhancements to Python in order to provide the capability of using Python generators usable as simple coroutines. While more innocuous feature additions like decorators get all the attention, I think PEP 342 has far more potential to change the way I program.
The PEP has been accepted, and the functionality will be included in the forthcoming Python 2.5 release. Once one implements a simple coroutine scheduler (or “trampoline”), it is possible to write highly efficient, non-blocking, single threaded applications that are very easy to read and understand, without having to use the forked stackless python implementation.
Threads Schmeds…
So, don’t take my word for it, listen to all these other smart guys. Let all your Java and C# pals have the headaches of threads. Join me and kick it old school with multiple processes, coroutines, and asynchronous IO. Sure, threads have their place, but a world without them is full of possibilities.
Comment
- This seems like another “we’re protecting you from yourself” argument combined (another 10 points) with “this isn’t a bug, it’s a feature!”
It’s possible to write bad multi-threaded apps, and I’ve seen my share of them. But I’ve seen more than enough poorly written single-threaded apps as well. I agree that async routines are what we really want, but you need good multithread support if you’re going to write your own. This is especially true when you consider that most available third party APIs don’t expose an async interface.
The assertion that the current set of HPSA apps don’t benefit from multi-processor support is nuts. Orient your app along a work queue / work item model, put nearly all of the synchronization and locking code in the queue, spawn as many worker threads as CPUs, and you will see the app scale.
— Moxie 1160 days ago # - Its neither argument! Python supports threading, and you can write perfectly good threaded applications with Python. So, from that perspective, Python isn’t protecting you from threads at all. You can get yourself into thread locking hell a lot easier in Python, for example, than in a language like Java, since Python’s thread implementation exposes a lot more low-level functionality of threads.
And sure, Python’s threading implementation is held back a bit by the GIL, but the GIL is most certainly not a bug — its a tradeoff. The GIL could be removed from the CPython interpreter, but it would take a lot of work, would make it much more difficult to write extension modules, and wouldn’t really buy you that much. The GIL was put in place to keep the CPython interpreter simple, and to make it much much easier to write extension modules in C (which does a lot more for performance than threads in many cases).
My point being this: while you can write threaded applications in Python, you can often solve the same class of problems better by using multiple processes. Thats not to say that there is no place for threads — I just happen to think that they are highly overused. Also, Guido wasn't saying that most problems won't benebit from multiple processors _in general_. I believe he was trying to say that most problems won't benefit from massively threaded implementations that span multiple processors in a single address space. The product I work on on a daily basis in Python definitely benefits from multiple processors, but it does it entirely based upon multiple processes working off of work queues (surprise, surprise...).
— Jonathan LaCour 1159 days ago # - What about up and coming multicore processors? Will the GIL be a feature or a bug then?
— po84 1158 days ago # - The GIL is neither a feature or a bug — its a design decision. I have never called it a feature, and I have never called it a bug. I have merely explained what it is, and the tradeoffs associated with it: easy to write extension modules, hard to scale a single process across multiple processors.
The solution of multi-processing instead of multi-threading should scale just as well on processors with multiple cores as it scales with multiple processors with single cores, regardless of the GIL.
I still fail to see why people get so focused on the GIL and are so excited about threading.
— Jonathan LaCour 1158 days ago # - I am currently developing on a multi-threaded Python app. In addition to the fact that the GIL makes multiple processors not very useful, the GIL also makes it hard to control thread priority. We have had to come up with some hacks to give the foreground thread priority.
The thread-vs-process debate is silly, IMHO. If the various execution paths (or whatever the neutral term is for thread or process) need to coordinate with one another a great deal, then it is extremely difficult to do that robustly with processes. If, on the other hand, the execution paths do not interact with one another very much, then you can get away with using processes, and avoid the difficulties associated with threads.
If there are shared data structures that the multiple execution paths need to access, then there will need to be a locking mechanism even if the execution paths are in multiple processes, so the change of deadlock still exists.
Multi-threading may be somewhat error-prone, but it is the best alternative available for situation in which concurrency is desirable, and amount of coordination needed is high.
— Alec Wysoker 1157 days ago # - Read it and weep, GIL lovers.
Why Events Are A Bad Idea http://www.usenix.org/events/hotos03/tech/vonbehren.html
— John Mudd 1157 days ago # - John — I am not sure how much I should weep over one paper that expresses one opinion on an issue. For every paper like that, I am sure I could dig up another paper that says the opposite.
— Jonathan LaCour 1157 days ago # - The king, Guido, is definitely not wearing any clothes IMO. The only thing more disturbing are the weak minded coders who follow him blindly.
The only programmers I hear in favor of the GIL getting in the way of the underlying thread library are coders that simply don’t have the brain power for multiple threads. If that’s a valid reason to hobble the rest of us then we might as well outlaw all software development in order to cater to those who can’t get a simple “Hello, World” program to run on their first try.
The GIL gets in my way, exactly what Python is NOT supposed to do. That’s why I call it a design defect. Either Python will be fixed or it will be replaced: Jython, Prothon, I-Python, Ruby?, etc. Anyone who’ll miss the GIL can always buy themselves a bib and training pants. The first pair should go to Guido.
— John Mudd 1157 days ago # - Alec — thread priority hacks are always just that: hacks. Thread implementations are different on every operating system on the planet, and when and how context switches occur is often unpredictable anyway. This is just the nature of threads.
Your point about coordination requirements with concurrent systems is entirely valid though. Threads have the benefit of shared address space, where processes don’t, so it certainly makes the coordination easier because you can wrap locks around your shared data structures, and hope for the best. With multi-processing you are stuck with some form of IPC, and depending on your needs, this can become quite cumbersome, and even slow.
I don’t think that the situation necessarily has to stay this way though. I imagine that as MP systems become more and more common, Python will have to develop improved support for multi-process concurrency in order to take full advantage of system resources. There are some really smart people working on the development of Python, and I am betting that we see some work on this sooner rather than later. (My money is on Phillip J. Eby — that guy always seems to have something up his sleeve).
Again, threads aren’t necessarily a bad way of doing things, and thats not at all what I am trying to say. I am just pointing out that multi-processing is also a valid way of dealing with similar problems, and that in Python its often a better solution because of the GIL.
— Jonathan LaCour 1157 days ago # - Well, if you want to make the assertion that Guido is weak-minded, you go right ahead. Personally, I am not following Guido blindly here. There are several things recently that he has pushed that I am in total disagreement with him on (interfaces, for one).
Also, one of the most important aspects of software development is humility. Something tells me that even you, with all of your “brain power”, can get confused by threading every once in a while! Tracking down bugs in threaded software can be difficult, no matter how smart you are.
I am not “in favor” of the GIL so much as I am not bothered by it. Sure, CPython may end up getting replaced by IronPython, Jython, or something that comes out of the PyPy project. The result of this may mean that Python’s current threading limitations may go away, and this would certainly be useful.
But, I am not going to cry about the GIL. I have found ways to deal with it that I am pretty happy with. If it bothers you so much, fix it without adding too much complexity to CPython. We would all appreciate the added power.
— Jonathan LaCour 1157 days ago # - Moxie: It isn’t how many bad single threaded apps in question, but how many good multi-threaded apps are there? I have not seen many. I have seen many that seem to work great for years on end, but there are thousands of subtile bugs that bite every few years.
po84:multi-core makes threaded programs worse, not better. In a single core system, it is easy to get the lock, just take it. In a multi-core system you have to shut down the other CPU to get the lock. Then you have the other CPU flush the data out of it’s local cache (which is typically not directly accessable by the other core even though they are on the same chip), before you can do anything. Then the reverse happens a little latter when the other CPU grabs the lock.
Now multi-process doesn’t directly change things. However locks are easy, so you typically don’t realize they are slow. Interprocess communication with sockets is harder, so you pay more attention to overhead, sending only the needed data. (libraries can mask this, but they still force more consideration of what you send)
Going multi-process also solves the problem (which doesn’t apply to python programs, though could apply to the python interpreter) where one thread is accessing memory though a bad pointer that belongs to a different thread. These bugs are hard enough to find when your program in single threaded.
— Hank Miller 1157 days ago # - Interesting points, which seem to have been rehashed quite a bit recently.
Note that the GIL is only an issue if your multi-threaded program contains more than a single Python interpreter in it, i.e. you could have multiple computation threads running C code and a single interpreter thread that handles the GUI and some high-level stuff. That’s not an issue at all.
— Martin Blais 1157 days ago # - Here is the reasoning behind the Twisted concurrency model:
Generalization of Deferred Execution in Python
ttp://www.python.org/pycon/papers/deferex/
— Nicola Larosa 1156 days ago # - A lot of the people jumping down the original poster’s throat for having the temerity to question threads would do well to survey the available literature on concurrency. Threads are not the whole story by a long shot.
Much of what the original poster said applies to the computation model underlying threads (Shared State) as much as to any implementation. The idea of multiple processes also corresponds to a computational model. Which model depends on what form of IPC you use; e.g. CSP has quite a lot in common with using a socket with an application protocol.
As a starting point in investigating the breadth of models available, some of which are far more effective and simple (at the same time) than threads/shared state, I suggest CTM, ‘Concepts, Techniques, and Models of Computer Programming’ by Van Roy and Haridi. There are reviews and a link to an old PDF at http://c2.com/cgi/wiki?ConceptsTechniquesAndModelsOfComputerProgramming.
There are reasons why Erlang does concurrency so well, and most of them have nothing whatsoever to do with threads and shared state.
— Vincent D Murphy 1155 days ago # - Give me a multi-process Queue.Queue in the standard library and I’ll never complain about the GIL :)
(and perhaps a shared-memory dict)
— DG 947 days ago # - Processes scale better than threads. Even if Python didn’t have the GIL issue, you’d still be better off using fork(2).
If you develop your application to run across many processes on a single machine, those processes are not going to be very tightly coupled to having to run on the same machine. You will be able to “throw cheap hardware at the problem”.
OTOH if you use threads, you’ve got a lot of work to do in order to distribute them across many machines. OTW you’ve got to buy bigger and more expensive SMP machines to replace older ones.
— Casey Marshall 798 days ago #
commenting closed for this article