python thread.error: can’t start new thread

Posted on March 5, 2010. Filed under: Python | Tags: error, Python, thread, ulimit |

The python has thread.error such as:

File "/usr/lib/python2.5/threading.py", line 440, in start _start_new_thread(self.__bootstrap, ()) thread.error: can't start new thread

The “can’t start new thread” error almost certainly due to the fact that you have already have too many threads running within your python process, and due to a resource limit of some kind the request to create a new thread is refused.

You should probably look at the number of threads you’re creating(maybe in the /proc/pid/); the maximum number you will be able to create will be determined by your environment, but it should be in the order of hundreds at least. (Can try ulimit to solve this issue)

It would probably be a good idea to re-think your architecture here; seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request.

Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor.

Reference:

http://stackoverflow.com/questions/1834919/error-cant-start-new-thread

http://adywicaksono.wordpress.com/2007/07/10/i-can-not-create-more-than-255-threads-on-linux-what-is-the-solutions/

http://www.afnog.org/archives/2008-September/004535.html

http://rcsg.rice.edu/rcsg/shared/ulimit.html

http://answers.google.com/answers/threadview/id/311442.html

http://ubuntuforums.org/archive/index.php/t-114071.html