Celery tasks not works with gevent
- by Novarg
When i use celery + gevent for tasks that uses subprocess module i'm getting following stacktrace:
Traceback (most recent call last):
File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__
return self.run(*args, **kwargs)
File "/home/webapp/admin/webadmin/apps/loggingquarantine/tasks.py", line 107, in release_mail_task
res = call_external_script(popen_obj.communicate)
File "/home/webapp/admin/webadmin/apps/core/helpers.py", line 42, in call_external_script
return func_to_call(*args, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 740, in communicate
return self._communicate(input)
File "/usr/lib64/python2.7/subprocess.py", line 1257, in _communicate
stdout, stderr = self._communicate_with_poll(input)
File "/usr/lib64/python2.7/subprocess.py", line 1287, in _communicate_with_poll
poller = select.poll()
AttributeError: 'module' object has no attribute 'poll'
My manage.py looks following (doing monkeypatch there):
#!/usr/bin/env python
from gevent import monkey
import sys
import os
if __name__ == "__main__":
if not 'celery' in sys.argv:
monkey.patch_all()
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webadmin.settings")
from django.core.management import execute_from_command_line
sys.path.append(".")
execute_from_command_line(sys.argv)
Is there a reason why celery tasks act like it wasn't patched properly?
p.s. strange thing that my local setup on Macos works fine while i getting such exceptions under Centos (all package versions are the same, init and config scripts too)
