Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I have couple of celery tasks. The crawl_all_sites_tasks submits multiple crawl_each_batch tasks.

The first tasks processes the first for loop - crawl_each_batch but once thats done I get this error -

tasks.py

 from utils import crawl_all_sites
    @app.task(name='webcrawler.crawl_all_sites')
    def crawl_all_sites_task():
        print 'crawling tasks'
        crawl_all_sites()
    from utils import crawl_each_batch
    @app.task(name='webcrawler.crawl_each_sites')
    def crawl_each_batch_task(filename):
        crawl_each_batch(filename)
utils.py
from p_webcrawler_lib.p_crawler import main
files = ['splitfilesaa'
,'splitfilesab'
,'splitfilesac'
def crawl_each_batch(filename):
    print 'utils_crawl_each_batch'
    main('./apps/webcrawler/batch_files/'+filename)
def crawl_all_sites():
    print 'utils_crawl_all_sites'
    from tasks import crawl_each_batch_task
    for each in files:
        #crawl_each_batch(each).delay()
        crawl_each_batch_task(each).delay()
Error:-
[2015-11-18 03:36:52,013: ERROR/MainProcess] Task webcrawler.crawl_all_sites[31c84a68-0171-45ee-93ce-ab3a879dd8a7] raised unexpected: AttributeError("'NoneType' object has no attribute 'delay'",)
Traceback (most recent call last):
  File "/Users/prem/state-dept/refactor/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/Users/prem/state-dept/refactor/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "/Users/prem/state-dept/analytics_refactor/apps/webcrawler/tasks.py", line 21, in crawl_all_sites_task
    crawl_all_sites()
  File "/Users/prem/state-dept/analytics_refactor/apps/webcrawler/utils.py", line 21, in crawl_all_sites
    crawl_each_batch_task(each).delay()
AttributeError: 'NoneType' object has no attribute 'delay'
You should call crawl_each_batch_task.delay(each). This calls the delay method of the Task instance which represents your task.
The way you are doing it, you are calling the task as if it were any regular Python function. So you call .delay() on it's return value, which is None. Hence the exception you get.
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.