zhang_crawl_bio/worker/readme.md

To use this system you would:

1 Start Celery worker with: celery -A worker.celery.app worker --loglevel=info --concurrency=1 2 Send tasks using:

脚本方式

from worker.celery.models import KeywordTaskModel
from worker.celery.tasks import search_keyword_task
task_data = KeywordTaskModel(keyword="Acampe carinata essential oil").model_dump()
search_keyword_task.delay(task_data)

命令行方式

celery -A worker.celery.tasks flower --persistent=True --db=".\output\celery\flower_db"
# 为不同PC启动worker时指定配置
$env:PC_NAME="w1"; celery -A worker.celery.app worker --hostname=$env:PC_NAME@%h
$env:PC_NAME="w2"; celery -A worker.celery.app worker --hostname=$env:PC_NAME@%h
$env:PC_NAME="w3"; celery -A worker.celery.app worker --hostname=$env:PC_NAME@%h
$env:PC_NAME="w4"; celery -A worker.celery.app worker --hostname=$env:PC_NAME@%h
$env:PC_NAME="w5"; celery -A worker.celery.app worker --hostname=$env:PC_NAME@%h



# 单机多worker不同配置（使用不同配置文件）

PC_NAME=worker1_config celery -A worker.celery.app worker --hostname=worker1@%h

PC_NAME=worker2_config celery -A worker.celery.app worker --hostname=worker2@%h