Background Jobs & Tasks

Lexigram provides a unified way to handle long-running or deferred work outside the request-response cycle. Built on top of lexigram-tasks and lexigram-queue, it supports multiple backends (Redis, RabbitMQ, PostgreSQL) and robust retry logic.

---

1. Defining Tasks

Tasks are simple Python functions decorated with @task. They should be idempotent, as they may be retried in case of failure.

from lexigram.tasks import task

@task(queue="emails", retries=3, backoff=2.0)
async def send_welcome_email(user_id: str, email: str) -> None:
    # Service dependencies are automatically injected
    # from the DI container when the worker executes this.
    ...

Task Options

Option	Default	Description
queue	"default"	The name of the queue this task belongs to.
retries	0	Number of times to retry on failure.
backoff	1.0	Exponential backoff multiplier for retries.
timeout	None	Max execution time (seconds) before the task is aborted.
priority	10	Task priority (lower is higher).

---

2. Enqueueing Work

To run a task, you inject the TaskQueueProtocol and call enqueue.

from lexigram.contracts.tasks import TaskQueueProtocol

class EnrollmentService:
    def __init__(self, queue: TaskQueueProtocol) -> None:
        self.queue = queue

    async def enroll_user(self, user: User) -> None:
        # Business logic...

        # Offload the email to the background
        await self.queue.enqueue(
            "send_welcome_email",
            user_id=user.id,
            email=user.email,
        )

---

3. Scheduled Tasks (Cron)

Lexigram supports cron-based scheduling for recurring maintenance or reports.

from lexigram.tasks import scheduled

@scheduled(cron="0 0 * * *")  # Every night at midnight
async def cleanup_expired_sessions() -> None:
    ...

---

4. Workflows & Architecture

Lexigram tasks follow the Distributed Worker pattern.

graph LR
    App[Lexigram App] -- Enqueue --> Broker((Message Broker))
    Broker -- Fetch --> Worker1[Worker Instance 1]
    Broker -- Fetch --> Worker2[Worker Instance 2]
    
    subgraph Infrastructure
        Broker
    end
    
    subgraph Execution Pool
        Worker1
        Worker2
    end

Backends

• Redis: Recommended for high-performance, low-latency task queues.
• PostgreSQL: Best for transactional tasks (ensures task is enqueued only if DB transaction commits).
• RabbitMQ: Best for complex routing requirements.
• In-Memory: Used for local development and testing.

---

5. Configuration

Tasks are configured in the tasks section of your application.yaml.

tasks:
  backend: redis
  redis_url: redis://localhost:6379/0
  workers: 4                 # Total worker processes
  enable_scheduling: true    # Run the scheduler thread
  dlq_enabled: true          # Route failed jobs to Dead Letter Queue

---

[!TIP] Use the Transactional Outbox pattern if you need to ensure a task is enqueued only if your database transaction succeeds. This is built into lexigram-queue and integrates seamlessly with lexigram-tasks.