Project Links

Meta

Author: Clarifai

Requires Python: >=3.9

Classifiers

Topic

Scientific/Engineering :: Artificial Intelligence

Programming Language

Python :: 3
Python :: 3 :: Only
Python :: 3.9
Python :: 3.10
Python :: 3.11
Python :: 3.12
Python :: 3.13
Python :: Implementation :: CPython

License

OSI Approved :: Apache Software License

Operating System

OS Independent

Clarifai Protocol

This is a proprietary protocol used by our runners to communicate with our API. This should be installed as part of our python SDK.

Installation

Install the base package:

pip install clarifai-protocol

To use the auto-annotation feature (requires av for video decoding and shapely for polygon geometry), install the optional extra:

pip install 'clarifai-protocol[auto-annotation]'

The dependencies for this extra are listed in requirements-auto-annotation.txt.

Request Cancellation Support

The protocol now supports request cancellation, allowing models to abort in-flight requests to external inference servers when a user cancels their request.

Features

Request ID Access: Models can access the current request ID via get_request_id()
Abort Callbacks: Models can register a callback that will be invoked when requests are cancelled or when connections are aborted
Thread Safety: All operations are thread-safe and work correctly with concurrent requests
Background Execution: Abort callbacks run in background threads to avoid blocking the protocol

Usage

from clarifai_protocol import get_request_id, register_abort_callback
import requests

class MyModel:
    def __init__(self):
        super().__init__()
        self.sglang_url = "http://localhost:30000"
        # Register abort callback during initialization
        register_abort_callback(self._abort_sglang)
    
    def _abort_sglang(self, req_id: str) -> None:
        """Abort handler called when requests are cancelled."""
        try:
            requests.post(
                f"{self.sglang_url}/abort_request",
                json={"rid": req_id},
                timeout=2.0
            )
        except Exception:
            pass  # Handle exceptions gracefully
    
    def generate(self, prompt: str):
        """Generate text using external inference server."""
        # Get the request ID to pass to the external server
        req_id = get_request_id()
        
        # Use req_id when calling external services
        for token in self._call_sglang(prompt, req_id):
            yield token

API Reference

`get_request_id() -> Optional[str]`

Returns the current request ID, or None if called outside of a request context.

Use this as an identifier when calling external inference servers so that cancellation can properly abort the request.

`register_abort_callback(callback: Callable[[str], None]) -> None`

Should be called once during model initialization
The callback receives the cancelled request's ID as a parameter
The callback runs in a background thread
The callback should be idempotent (safe to call multiple times with the same req_id)
Exceptions in the callback are logged but don't crash the protocol
Triggered when: explicit cancellation (RUNNER_ITEM_CANCELLED) or connection abort (stream done/cancelled)

0.0.62 May 05, 2026

0.0.61 Apr 14, 2026

0.0.61rc2 Apr 08, 2026

0.0.61rc1 Mar 26, 2026

0.0.60 Mar 12, 2026

0.0.60rc6 Mar 06, 2026

0.0.60rc4 Mar 05, 2026

0.0.60rc2 Mar 05, 2026

0.0.60rc1 Mar 04, 2026

0.0.59 Mar 02, 2026

0.0.58rc1 Feb 27, 2026

0.0.31rc1 Aug 27, 2025

0.0.21rc1 Apr 03, 2025

Wheel compatibility matrix

Platform	CPython 3.9	CPython 3.10	CPython 3.11	CPython 3.12	CPython 3.13
macosx_11_0_universal2
manylinux2014_aarch64
manylinux2014_x86_64
manylinux_2_17_aarch64
manylinux_2_17_x86_64
manylinux_2_28_aarch64
manylinux_2_28_x86_64
musllinux_1_2_aarch64
musllinux_1_2_x86_64
win_amd64