TwelveLabs Python SDK

The TwelveLabs Python SDK provides a set of intuitive classes and methods that streamline platform interaction, minimizing the need for boilerplate code.

Note: The examples in this guide show only the required parameters. For the complete guides, see the Search and Analyze videos pages.

Prerequisites

Ensure that the following prerequisites are met before using the SDK:

• Python 3.7 or newer must be installed on your machine.

• To use the platform, you need an API key:

  1. If you don't have an account, sign up for a free account.
  2. Go to the API Keys page.
  3. If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.
  4. Select the Copy icon next to your key to copy it to your clipboard.

• Your video files must meet the following requirements:

  • For this guide: Files up to 4 GB when using publicly accessible URLs or 200 MB for local files
  • Model capabilities: See the complete requirements for Marengo and Pegasus for resolution, aspect ratio, and supported formats.

  For upload size limits and processing modes, see the Upload and processing methods page.

Install the SDK

Install the latest version of the twelvelabs package:

pip install twelvelabs

Initialize the SDK

1. Import the SDK into your application:

   from twelvelabs import TwelveLabs
   

2. Instantiate the SDK client with your API key:

   client = TwelveLabs(api_key="<YOUR_API_KEY>")
   

Use the SDK

To get started with the SDK, follow these basic steps:

1. Create an index.
2. Upload videos.
3. Perform downstream tasks, such as searching or analyzing videos to generate text based on their content.

Create an index

Indexes store and organize your video data, allowing you to group related videos. When you create an index, configure which video understanding models process your videos and what modalities those models analyze.

To create an index, call the client.indexes.create method with the following parameters:

• index_name: The name of the index.
• models: An array of models to enable. Each entry has two fields:
  • model_name: The model to enable. Use "marengo3.0" for search or "pegasus1.2" for text generation.
  • model_options: The modalities to analyze.

index = client.indexes.create(
    index_name="<YOUR_INDEX_NAME>",
    models=[
        {"model_name": "marengo3.0", "model_options": ["visual", "audio"]},
        {"model_name": "pegasus1.2", "model_options": ["visual", "audio"]}
    ]
)
if not index.id:
    raise RuntimeError("Failed to create an index.")
print(f"Created index: id={index.id}")

The client.indexes.create method returns an object that includes, among other information, a field named id representing the unique identifier of your new index.

See the Indexes page for more details.

Upload videos

To upload a video, call the client.assets.create method with the following parameters:

• method: Upload method. Use "url" for publicly accessible URLs or "direct" for local files.
• url or file: The video URL or an opened file object in binary read mode. Use direct links to raw media files. Hosting platform links and cloud storage sharing links are not supported.

asset = client.assets.create(
    method="url",
    url="<YOUR_VIDEO_URL>"
    # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file.
)
print(f"Created asset: id={asset.id}")

The client.assets.create method returns an object that includes, among other information, a field named id representing the unique identifier of your asset. Use this identifier in subsequent steps.

Index your video

To index your video, call the client.indexes.indexed_assets.create method with the following parameters:

• index_id: The unique identifier of your index.
• asset_id: The unique identifier of the asset to index.

indexed_asset = client.indexes.indexed_assets.create(
    index_id=index.id,
    asset_id=asset.id
)
print(f"Created indexed asset: id={indexed_asset.id}")

The client.indexes.indexed_assets.create method returns an object that includes, among other information, a field named id representing the unique identifier of your indexed asset.

Monitor the indexing process

The platform indexes videos asynchronously. To monitor the indexing process, call the client.indexes.indexed_assets.retrieve method with the following parameters:

• index_id: The unique identifier of your index.
• indexed_asset_id: The unique identifier of your indexed asset.

import time

print("Waiting for indexing to complete.")
while True:
    indexed_asset = client.indexes.indexed_assets.retrieve(
        index_id=index.id,
        indexed_asset_id=indexed_asset.id
    )
    print(f"  Status={indexed_asset.status}")
    if indexed_asset.status == "ready":
        print("Indexing complete!")
        break
    elif indexed_asset.status == "failed":
        raise RuntimeError("Indexing failed")
    time.sleep(5)

The client.indexes.indexed_assets.retrieve method returns an object that includes, among other information, a field named status representing the status of the indexing process. Poll this method until status is "ready" before performing downstream tasks.

Perform downstream tasks

The sections below show the most common downstream tasks. See our documentation for the complete list of features the platform provides.

Search

Use natural language, images, or both to find matching video segments. Search operates within a single index.

Text queries

To search using a text query, call the client.search.query method with the following parameters:

• query_text: Natural language query. The maximum length of a query is 500 tokens.
• search_options: Modalities to search. Valid values: "visual", "audio", "transcription" (spoken words). See the Search options page for details.

search_results = client.search.query(
    index_id=index.id,
    query_text="<YOUR_QUERY>",
    search_options=["visual", "audio"]
)
for i, clip in enumerate(search_results):
    print(f"Result {i + 1}: video_id={clip.video_id} rank={clip.rank} start={clip.start}s end={clip.end}s")

The client.search.query method returns an iterable where each item contains, among other information, the following fields:

• video_id: The unique identifier of the matching video.
• rank: The relevance ranking (1 = most relevant).
• start, end: The start and end time of the matching clip, expressed in seconds.

Image queries

To search using an image query, call the client.search.query method with the following parameters:

• query_media_type: Must be "image".
• query_media_file, query_media_url, query_media_files, or query_media_urls: The image or images to use as a query (up to 10 total). Provide at least one of the following:
  • (Optional) query_media_file: An opened file object in binary read mode.
  • (Optional) query_media_url: The publicly accessible URL of your image file.
  • (Optional) query_media_files: A list of opened file objects in binary read mode.
  • (Optional) query_media_urls: A list of publicly accessible URLs.

search_results = client.search.query(
    index_id=index.id,
    query_media_type="image",
    query_media_url="<YOUR_IMAGE_URL>",
    # Or use query_media_file=open("<PATH_TO_IMAGE_FILE>", "rb") for a local file.
    # Or use query_media_urls=["<URL_1>", "<URL_2>"] for multiple URLs.
    # Or use query_media_files=[open("<FILE_1>", "rb"), open("<FILE_2>", "rb")] for multiple local files.
    search_options=["visual"]
)
for i, clip in enumerate(search_results):
    print(f"Result {i + 1}: video_id={clip.video_id} rank={clip.rank} start={clip.start}s end={clip.end}s")

The response is similar to that received when using text queries.

Composed queries

Combine up to 10 images with text to narrow results. For example, provide an image of a car and add "red color" to find only red instances of that vehicle.

To perform a composed query, call the client.search.query method with the following parameters:

• query_media_file, query_media_url, query_media_files, or query_media_urls: The image or images to use as a query (up to 10 total). Provide at least one of the following:
  • (Optional) query_media_file: An opened file object in binary read mode.
  • (Optional) query_media_url: The publicly accessible URL of your image file.
  • (Optional) query_media_files: A list of opened file objects in binary read mode.
  • (Optional) query_media_urls: A list of publicly accessible URLs.
• query_text: Text that refines the image query.

search_results = client.search.query(
    index_id=index.id,
    query_media_type="image",
    query_media_url="<YOUR_IMAGE_URL>",
    # Or use query_media_file=open("<PATH_TO_IMAGE_FILE>", "rb") for a local file.
    # Or use query_media_urls=["<URL_1>", "<URL_2>"] for multiple URLs.
    # Or use query_media_files=[open("<FILE_1>", "rb"), open("<FILE_2>", "rb")] for multiple local files.
    query_text="<YOUR_QUERY>",
    search_options=["visual"]
)
for i, clip in enumerate(search_results):
    print(f"Result {i + 1}: video_id={clip.video_id} rank={clip.rank} start={clip.start}s end={clip.end}s")

The response is similar to that received when using text queries.

Analyze videos

The platform uses a multimodal approach to analyze video content, processing visuals, sounds, spoken words, and on-screen text. Use a custom prompt to generate summaries, extract insights, answer questions, or produce structured output.

Note the following about using these methods:

• The Pegasus model must be enabled for the index.
• Your prompts can be instructive or descriptive, or you can phrase them as questions.
• The maximum length of a prompt is 2,000 tokens.

Streaming responses

Streaming delivers text fragments in real-time. Use it for live transcription or when you need immediate output.

To analyze a video with streaming responses, call the client.analyze_stream method with the following parameters:

• video_id: The unique identifier of the indexed asset.
• prompt: Guides text generation, and it can be instructive, descriptive, or a question. The maximum length is 2,000 tokens.

text_stream = client.analyze_stream(
    video_id=indexed_asset.id,
    prompt="<YOUR_PROMPT>"
)
for text in text_stream:
    if text.event_type == "text_generation":
        print(text.text)

The client.analyze_stream method returns a stream of objects. The event_type field can be "stream_start", "text_generation", or "stream_end".

Non-streaming responses

Non-streaming returns the complete text in a single response. Use it for reports or summaries where you need the full result at once. Call the client.analyze method with the same parameters as client.analyze_stream.

result = client.analyze(
    video_id=indexed_asset.id,
    prompt="<YOUR_PROMPT>"
)
print(result.data)

The client.analyze method returns an object where the data field contains the complete generated text string (up to 4,096 tokens).

Error Handling

The SDK includes a set of exceptions that are mapped to specific HTTP status codes, as shown in the table below:

The following example shows how you can handle specific HTTP errors in your application:

import os
from twelvelabs import TwelveLabs
from twelvelabs.errors import BadRequestError, NotFoundError

client = TwelveLabs(api_key=os.getenv("TWELVELABS_API_KEY"))
try:
    indexes = client.indexes.list()
    print(indexes)
except BadRequestError as e:
    print("Bad request.")
except NotFoundError as e:
    print("Not found.")
except Exception as e:
    print(f"An error occurred: {e}")

Contributing

This repository contains code that has been automatically generated from an OpenAPI specification. We are unable to merge direct code contributions to the SDK because the code generation tool overwrites manual changes with each new release.

To contribute, follow these steps:

1. Open an issue to discuss your proposed changes with our team.
2. If you would like to submit a proof of concept, create a pull request. We will review your pull request, but we cannot merge it.
3. We will transfer any approved changes to the repository where the code generation tool operates.

We welcome contributions to the README file. You can submit pull requests directly.