TOSA Tools

A toolkit for working with the Tensor Operator Set Architecture (TOSA) Specification (https://github.com/arm/tosa-specification), providing serialization, reference implementation, and MLIR translation capabilities.

Introduction

TOSA Tools is a composite project that integrates three main components:

• reference_model – a reference implementation of TOSA operators.
• serialization – provides methods to read and write serialized TOSA graphs.
• mlir_translator – implements translators between the TOSA MLIR dialect and serialized representations.

Each subfolder contains its own README with more detailed information.

PyPI package

From v2026.02.0 onward, the tosa-tools Python package is published on PyPI. For most users, the quickest start is:

python -m pip install tosa-tools

This installs the Python APIs and command-line entrypoints from all the components (serialization + reference model + MLIR translator). For a concise list of the Python modules, executables, console scripts and shared libraries installed by the package, see PYTHON_PACKAGE.md.

Prerequisites

Platform support

The following platforms are supported:

For build instructions see README.md for individual components.

System Requirements

External Dependencies

LLVM/MLIR

The TOSA MLIR Translator must be linked against a build of LLVM with the MLIR project enabled. Currently, the TOSA MLIR Translator aims to be compatible with the main branch of LLVM. However, when breaking API changes are introduced in LLVM/MLIR, compatibility may not be guaranteed. We regularly update the tested SHA; see ci/vars/llvm-sha.yml for the current SHA.

Quick Build (POSIX)

Build all projects with default configuration:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build

Selective Build

Use TOSA_ENABLE_PROJECTS to build specific components:

Build only serialization library:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTOSA_ENABLE_PROJECTS=serialization
cmake --build build

Build serialization and reference model:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTOSA_ENABLE_PROJECTS="serialization;reference_model"
cmake --build build

Build MLIR translator:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTOSA_ENABLE_PROJECTS="mlir_translator" \
    -DMLIR_TOSA_OPT=ON \
    -DMLIR_DIR=${LLVM_BUILD}/lib/cmake/mlir \
    -DLLVM_DIR=${LLVM_BUILD}/lib/cmake/llvm
cmake --build build

Build all projects (including MLIR translator):

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DTOSA_ENABLE_PROJECTS=all \
    -DMLIR_TOSA_OPT=ON \
    -DMLIR_DIR=${LLVM_BUILD}/lib/cmake/mlir \
    -DLLVM_DIR=${LLVM_BUILD}/lib/cmake/llvm
cmake --build build

Note: The MLIR translator requires MLIR to be available on your system. It is not enabled by default. LLVM_BUILD must be absolute path to llvm-project build directory.

Debug Build

For debug symbols:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DTOSA_ENABLE_PROJECTS=reference_model
cmake --build build

Build and install with pip

The Python package is built with scikit-build-core, which drives a CMake build under the hood. This means pip install will compile native code and therefore requires a working C/C++ toolchain.

If you just want the prebuilt package, use python -m pip install tosa-tools as shown above. The steps below are for building from source.

Note: Starting with release 2026.05.0, the tooling to build/install the individual component Python packages from subdirectories will be removed. Use the unified tosa-tools package from the repository root (python -m pip install .).

Install default components (serialization + reference model)

This uses the default CMake definitions from pyproject.toml.

python -m venv .venv
source .venv/bin/activate

python -m pip install -U pip

# Build + install from the current checkout
python -m pip install . -v -Cbuild-dir=build/pip-install

Install with MLIR translator enabled

To build the MLIR translator you need a compatible LLVM/MLIR build. The tested LLVM commit is pinned in ci/vars/llvm-sha.yml.

Install tosa-tools with the required CMake config settings:

python -m pip install . -v -Cbuild-dir=build/pip-install \
    --config-settings=cmake.define.TOSA_ENABLE_PROJECTS="serialization;reference_model;mlir_translator" \
    --config-settings=cmake.define.MLIR_TOSA_OPT=ON \
    --config-settings=cmake.define.MLIR_DIR="${LLVM_BUILD}/lib/cmake/mlir" \
    --config-settings=cmake.define.LLVM_DIR="${LLVM_BUILD}/lib/cmake/llvm"

Build a wheel locally (optional)

python -m pip wheel . -w dist -v -Cbuild-dir=build/pip-wheel
python -m pip install --no-deps dist/*.whl

Development Environment

For a consistent setup, use the provided Docker configuration for development. This mirrors the CI environment closely. Please refer to the CI README for more details.

Pre-Commit Checks

Before pushing a commit, pre-commit checks must be run to ensure conformity.

NOTE: All commits pushed to remote must adhere to the structured commit format, please refer to CONTRIBUTING.md for detailed guidelines.

pre-commit install
pre-commit run --all

Versioning

This project uses Calendar Versioning (CalVer) in the format: YYYY.0M.Micro[.Modifier]

• YYYY – 4-digit release year (e.g., 2025)
• 0M – 2-digit Zero-padded release month (e.g., 07 for July)
• Micro – The third and usually final number in the version. Sometimes referred to as the "patch" segment.
• Modifier - Optional text tag used to signify extra information about the build.
  • no tag (empty string) - builds produced for release will have no tag at all.

The main branch is used as the active development branch for the next version.

Backward Compatibility

The TOSA Specification outlines its own backward compatibility guarantees and rules.

This section focuses on the backward compatibility guarantees of this project, which implements the specification.

"Software" refers to tools provided by this project or otherwise explicitly approved external ones. There are no guarantees when using hand-written files or ones generated by un-vetted tools.

• Serialization

  • The Serialization library guarantees the ability to load .tosa flatbuffer files produced by earlier versions of the software, within a major TOSA specification version.
  • TOSA Specification features marked as experimental are an exception to the above and we may remove them and the ability to parse them after a notice period.

• Reference Model & Conformance Tests

  • The Reference Model uses the Serialization library and is able to read flatbuffer files produced by older software.

  Compatibility between versions is determined by what changed in the following three areas: reference model result, compliance rules (defining tolerance), and conformance test content. For details on exact changes please refer to the RELEASE_NOTES.md supplied with each release.

  These changes are applied on a per-operator basis.

  • Reference model result

    • A bug fix to an operator can make it incompatible with conformance tests and numerical tolerances from previous software versions.

  • Compliance rules (Tolerance) With regards to compatibility between reference model results and tolerances, we currently make no guarantees that two versions would be compatible with each other.

    TOSA Specification outlines that changes to the compliance rules are allowed in both directions: tightening and restricting. And as this implementation follows the specification it is possible that a change in tolerance causes incompatibility between two versions.

  • Conformance test content

    • Bug fixes in the CTS can alter generated tests and their values, making new Reference Model implementations potentially incompatible with older Conformance Tests.

• TOSA MLIR Translator

  • The MLIR Translator uses Serialization library and is able to read flatbuffer files produced by older software.
  • MLIR Translator depends on the TOSA MLIR dialect which can change in incompatible ways and there is no backward compatibility guarantee when reading older TOSA MLIR dialect files and/or generated with a version of LLVM that is earlier than the tested version. Refer LLVM/MLIR for more details.

License

The TOSA Tools components are licensed as follows:

• TOSA Serialization Library: Apache-2.0
• TOSA Reference Model: Apache-2.0
• TOSA MLIR Translator: Apache-2.0 with LLVM Exceptions

See individual component directories for detailed license information and third-party project licenses.