Project Links

Meta

Author: Jim Pivarski

Maintainer: Scikit-HEP

Requires Python: >=3.7

Classifiers

Development Status

5 - Production/Stable

Intended Audience

Developers
Information Technology
Science/Research

License

OSI Approved :: BSD License

Operating System

MacOS :: MacOS X
Microsoft :: Windows
POSIX :: Linux
Unix

Programming Language

Python
Python :: 3
Python :: 3 :: Only
Python :: 3.7
Python :: 3.8
Python :: 3.9
Python :: 3.10
Python :: 3.11

Topic

Scientific/Engineering
Scientific/Engineering :: Information Analysis
Scientific/Engineering :: Mathematics
Scientific/Engineering :: Physics
Software Development
Utilities

Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms.

Arrays are dynamically typed, but operations on them are compiled and fast. Their behavior coincides with NumPy when array dimensions are regular and generalizes when they're not.

Motivating example

Given an array of objects with x, y fields and variable-length nested lists like

array = ak.Array([
    [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
    [],
    [{"x": 4.4, "y": {1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}]
])

the following slices out the y values, drops the first element from each inner list, and runs NumPy's np.square function on everything that is left:

output = np.square(array["y", ..., 1:])

The result is

[
    [[], [4], [4, 9]],
    [],
    [[4, 9, 16], [4, 9, 16, 25]]
]

The equivalent using only Python is

output = []
for sublist in array:
    tmp1 = []
    for record in sublist:
        tmp2 = []
        for number in record["y"][1:]:
            tmp2.append(np.square(number))
        tmp1.append(tmp2)
    output.append(tmp1)

Not only is the expression using Awkward Arrays more concise, using idioms familiar from NumPy, but it's much faster and uses less memory.

For a similar problem 10 million times larger than the one above (on a single-threaded 2.2 GHz processor),

the Awkward Array one-liner takes 4.6 seconds to run and uses 2.1 GB of memory,
the equivalent using Python lists and dicts takes 138 seconds to run and uses 22 GB of memory.

Speed and memory factors in the double digits are common because we're replacing Python's dynamically typed, pointer-chasing virtual machine with type-specialized, precompiled routines on contiguous data. (In other words, for the same reasons as NumPy.) Even higher speedups are possible when Awkward Array is paired with Numba.

Our presentation at SciPy 2020 provides a good introduction, showing how to use these arrays in a real analysis.

Installation

Awkward Array can be installed from PyPI using pip:

pip install awkward

You will likely get a precompiled binary (wheel), depending on your operating system and Python version. If not, pip attempts to compile from source (which requires a C++ compiler, make, and CMake).

Awkward Array is also available using conda, which always installs a binary:

conda install -c conda-forge awkward

If you have already added conda-forge as a channel, the -c conda-forge is unnecessary. Adding the channel is recommended because it ensures that all of your packages use compatible versions:

conda config --add channels conda-forge
conda update --all

Getting help

How-to tutorials

Python API reference

C++ API reference

Report bugs, request features, and ask for additional documentation on GitHub Issues.
You can vote for issues by adding a "thumbs up" (👍) using the "smile/pick your reaction" menu on the top-right of the issue. See the prioritized list of open issues.
If you have a "How do I...?" question, start a GitHub Discussion with category "Q&A".
Alternatively, ask about it on StackOverflow with the [awkward-array] tag. Be sure to include tags for any other libraries that you use, such as Pandas or PyTorch.
To ask questions in real time, try the Gitter Scikit-HEP/awkward-array chat room.

2.6.3rc2 Mar 22, 2024

2.5.1rc1 Dec 06, 2023

2.5.0 Nov 16, 2023

2.5.0rc0 Oct 27, 2023

2.0.0rc8 Dec 09, 2022

2.0.0rc7 Dec 08, 2022

2.0.0rc6 Dec 06, 2022

2.0.0rc5 Dec 06, 2022

2.0.0rc4 Nov 19, 2022

0.12.0rc2 Jul 09, 2019

0.12.0rc1 Jul 09, 2019

0.11.1 Jun 17, 2019

0.11.0 Jun 17, 2019

0.11.0rc8 Jun 14, 2019

0.11.0rc7 Jun 14, 2019

0.11.0rc4 Jun 14, 2019

0.11.0rc3 Jun 14, 2019

0.9.0rc3 Apr 12, 2019

0.9.0rc2 Apr 12, 2019

0.9.0rc1 Apr 11, 2019

0.8.0rc13 Jan 29, 2019

0.8.0rc12 Jan 27, 2019

0.8.0rc11 Jan 25, 2019

0.8.0rc10 Jan 25, 2019

0.8.0rc9 Jan 25, 2019

0.8.0rc8 Jan 25, 2019

0.8.0rc7 Jan 25, 2019

0.8.0rc6 Jan 25, 2019

0.8.0rc5 Jan 25, 2019

0.8.0rc4 Jan 25, 2019

0.8.0rc3 Jan 25, 2019

0.8.0rc2 Jan 25, 2019

Wheel compatibility matrix

Platform	CPython 3.7	CPython 3.8	CPython 3.9	CPython 3.10	CPython 3.11	PyPy 3.7 (pp73)	PyPy 3.8 (pp73)	PyPy 3.9 (pp73)
macosx_10_9_universal2
macosx_10_9_x86_64
manylinux2014_aarch64
manylinux2014_x86_64
manylinux_2_17_aarch64
manylinux_2_17_x86_64
musllinux_1_1_aarch64
musllinux_1_1_x86_64
win32
win_amd64