Async flows
Project Links
Meta
Author: Iguazio
Requires Python: >=3.11
Classifiers
Development Status
- 4 - Beta
Intended Audience
- Developers
License
- OSI Approved :: Apache Software License
Operating System
- POSIX :: Linux
- Microsoft :: Windows
- MacOS
Programming Language
- Python :: 3
- Python :: 3.11
- Python
Topic
- Software Development :: Libraries :: Python Modules
- Software Development :: Libraries
Storey
Storey is an asynchronous streaming library for real-time event processing and feature extraction. It is part of the MLRun ecosystem.
Features
- Async event processing with backpressure
- Rich set of built-in transformations: Map, Filter, FlatMap, Batch, Choice, JoinWithTable, and more
- Time-window aggregations with pluggable storage (V3IO, Redis, SQL)
- Sources and targets for files (CSV, Parquet), streams (Kafka, V3IO), and databases (Redis, TimescaleDB)
- Streaming (generator) support in Map steps
Installation
pip install storey
Optional extras:
pip install storey[kafka] # Kafka support
pip install storey[redis] # Redis support
pip install storey[psycopg] # TimescaleDB support
Quick Example
from storey import build_flow, SyncEmitSource, Map, Filter, ParquetTarget
controller = build_flow([
SyncEmitSource(),
Filter(lambda event: event["amount"] > 0),
Map(lambda event: {**event, "amount_cents": int(event["amount"] * 100)}),
ParquetTarget("output.parquet", columns=["user", "amount", "amount_cents"]),
]).run()
controller.emit({"user": "alice", "amount": 9.99})
controller.emit({"user": "bob", "amount": -1.00})
controller.emit({"user": "carol", "amount": 24.50})
controller.terminate()
controller.await_termination()
# output.parquet now contains the two events with positive amounts,
# each enriched with amount_cents.
Documentation
See the MLRun documentation and the storey transformations API reference.
Contributing
See CONTRIBUTING.md for development setup, testing, and coding conventions.
License
Apache License 2.0
1.12.0
Mar 27, 2026
1.11.26
Mar 24, 2026
1.11.25
Mar 23, 2026
1.11.24
Mar 23, 2026
1.11.23
Mar 16, 2026
1.11.22
Mar 12, 2026
1.11.21
Mar 08, 2026
1.11.20
Mar 04, 2026
1.11.19
Mar 03, 2026
1.11.18
Feb 26, 2026
1.11.17
Feb 24, 2026
1.11.16
Feb 24, 2026
1.11.15
Feb 12, 2026
1.11.14
Feb 12, 2026
1.11.13
Feb 10, 2026
1.11.12
Feb 02, 2026
1.11.11
Feb 01, 2026
1.11.10
Feb 01, 2026
1.11.9
Jan 26, 2026
1.11.8
Jan 26, 2026
1.11.7
Jan 18, 2026
1.11.6
Jan 12, 2026
1.11.5
Jan 11, 2026
1.11.4
Dec 22, 2025
1.11.3
Dec 21, 2025
1.11.2
Dec 21, 2025
1.11.1
Dec 17, 2025
1.11.0
Nov 17, 2025
1.10.20
Dec 22, 2025
1.10.19
Dec 21, 2025
1.10.18
Dec 09, 2025
1.10.17
Oct 05, 2025
1.10.16
Sep 29, 2025
1.10.15
Sep 29, 2025
1.10.14
Sep 21, 2025
1.10.13
Sep 08, 2025
1.10.12
Sep 08, 2025
1.10.11
Aug 20, 2025
1.10.10
Aug 07, 2025
1.10.9
Jul 28, 2025
1.10.8
Jul 13, 2025
1.10.7
Jul 01, 2025
1.10.6
Jun 30, 2025
1.10.5
Jun 29, 2025
1.10.4
Jun 19, 2025
1.10.3
Jun 19, 2025
1.10.2
Jun 09, 2025
1.10.1
May 21, 2025
1.10.0
May 14, 2025
1.9.1
May 21, 2025
1.9.0
Apr 08, 2025
1.8.11
Apr 02, 2025
1.8.10
Mar 27, 2025
1.8.9
Jan 26, 2025
1.8.8
Jan 15, 2025
1.8.7
Dec 19, 2024
1.8.6
Dec 17, 2024
1.8.5
Dec 16, 2024
1.8.4
Dec 15, 2024
1.8.3
Dec 09, 2024
1.8.2
Nov 25, 2024
1.8.1
Nov 19, 2024
1.8.0
Oct 15, 2024
1.7.51
Nov 18, 2024
1.7.50
Oct 29, 2024
1.7.28
Sep 30, 2024
1.7.27
Sep 25, 2024
1.7.26
Sep 17, 2024
1.7.25
Sep 10, 2024
1.7.24
Sep 02, 2024
1.7.23
Jul 23, 2024
1.7.22
Jul 17, 2024
1.7.21
Jun 23, 2024
1.7.20
Jun 20, 2024
1.7.19
Jun 10, 2024
1.7.18
May 28, 2024
1.7.17
May 21, 2024
1.7.16
May 16, 2024
1.7.15
May 15, 2024
1.7.14
May 15, 2024
1.7.13
May 09, 2024
1.7.12
May 07, 2024
1.7.11
Apr 25, 2024
1.7.10
Apr 21, 2024
1.7.9
Apr 21, 2024
1.7.8
Apr 17, 2024
1.7.7
Apr 08, 2024
1.7.6
Mar 31, 2024
1.7.5
Feb 28, 2024
1.7.4
Feb 21, 2024
1.7.3
Feb 14, 2024
1.7.2
Feb 14, 2024
1.7.1
Feb 13, 2024
1.7.0
Feb 07, 2024
1.6.20
Apr 01, 2024
1.6.19
Mar 07, 2024
1.6.18
Jan 28, 2024
1.6.17
Jan 22, 2024
1.6.16
Jan 11, 2024
1.6.15
Jan 08, 2024
1.6.14
Dec 31, 2023
1.6.13
Dec 28, 2023
1.6.12
Dec 20, 2023
1.6.11
Dec 17, 2023
1.6.10
Dec 03, 2023
1.6.9
Dec 03, 2023
1.6.8
Nov 29, 2023
1.6.7
Nov 28, 2023
1.6.6
Nov 15, 2023
1.6.5
Nov 14, 2023
1.6.4
Nov 08, 2023
1.6.3
Nov 05, 2023
1.6.2
Nov 02, 2023
1.6.1
Oct 08, 2023
1.6.0
Sep 21, 2023
1.5.6
Sep 10, 2023
1.5.5
Aug 22, 2023
1.5.4
Aug 21, 2023
1.5.3
Aug 09, 2023
1.5.2
Jul 24, 2023
1.5.0
Jul 18, 2023
1.4.4
Jul 10, 2023
1.4.3
Jun 19, 2023
1.4.2
Jun 14, 2023
1.4.1
May 30, 2023
1.4.0
May 17, 2023
1.3.19
May 02, 2023
1.3.18
Apr 24, 2023
1.3.17
Apr 17, 2023
1.3.15
Mar 13, 2023
1.3.14
Mar 09, 2023
1.3.13
Mar 08, 2023
1.3.12
Mar 05, 2023
1.3.11
Feb 21, 2023
1.3.10
Feb 21, 2023
1.3.9
Feb 16, 2023
1.3.8
Feb 13, 2023
1.3.7
Jan 19, 2023
1.3.6
Jan 15, 2023
1.3.5
Jan 10, 2023
1.3.4
Jan 09, 2023
1.3.3
Jan 01, 2023
1.3.2
Dec 26, 2022
1.3.1
Dec 15, 2022
1.3.0
Dec 05, 2022
1.2.5
Dec 15, 2022
1.2.4
Nov 29, 2022
1.2.3
Nov 03, 2022
1.2.2
Oct 24, 2022
1.2.1
Oct 18, 2022
1.2.0
Oct 06, 2022
1.1.9
Sep 01, 2022
1.1.8
Aug 30, 2022
1.1.7
Aug 15, 2022
1.1.6
Aug 03, 2022
1.1.5
Jul 11, 2022
1.1.4
Jun 30, 2022
1.1.3
Jun 09, 2022
1.1.2
Jun 08, 2022
1.1.1
Jun 07, 2022
1.1.0
Jun 01, 2022
1.0.6
Jun 13, 2022
1.0.5
Jun 07, 2022
1.0.4
Apr 25, 2022
1.0.3
Apr 19, 2022
1.0.2
Apr 04, 2022
1.0.1
Feb 24, 2022
1.0.0
Feb 22, 2022
0.10.5
Feb 08, 2022
0.10.4
Jan 26, 2022
0.10.3
Jan 20, 2022
0.10.2
Jan 18, 2022
0.10.1
Jan 17, 2022
0.10.0
Jan 03, 2022
0.9.2rc1
Jan 26, 2022
0.9.1
Dec 16, 2021
0.9.0
Dec 09, 2021
0.8.15
Dec 07, 2021
0.8.14
Dec 07, 2021
0.8.13
Dec 06, 2021
0.8.12
Dec 06, 2021
0.8.11
Dec 05, 2021
0.8.10
Nov 24, 2021
0.8.9
Nov 18, 2021
0.8.8
Nov 09, 2021
0.8.7
Nov 04, 2021
0.8.6
Oct 31, 2021
0.8.5
Oct 25, 2021
0.8.4
Oct 17, 2021
0.8.3
Oct 10, 2021
0.8.2
Oct 04, 2021
0.8.1
Sep 30, 2021
0.8.0
Sep 12, 2021
0.7.8
Sep 01, 2021
0.7.7
Aug 31, 2021
0.7.6
Aug 31, 2021
0.7.5
Aug 29, 2021
0.7.4
Aug 24, 2021
0.7.3
Aug 18, 2021
0.7.2
Aug 05, 2021
0.7.1
Aug 01, 2021
0.7.0
Jul 27, 2021
0.6.15
Aug 05, 2021
0.6.14
Jul 22, 2021
0.6.13
Jul 13, 2021
0.6.12
Jul 13, 2021
0.6.11
Jul 13, 2021
0.6.10
Jun 30, 2021
0.6.9
Jun 30, 2021
0.6.8
Jun 29, 2021
0.6.7
Jun 28, 2021
0.6.6
Jun 27, 2021
0.6.5
Jun 24, 2021
0.6.4
Jun 23, 2021
0.6.3
Jun 21, 2021
0.6.2
Jun 15, 2021
0.6.1
Jun 13, 2021
0.6.0
Jun 08, 2021
0.5.5
Jul 22, 2021
0.5.4
Jun 02, 2021
0.5.3
Jun 02, 2021
0.5.2
May 30, 2021
0.5.1
May 26, 2021
0.5.0
May 13, 2021
0.4.10
Jul 22, 2021
0.4.9
May 04, 2021
0.4.8
Apr 29, 2021
0.4.7
Apr 28, 2021
0.4.6
Apr 27, 2021
0.4.5
Apr 22, 2021
0.4.4
Apr 14, 2021
0.4.3
Apr 12, 2021
0.4.2
Apr 11, 2021
0.4.1
Apr 07, 2021
0.4.0
Apr 04, 2021
0.3.8
Mar 21, 2021
0.3.7
Mar 15, 2021
0.3.6
Mar 11, 2021
0.3.5
Feb 25, 2021
0.3.4
Feb 15, 2021
0.3.3
Feb 11, 2021
0.3.2
Jan 25, 2021
0.3.1
Dec 31, 2020
0.3.0
Dec 30, 2020
0.2.0
Dec 14, 2020
0.1.0
Dec 09, 2020
