Intended Audience
- Science/Research
Programming Language
- Python :: 3
Topic
- Scientific/Engineering :: Artificial Intelligence
Beaker-py 
A lightweight pure-Python client for Beaker.
Quick Links
- ๐ Docs and examples:
- ๐ Ecosystem:
Installing
Installing with pip
beaker-py is available on PyPI. Just run
pip install beaker-py
Installing from source
To install beaker-py from source, first clone the repository:
git clone https://github.com/allenai/beaker.git
Then create or activate a Python virtual environment, and run:
cd beaker/bindings/python
make dev-install
Quick start
If you've already configured the Beaker command-line client,
beaker-py will find and use the existing configuration file (usually located at $HOME/.beaker/config.yml) or BEAKER_TOKEN environment variable.
Then you can instantiate the Beaker client with the .from_env() class method:
from beaker import Beaker
with Beaker.from_env() as beaker:
...
With the Python client, you can:
-
Query Clusters with
beaker.cluster.*methods. For example:beaker.cluster.get("ai2/jupiter-cirrascale-2")
-
Manage Datasets with
beaker.dataset.*methods. For example:beaker.dataset.create(dataset_name, source_dir)
-
Manage Experiments with
beaker.experiment.*andbeaker.workload.*methods. For example:beaker.experiment.create(spec=spec, name=name)
-
Manage Groups with
beaker.group.*methods. For example:beaker.group.create(name)
-
Manage Images with
beaker.image.*methods. For example:beaker.image.update(image, name=name)
-
Manage Secrets with
beaker.secret.*methods. For example:beaker.secret.write(name, value)
-
Manage Workspaces with
beaker.workspace.*methods. For example:beaker.workspace.create("ai2/new_workspace")
-
Track Jobs with
beaker.job.*methods. For example:beaker.job.logs(job, follow=True)
-
Create and process Queues with
beaker.queue.*methods. For example:beaker.queue.create("my-work-queue", batch_size=4)
If you're coming from v1 of beaker-py, consider reading the migration guide.
Example workflow
Launch and follow an experiment like beaker-gantry does:
import time
from beaker import Beaker, BeakerExperimentSpec, BeakerJobPriority
with Beaker.from_env() as beaker:
# Build experiment spec...
spec = BeakerExperimentSpec.new(
description="beaker-py test run",
beaker_image="petew/hello-world",
priority=BeakerJobPriority.low,
preemptible=True,
)
# Create experiment workload...
workload = beaker.experiment.create(spec=spec)
# Wait for job to be created...
while (job := beaker.workload.get_latest_job(workload)) is None:
print("waiting for job to start...")
time.sleep(1.0)
# Follow logs...
print("Job logs:")
for job_log in beaker.job.logs(job, follow=True):
print(job_log.message.decode())
See the integration tests for more examples.
Development
After installing from source, you can run checks and tests locally with:
make checks
Releases
At the moment releases need to be published manually by following these steps:
- Ensure you've authenticated with PyPI through a
~/.pypircfile and have write permissions to the beaker-py project. The local~/.pypircfile looks like:
[distutils]
index-servers = pypi
[pypi]
username = __token__
password = #beaker-py pypi API token from 1pass#
- Ensure the target release version defined in
src/beaker/version.pyis correct, or change the version on the fly by adding theMakeargumentBEAKER_PY_VERSION=X.X.Xto the command in the next step. - Ensure the CHANGELOG.md has a section at the top for the new release (
## vX.X.X - %Y-%m-%d). - Run
make publishfor a stable release ormake publish-nightlyfor a nightly pre-release.