BentoML

Structure

For the simplest BentoML model package, it has the following structure.

simple_model.zip

├── bentofile.yaml
├── requirements.txt
├── packages.txt
└── service.py

Environment

In the BentoML deployment environment, Python 3.10 is used based on buildpack-deps:jammy (Ubuntu 22.04). The extracted model will be located in /user/model.

Aptitude

You can install packages using apt-get, the Linux package manager. Each package to be installed can be written on a separate line in packages.txt.

packages.txt

ffmpeg
libopenal1

The packages listed in packages.txt will be installed on the system using the following command.

xargs sudo apt-get --no-install-recommends -y install < packages.txt

Virtualenv

The environment preparation script only installs the pip packages listed in requirements.txt. bentoml must be included in requirements.txt.

pip install --no-cache-dir -r requirements.txt

To list all packages currently installed in the environment, use the following command.

pip freeze > requirements.txt

BentoML

In the execution container, BentoML is executed using the following command. You can check if it runs using the same command in the local environment.

bentoml serve /user_model --host 0.0.0.0 --p 8080

For more information on BentoML-based model development, refer to the BentoML docs.