BentoML

Structure

For the simplest BentoML model package, it has the following structure.

simple_model.zip

├── bentofile.yaml
├── requirements.txt
├── packages.txt
└── service.py

Environment

In the BentoML deployment environment, Python 3.10 is used based on buildpack-deps:jammy (Ubuntu 22.04). The extracted model will be located in /user/model.

Aptitude

You can install packages using apt-get, the Linux package manager. The packages to be installed can be written line by line in packages.txt.

packages.txt

ffmpeg
libopenal1

The packages in packages.txt will be installed on the system using the following command.

xargs sudo apt-get --no-install-recommends -y install < packages.txt

Virtualenv

The environment preparation script only installs the pip packages specified in requirements.txt. bentoml must be included in requirements.txt.

pip install --no-cache-dir -r requirements.txt

To list all the packages currently installed in the environment, you can use the following command.

pip freeze > requirements.txt

BentoML

In the execution container, BentoML is run using the following command. You can use the same command in the local environment to verify the execution.

bentoml serve /user_model --host 0.0.0.0 --p 8080

For more information on BentoML-based model development, please refer to the BentoML docs.