Simplest Way to Serve Your Machine Learning Model

When reading this post, you perhaps have already known or tried torchserve, triton, seldon core, tf serving, even kserve. They are good products. However, if you are not using a very simple model or you have written many codes and the model is just a part of it. It is not that easy to integrate your codes with them.

Here, you have another alternative: Pinferencia

Github: Pinferencia —Check it out.

What is Pinferencia?

Pinferencia (python + inference) aims to provide the simplest way to serve any of your machine learning models with a fully functioning Rest API.

Straight forward. Simple. Powerful.

Install

pip install “pinferencia[uvicorn]”

Try it now!

Create the App

app.py

from pinferencia import Server
class MyModel:
    def predict(self, data):
        return sum(data)
model = MyModel()
service = Server()
service.register(
    model_name="mymodel",
    model=model,
    entrypoint="predict",
)

Run

uvicorn app:service --reload

Hooray, your service is alive. Go to http://127.0.0.1:8000/ and have fun.

You will have a full API documentation page to play with:

You can test your model right here:

The APIs are all here.

Github: Pinferencia.