Python Numpy Array in ML Services

It can be not obvious how to pass Numpy array across separate services when running ML infra in separate containers. When data preparation service runs in a different container than training service. In this tutorial, I show how to convert Numpy to list and JSON to be able to send it through RabbitMQ message broker and consume it on the receiver side.

 

BIY Workflow with FastAPI, Python and Skipper

Build It Yourself. In this video, I explain how you can build workflow running with FastAPI REST generic endpoints. The best thing about it, Skipper architecture is modular and workflow runs in a separate Docker container. This means you can replace it with your own implementation if needed. I explain how workflow call is integrated into FastAPI logic and how the call is made to get the queue name from the workflow. Using this queue name and RabbitMQ message broker, event-based communication runs between containers.

 

MLOps: Extend Skipper ML Services

The goal of this video is to explain Skipper from MLOps user perspective, different blocks of Skipper and how they fit together. I show how a sample set of ML services works and how you could replace it or add your own service. Skipper engine is implemented with Python, but you could add service container implemented in any language. All runs on Kubernetes.

 

Running Kubernetes on Oracle Cloud OCI

Oracle Cloud OCI provides a good environment to run your Kubernetes workloads. In this video, I show how to access Kubernetes cluster in OCI, explain artifacts related to the cluster. I show how Skipper API runs on Kubernetes deployed on OCI. Cluster runtime is accessed through cloud shell.

 

MLOps: Scaling TensorFlow Model on Kubernetes

ML model serving/prediction API can be scaled on Kubernetes by adding or removing Pod instances. I show you a live demo and explain how scaling can be done for TensorFlow model running on Kubernetes.

  

MLOps: Sharing Model Across Services

Typically you would want to scale ML model training and ML inference/prediction services separately. This means services should not share the file storage, at least this is true for the Kubernetes cluster environment. I explain how you could transfer files across services using RabbitMQ.

 

MLOps with TensorFlow and Kubernetes Powered by RabbitMQ and FastAPI

I show how to run TensorFlow model training and data processing containers in a single Pod in Kubernetes. Model training container runs as a main container and data processing as a sidecar. Running both containers in a single Pod, allow to share files in common storage. I'm using persistence volume to store TensorFlow model and stats data from the data processing container. This video shows a walk through the complete use case of data processing, model training and ML microservices communication.