Integrate Distributed Ray Serve Deployment with Kafka | by Rostyslav …

Learn how to simply combine Ray Serve Deployment with asynchronous Kafka Consumer

Ray is a modern open source framework that allows you to create distributed applications in Python with ease. You can create simple training pipelines, do hyperparameter tuning, data processing and model serving.

Ray allows you to create online inference APIs with Ray Serve. You can easily combine several ML models and custom business logic in one application. Ray Serve automatically creates an HTTP interface for your deployments, taking care of fault tolerance and replication.

But there is one thing that Ray Serve misses for now. Many modern distributed applications communicate through Kafka, but there is no out-of-the-box way to connect Ray Serve service to Kafka topic.

But don’t panic. It will not take too much effort to teach Ray Serve to communicate with Kafka. So,…