Migrating from a monolithic Java 8 system to a reactive microservice architecture on Kubernetes allowed us to dramatically improve performance and maintainability. In this article, I’ll share our journey, key Spring Cloud Kubernetes features we adopted, the challenges we faced during development, and the lessons we learned along the way.

Business Logic

We have a data processing logic that streams information into S3 storage using Kafka, Spark Streaming, and Iceberg. Initially, we encountered multiple challenges, such as file optimization issues and Spark’s unpredictable memory behavior. After addressing these issues, we achieved significant cost savings. Once the insert service was completed, we needed to select an appropriate search engine service. We chose Trino as it fulfilled the needs of our data science department. We also serve customers who perform operations on our S3 data, which can result in high system load. Before this modernization, our platform ran on an old monolithic architecture built with Java 8, which created several performance and maintenance challenges.