Reducing Microservice Complexity with Kafka and Reactive Streams

Transitioning from a monolithic application to a set of microservices can help increase performance and scalability, but it can also drastically increase complexity. Layers of inter-service network calls for add latency and an increasing risk of failure where previously only local function calls existed.

In my talk from Scala Days 2016 in New York, I spoke about how to tame this complexity using Apache Kafka and Reactive Streams to:

  • Extract non-critical processing from the critical path of your application to reduce request latency
  • Provide back-pressure to handle both slow and fast producers/consumers
  • Maintain high availability, high performance, and reliable messaging
  • Evolve message payloads while maintaining backwards and forwards compatibility.

Video

Slides

Reducing Microservice Complexity with Kafka and Reactive Streams from jimriecken
About the Author
Jim Jim is a Senior Engineer on Hootsuite’s Platform team. When he’s not working on building a rapidly expanding collection of Scala/Akka micro services, he likes cycling and writing/playing music. Follow him on Twitter @jimriecken.