Madhushree Kumari

Title of the Talk:
From Microservices to Micro-Agents: Architecting Multi-Agent LLM Choreography.

Abstract:
In modern software engineering, we rely on microservices to break down large, clumsy applications into small, independent pieces. Today, Artificial Intelligence (AI) is going through a similar shift. Instead of using one massive, monolithic AI model to handle everything, developers are beginning to build networks of smaller, specialized AI agents that work together to solve complex tasks.

However, most current multi-agent AI systems are built using centralized orchestration. This means a single “master” AI controller must constantly direct traffic and make every decision. This approach creates a massive single point of failure, slows down response times, and makes the system rigid. Furthermore, without strict guardrails, autonomous AI agents can easily get stuck in infinite conversational loops, wasting computing power and cloud budget.

This presentation introduces a better design pattern: Multi-Agent Choreography. Instead of using a central manager, we look at how to turn standard microservices into completely autonomous “Micro-Agents.” These AI agents collaborate independently over an event-driven message network powered by Apache Kafka. By reading and writing to Kafka streams, agents pick up tasks and pass results along asynchronously, without waiting for a central boss to tell them what to do.

We will focus on two major architectural challenges:

  1. Efficient AI Routing: How to use fast, lightweight routing rules so the system knows exactly which agent should handle a message, without wasting time or money on heavy AI processing just to route data.
  2. System Stability: How to implement automated circuit-breakers and structural rules within the Kafka network to stop runaway agent conversations before they cause crashes or spike cloud costs.

Ultimately, this talk provides a clear, practical blueprint for software architects looking to build scalable, resilient, and intelligent systems that combine the flexibility of modern AI with the stability of enterprise engineering.

Bio:
Madhushree Kumari is an accomplished technology leader with over 12 years of experience architecting and delivering scalable distributed systems. As a Staff Software Engineer at Visa, she functions as a hands-on technical lead for the FX trading platform within the Clearance and Settlement (CAS) team, leading a globally distributed team across the U.S., India, and Poland. Her expertise spans microservices and micro frontend architectures within the FXTI group. Throughout her career, Madhushree has held impactful roles at SAP, McAfee, IBM, and FIS, consistently delivering high-performance, cloud-native solutions in regulatory environments.

A dedicated tech community leader, she is a Senior Member of IEEE, Chair for IEEE Young Professionals Atlanta, and a Distinguished Fellow of the Soft Computing Research Society (DisF-SCRS) and Hackathon Raptors. She also sits on the Advisory Council for Products That Count, is a Founding Member at WomenTech, and contributes to the IEEE Artificial Intelligence Policy Committee (AIPC), advocating for female workforce participation and responsible AI governance.

An award-winning industry figure, she is the Silver Winner of the WomenTech Global Awards 2025 (Software Engineering Leader of the Year), and a recipient of the ISRLA 2025 Women in Tech Excellence Award, the TechHER Impact Award (ITLA 2025), and the SPARKS AWARDS 2026 (Women Tech Power 100). Madhushree frequently contributes to the global tech ecosystem as a keynote speaker, session chair, hackathon judge, and peer reviewer for IEEE journals, while volunteering as a mentor with AnitaB.org, NextUp, and Georgia Tech.