Self-Hosted: Navigating The Challenges of Distributed Agent Management, Cross-Platform and Cloud Support

In this presentation, we'll explore the complexities of managing distributed agents across cloud and cross-platform environments. From developing sync protocols to handling race conditions and scaling challenges, we'll dive into real-world problems faced when building a robust system for managing agents and job execution. We'll cover the challenges of supporting agents running on multiple platforms — Windows, AWS, macOS, Kubernetes — while maintaining seamless communication with a control plane. Key topics include managing distributed load, enforcing quotas, solving authentication issues, and handling logs in a scalable way. Throughout the talk, we'll share the solutions implemented to tackle these issues and ensure reliable agent management, offering practical lessons for scaling distributed systems and overcoming cloud-specific hurdles.