A Closed-Loop Platform for Prompt-to-Production Development and Autonomous Self-Repair of Apache Flink Streaming Jobs

An integrated AI platform that autonomously develops, deploys, monitors, and repairs Apache Flink streaming jobs using RAG-powered code generation and a real-time debugging sidecar on Kubernetes.

Description

We present an integrated compound AI platform that unifies three novel components to autonomously develop, deploy, monitor, and repair Apache Flink streaming applications. The system comprises: (1) a streaming-native RAG service that continuously ingests knowledge from Git commits, pull requests, issue trackers, and mailing lists into a vector-indexed knowledge graph with domain-aware chunking and cross-source exception linking; (2) a LangGraph-based code generation agent that translates natural-language prompts into production-ready Flink Maven projects with self-healing compilation loops; and (3) an AI debugging sidecar deployed alongside Flink JobManagers on Kubernetes that performs real-time diagnostics via a 17-tool ReAct agent, generates visual analytics, executes savepoint-backed auto-remediation, automatically creates GitHub pull requests for code-level fixes, and integrates with monitoring webhooks for alert-driven autonomous diagnosis. Together, these components form a closed-loop system: the RAG service powers both the code generator and the debugger; failures detected by the debugger feed back into the knowledge base; and the agent autonomously generates fixes, opens pull requests, and redeploys—creating a continuously improving development and operations cycle. We demonstrate the system on production Apache Flink workloads running on Kubernetes.