Mitigating Inter-datacenter Incast with a Proxy: The shortest path is not necessarily the fastest

Many-to-one communication (i.e., incast) is a long-standing challenge in networking with a wide range of proposed solutions. However, as incast-inducing applications today (e.g., storage, ML training) scale beyond a single datacenter, they introduce new challenges that current solutions do not handle. In particular, inter-datacenter links have orders of magnitude higher latency than intra-datacenter paths, lengthening the feedback loop that senders rely on to adjust their sending rates and drastically increasing incast completion times.To reduce inter-datacenter incast latency, we propose adding a proxy server in the sending datacenter to relay traffic between the senders and the receiver. Surprisingly, adding this extra hop reduces incast latency! The insight is that the added hop shifts the congestion point closer to the senders, shortening the feedback loop and allowing senders to converge quickly at a rate that fully utilizes the link while avoiding severe congestion. Motivated by preliminary results, we investigate low-overhead proxy designs and explore ways to expose the proxy as a broader optimization opportunity for application developers, cloud operators, and tenants.