Gimlet Labs has raised a $300 million Series B to expand its AI inference platform, following an $80 million Series A earlier in the year.
Gimlet Cloud is built for workloads where one request can trigger several model calls, retrieval steps and tools. Instead of forcing every stage onto the same type of accelerator, the platform can decompose the workload and place different stages on hardware better suited to them.
One example is separating prefill from token-by-token decoding, which have different compute and memory demands. Gimlet says its scheduler can also work across GPUs, CPUs and other accelerator architectures as workload conditions change.
The company reports performance gains of several times over conventional deployments in some configurations. Those are provider-run results, so the exact benefit will depend on model, latency target and hardware mix. The funding round nevertheless shows how valuable inference efficiency has become as agent workloads grow more complex.