WebSocket support for real-time model streaming

We’ve revamped the experience of using WebSockets to invoke your deployments. Here’s what’s new:

• WebSocket-specific metrics in the UI, like average connection duration

• Improved UX for managing and invoking WebSocket deployments

WebSockets allow your applications to have a persistent, bidirectional communication channel with your models. This is useful for:

• Real-time applications (e.g., audio transcription)

• Streaming responses (e.g., in chat applications)

We also support using WebSockets to invoke Chains for agents or compound AI workflows.

Read the docs to get started!