Server-sent events (also called EventSource)
is a simple mechanism to stream data over HTTP. Arroyo supports using SSE/EventSource APIs as sources. Arroyo will
attempt to provide exactly-once semantics on top of SSE by use of the
Last-Event-ID header, but this relies on
proper implementation of the protocol by the server.
Configuring the Connection
SSE/EventSource sources can be created via the Web UI or directly in SQL.
An SSE connection has several required and optional fields:
|endpoint||The endpoint for the SSE server||Yes|
|headers||A comma-separated list of colon separated key-value pairs of headers that will be sent to the server||No|
|events||A comma-separated list of events to listen for. If not set, all events will be used.||No|
For example, in SQL:
CREATE TABLE mastodon ( value TEXT ) /WITH ( connector = 'sse', endpoint = 'https://api./v1/streaming/public', events = 'update', format = 'raw_string');