Server-sent events (also called EventSource) is a simple mechanism to stream data over HTTP. Arroyo supports using SSE/EventSource APIs as sources. Arroyo will attempt to provide exactly-once semantics on top of SSE by use of the Last-Event-ID header, but this relies on proper implementation of the protocol by the server.

​
Configuring the Connection

SSE/EventSource sources can be created via the Web UI or directly in SQL.

SSE creation flow

An SSE connection has several required and optional fields:

FieldDescriptionRequiredExample
endpointThe endpoint for the SSE serverYeshttps://api.example.com/v1/events
headersA comma-separated list of colon separated key-value pairs of headers that will be sent to the serverNoContent-Type:application/json,Authorization: Bearer mytoken
eventsA comma-separated list of events to listen for. If not set, all events will be used.Noevent1,event2

For example, in SQL:

CREATE TABLE mastodon (
    value TEXT
) /WITH (
    connector = 'sse',
    endpoint = 'https://api./v1/streaming/public',
    events = 'update',
    format = 'raw_string');
RedpandaWebhook