Binary protocol

Connecting to EdgeDB

The EdgeDB binary protocol has two modes of operation: sockets and HTTP tunnelling. When connecting to EdgeDB, the client can specify an accepted ALPN Protocol to use. If the client does not specify an ALPN protocol, HTTP tunnelling is assumed.

Sockets

When using the edgedb-binary ALPN protocol, the client and server communicate over a raw TCP/IP socket, following the message format and message flow described below.

HTTP Tunnelling

HTTP tunnelling differs in a few ways:

Authentication is handled at /auth/token.

Query execution is handled at /branch/{BRANCH}.

Transactions are not supported.

The authentication phase is handled by sending GET requests to /auth/token with the Authorization header containing the authorization payload with the format:

Authorization: {AUTH METHOD} data={PAYLOAD}

The client then reads the www-authenticate response header with the following format:

www-authenticate: {AUTH METHOD} {AUTH PAYLOAD}

The auth payload’s format is described by the auth method, usually SCRAM-SHA-256. If the auth method differs from the requested method, the client should abort the authentication attempt.

Once the authentication phase is complete, the final response’s body will contain an authorization token used to authenticate the HTTP connection. The client then sends any following message to /branch/{BRANCH} with the following headers:

X-EdgeDB-User: The username specified in the connection parameters.
Authorization: The authorization token received from the authentication phase, prefixed by Bearer.
Content-Type: Always application/x.edgedb.v_1_0.binary.

The response should be checked to match the content type, and the body should be parsed as the message format described below; multiple message can be included in the response body, and should be parsed in order.

Conventions and data Types

The message format descriptions in this section use a C-like struct definitions to describe their layout. The structs are packed, i.e. there are never any alignment gaps.

The following data types are used in the descriptions:

int8	8-bit integer
int16	16-bit integer, most significant byte first
int32	32-bit integer, most significant byte first
int64	64-bit integer, most significant byte first
uint8	8-bit unsigned integer
uint16	16-bit unsigned integer, most significant byte first
uint32	32-bit unsigned integer, most significant byte first
uint64	64-bit unsigned integer, most significant byte first
int8<T> or uint8<T>	an 8-bit signed or unsigned integer enumeration, where T denotes the name of the enumeration
string	a UTF-8 encoded text string prefixed with its byte length as uint32
bytes	a byte string prefixed with its length as uint32
KeyValue	Copy struct KeyValue { // Key code (specific to the type of the // Message). uint16 code; // Value data. bytes value; };
Annotation	Copy struct Annotation { // Name of the annotation string name; // Value of the annotation (in JSON // format). string value; };
uuid	an array of 16 bytes with no length prefix, equivalent to byte[16]

Message Format

All messages in the EdgeDB wire protocol have the following format:

Copy

struct {
    uint8    message_type;
    int32    payload_length;
    uint8    payload[payload_length - 4];
};

The server and the client MUST not fragment messages. I.e the complete message must be sent before starting a new message. It’s advised that whole message should be buffered before initiating a network call (but this requirement is neither observable nor enforceable at the other side). It’s also common to buffer the whole message on the receiver side before starting to process it.

Errors

At any point the server may send an ErrorResponse indicating an error condition. This is implied in the message flow documentation, and only successful paths are explicitly documented. The handling of the ErrorResponse message depends on the connection phase, as well as the severity of the error.

If the server is not able to recover from an error, the connection is closed immediately after an ErrorResponse message is sent.

Logs

Similarly to ErrorResponse the server may send a LogMessage message. The client should handle the message and continue as before.

Message Flow

There are two main phases in the lifetime of an EdgeDB connection: the connection phase, and the command phase. The connection phase is responsible for negotiating the protocol and connection parameters, including authentication. The command phase is the regular operation phase where the server is processing queries sent by the client.

Connection Phase

To begin a session, a client opens a connection to the server, and sends the ClientHandshake. The server responds in one of three ways:

One of the authentication messages (see below);
ServerHandshake followed by one of the authentication messages;
ErrorResponse which indicates an invalid client handshake message.

ServerHandshake is only sent if the requested connection parameters cannot be fully satisfied; the server responds to offer the protocol parameters it is willing to support. Client may proceed by noting lower protocol version and/or absent extensions. Client MUST close the connection if protocol version is unsupported. Server MUST send subset of the extensions received in ClientHandshake (i.e. it never adds extra ones).

While it’s not required by the protocol specification itself, EdgeDB server currently requires setting the following params in ClientHandshake:

user – username for authentication
branch – branch to connect to

Authentication

The server then initiates the authentication cycle by sending an authentication request message, to which the client must respond with an appropriate authentication response message.

The following messages are sent by the server in the authentication cycle:

AuthenticationOK: Authentication is successful.
AuthenticationSASL: The client must now initiate a SASL negotiation, using one of the SASL mechanisms listed in the message. The client will send an AuthenticationSASLInitialResponse with the name of the selected mechanism, and the first part of the SASL data stream in response to this. If further messages are needed, the server will respond with AuthenticationSASLContinue.
AuthenticationSASLContinue: This message contains challenge data from the previous step of SASL negotiation (AuthenticationSASL, or a previous AuthenticationSASLContinue). The client must respond with an AuthenticationSASLResponse message.
AuthenticationSASLFinal: SASL authentication has completed with additional mechanism-specific data for the client. The server will next send AuthenticationOK to indicate successful authentication, or an ErrorResponse to indicate failure. This message is sent only if the SASL mechanism specifies additional data to be sent from server to client at completion.

If the frontend does not support the authentication method requested by the server, then it should immediately close the connection.

Once the server has confirmed successful authentication with AuthenticationOK, it then sends one or more of the following messages:

ServerKeyData: This message provides per-connection secret-key data that the client must save if it wants to be able to issue certain requests later. The client should not respond to this message.
ParameterStatus: This message informs the frontend about the setting of certain server parameters. The client can ignore this message, or record the settings for its future use. The client should not respond to this message.

The connection phase ends when the server sends the first ReadyForCommand message, indicating the start of a command cycle.

Command Phase

In the command phase, the server expects the client to send one of the following messages:

Parse

Instructs the server to parse the provided command or commands for execution. The server responds with a CommandDataDescription containing the type descriptor data necessary to perform data I/O for this command.

Execute

Execute the provided command or commands. This message expects the client to declare a correct type descriptor identifier for command arguments. If the declared input type descriptor does not match the expected value, a CommandDataDescription message is returned followed by a ParameterTypeMismatchError in an ErrorResponse message.

If the declared output type descriptor does not match, the server will send a CommandDataDescription prior to sending any Data messages.

The client could attach state data in both messages. When doing so, the client must also set a correct type descriptor identifier for the state data. If the declared state type descriptor does not match the expected value, a StateDataDescription message is returned followed by a StateMismatchError in an ErrorResponse message. However, the special type id of zero 00000000-0000-0000-0000-000000000000 for empty/default state is always a match.

Each of the messages could contain one or more EdgeQL commands separated by a semicolon (;). If more than one EdgeQL command is found in a single message, the server will treat the commands as an EdgeQL script. EdgeQL scripts are always atomic, they will be executed in an implicit transaction block if no explicit transaction is currently active. Therefore, EdgeQL scripts have limitations on the kinds of EdgeQL commands they can contain:

Transaction control commands are not allowed, like start transaction, commit, declare savepoint, or rollback to savepoint.
Non-transactional commands, like create branch or configure instance are not allowed.

In the command phase, the server can be in one of the three main states:

idle: server is waiting for a command;
busy: server is executing a command;
error: server encountered an error and is discarding incoming messages.

Whenever a server switches to the idle state, it sends a ReadyForCommand message.

Whenever a server encounters an error, it sends an ErrorResponse message and switches into the error state.

To switch a server from the error state into the idle state, a Sync message must be sent by the client.

Dump Flow

Backup flow goes as following:

Client sends Dump message
Server sends Dump Header message
Server sends one or more Dump Block messages
Server sends CommandComplete message

Usually client should send Sync after Dump message to finish implicit transaction.

Restore Flow

Restore procedure fills up the branch the client is connected to with the schema and data from the dump file.

Flow is the following:

Client sends Restore message with the dump header block
Server sends RestoreReady message as a confirmation that it has accepted the header, restored schema and ready to receive data blocks
Clients sends one or more RestoreBlock messages
Client sends RestoreEof message
Server sends CommandComplete message

Note: ErrorResponse may be sent from the server at any time. In case of error, Sync must be sent and all subsequent messages ignored until ReadyForCommand is received.

Restore protocol doesn’t require a Sync message except for error cases.

Termination

The normal termination procedure is that the client sends a Terminate message and immediately closes the connection. On receipt of this message, the server cleans up the connection resources and closes the connection.

In some cases the server might disconnect without a client request to do so. In such cases the server will attempt to send an ErrorResponse or a LogMessage message to indicate the reason for the disconnection.

Binary protocol​

Connecting to EdgeDB​

Sockets​

HTTP Tunnelling​

Conventions and data Types​

Message Format​

Errors​

Logs​

Message Flow​

Connection Phase​

Authentication​

Command Phase​

Dump Flow​

Restore Flow​

Termination​