Consumer protocol

The Cumulocity Notifications 2.0 API uses a secure WebSocket to consume notifications generated by the Cumulocity API.

The endpoint is accessible using the external Cumulocity fully qualified domain name of your Cumulocity environment and the standard SSL port 443 using a secure WebSocket connection. It is also available on the unsecured port 80 and to microservices using “cumulocity:8111” but in most cases a secure connection is preferred.

The URI scheme therefore is “wss” and consumers use URLs starting with “wss://” followed by the fully qualified domain name of the Cumulocity environment or tenant, followed by a fixed URL path and a query string.

The fixed URL path is /notification2/consumer/ and there are only two query string arguments:

  • token (required). Its value must be a valid token in the form of a JWT token string as returned by a create token request to the Tokens methods of the Notifications 2.0 API. Including the token as a query string parameter avoids having to set an HTTP header which can be an issue for some WebSocket clients or proxies.

  • consumer (optional). Its value is a non blank unique name for the consumer.

In summary, the URLs used by consumers follow the following patterns:

wss://your.cumulocity.environment.fullqualifieddomainname/notification2/consumer/?token=yourJwtTokenRequestedFromNotification2TokenService

or

wss://your.cumulocity.environment.fullqualifieddomainname/notification2/consumer/?token=yourJwtTokenRequestedFromNotification2TokenService&consumer=aUniqueNameForThisConsumer

WebSocket timeouts

There is a timeout of 5 minutes set on idle WebSocket connections after which the connection will be closed by the server side. Therefore the consumer must be prepared to handle closed connections which is required for fault-tolerant operation in any case. All consuming microservices or applications should handle the WebSocket being closed and re-connect as necessary. Alternatively, if you would like to keep the connection from being closed due to idle timeout, implement a ping-pong handler in the WebSocket consumer. For example, you can implement this mechanism in Jetty by following Jetty Programming Guide > Client Libraries > WebSocket Client > WebSocket Session > Sending Ping/Pong. A few libraries also provide built-in support for keeping the connection open. Java-WebSocket, for example, does this by sending the ping requests to the server every minute by default.

Notification acknowledgments

The WebSocket service sends a sequence of UTF-8 encoded textual notification messages to the consumer. Each notification message includes headers and a data payload. Headers occur first in the message and are separated by new line characters. The data payload is last in the message, separated from the last header by 2 new line characters, showing one blank line occurs between the last header and the payload. The first header in the message is always the acknowledgment header.

When the client has finished processing a notification message, it must send that message’s acknowledgment header back to the server on the same connection the message was received on. Sending the acknowledgment tells the server that the consumer has successfully received and processed that message, allowing the server to forget the message. Each acknowledgment is unique to a particular notification and consumer. Note that batch and cumulative acknowledgments are not supported.

If too many of a consumer’s notifications (1000 by default) remain unacknowledged, the flow of notification messages to that consumer will stop until some of its unacknowledged messages are acknowledged. It is therefore best practice to process and acknowledge them quickly, to minimize the potential for a connection interruption causing a need to redeliver them. An acknowledgment should not be sent until its notification has been successfully processed. Otherwise, for example, if the client crashed during or before such processing, the message may be lost as acknowledged messages are not (usually) resent by the server.

The hello-world-notification-microservice example in the cumulocity-examples repository shows an example of how to send the acknowledgment back to the server in a self-contained WebSocket text message. It should be sent without quotation marks, as it is not a JSON message, and without any trailing new-line characters.

Notification message header and content

A notification (transmitted service to client) consists of a header and a body (similar to an HTTP request). The header is one or more (in practice at least 3) lines of text, separated by a \n (newline) character. The end of the header is demarcated by a double new line \n\n.

The notification body follows the header. This also consists of UTF text - for example a JSON document. If the notification is binary data or includes binary data then it will be Base64-encoded.

The header lines for a notification are as follows (separated by \n newlines):

  1. Encoded message identifier for message acknowledgment. After the consumer has finished processing a notification, it must send this header back to the server to acknowledge the notification.

  2. Notification description. This is a /-separated string with three components that describe the type and source of the notification:

    • tenant: This the identifier for the tenant under which the notification was generated.
    • type: The type of the notification generated. These correspond to the APIs that can be used in notification subscriptions, for example measurements or alarms.
    • source: The identifier of the managed object that generated or is the subject of the notification.

  3. An action string is the third header. This will be CREATE, UPDATE or DELETE. Together with the notification type the action describes the logical event that generated the notification, such as the creation of a new alarm or measurement.

After the headers, the notification body follows as UTF-8 text. This is typically a JSON document. Some examples are provided in Traces.

Info

To allow for backwards-compatible evolution of the protcol, consumer code:

  • Must continue to read header lines until the \n\n marker is found. Do not assume that there are only three header lines. Additional header lines beyond the three documented above are optional and can be safely ignored.
  • Must accept and acknowledge every message, even if the consumer did not expect to receive it. Do not assume that the description header and action type will always use the values documented above. Notifications with unexpected descriptions or actions can be safely ignored, but must still be acknowledged.

See the “hello world” microservice for an example of a consumer service that follows these guidelines.

Dealing with notification duplication

When a WebSocket connection is lost, whether that is due to deliberate connection closure or connection failure, messages that were received but not successfully acknowledged before the connection loss are sent to the consumer again when it reconnects. This can result in duplicate messages being received by the consumer. Those duplicates can sometimes include acknowledged messages as these may be on-the-wire from the consumer to the server when the connection is lost.

Notification messages do not contain any specific unique identifiers to aid in de-duplication. Therefore, any de-duplication of messages must be done by the consumer based upon the Notification description header and payload. Note that the acknowledgment header is not guaranteed to be unique across consumer reconnections, which is when duplicates are most likely.

Some events are easy to de-duplicate, such as inventory events where a unique source object is first created and then deleted. It will often be possible to use the notification message headers to determine these cases. However, inventory updates or logically sequenced events such as alarms and measurements will typically require application-specific understanding of the payload fields. Ideally the payload would include a field specifically for that purpose. If it is not possible to include such a field in the payload, then other existing fields will have to be used, maybe on a best effort basis.

A specific field in the payload would typically be a UUID or increasing sequence number. The latter may aid the efficiency of de-duplication by using ordering, though the sequence numbers will be unrelated across publishers, typically IoT devices, when the messages are originated from more than one publisher. Any messages received from the same publisher with a lower sequence number than the last one processed from that publisher can be quickly discarded, assuming the sequence has not rolled over. It is also easier for a human to understand that the ordering is correct and all messages are present.