Notifications 2.0

Overview

The Notifications 2.0 API allows applications or microservices to receive and process notifications generated by the use of the Cumulocity IoT REST or MQTT APIs (device management, measurements, alarms, events and other platform APIs) in a reliable manner.

Requirements

The Cumulocity Messaging Service is an optional component of the Cumulocity IoT platform that may need to be enabled before Notifications 2.0 can be used. Changes to the configuration of the load balancer or ingress controller may also be required to allow access to the Websocket endpoint used by Notifications 2.0.

For the shared public cloud instances of the Cumulocity IoT platform, the Messaging Service is enabled by default on release 10.13 and above. For dedicated and self-hosted instances, the Messaging Service and Notifications 2.0 are available for release 10.11 and above, but will need to be explicitly enabled.

Please contact product support to inquire about using the Messaging Service and Notifications 2.0 capabilities in your Cumulocity IoT environment. See the Messaging Service - Installation & operations guide for further technical details of the configuration required, but note that these tasks can only be performed by a Cumulocity IoT platform operator, not by a normal user.

Notifications 2.0 improves upon the Real-time notification API by providing stronger delivery semantics and ordering guarantees. It is also intended to be simpler to use than the “Bayeaux” protocol used in the Real-time notification API. New capabilities are added to Notifications 2.0 in each release of Cumulocity IoT. However, it does not yet support all of the notifications available from the Real-time notification API so it is not yet a complete replacement for the older API. See the rest of this section and the detailed API documentation for full details of the notifications supported by this release of the Notifications 2.0 API.

To receive notifications over the Notifications 2.0 protocol, an application or microservice must subscribe to notifications, either for notifications about a particular managed object or in a wider context that is scoped to a tenant. Subscriptions are persistent, long lived, and allow notifications to be stored reliably until consumed and acknowledged by a consuming microservice or application.

Managed object context

For managed objects, an object’s global identifier must be used to subscribe in the managed object context (“mo”) in order to receive notifications. There can be multiple subscriptions for the same subscribed object, with different filters or just to fan out to multiple interested consumer parties.

This subscribed object is known as the source object for the notification/event and it is referenced in the notifications delivered to subscribers. Subscriptions are set up using the subscription method of the Notifications 2.0 API. This API requires the calling user to be an authenticated Cumulocity IoT user and to have the new ROLE_NOTIFICATION_2_ADMIN role.

When subscribing to notifications, a filter for notifications can be specified which determines the APIs (alarms, events, measurements, managed objects or any combination of these) to filter by. It is also possible to filter by presence of a specific JSON fragment or “fragment type”. When matched, either the whole notification content is forwarded, or one or more fragments can be specified to be copied to the consumer.

In order to receive subscribed notifications, a consumer application or microservice must obtain an authorization token that provides proof that the holder is allowed to receive subscribed notifications.

This token is in the form of a string conforming to the JWT (JSON Web Token) standard that is obtained from the token method of the Notifications 2.0 API. This API requires the calling user to be an authenticated Cumulocity IoT user and to have the new ROLE_NOTIFICATION_2_ADMIN role.

See the Cumulocity IoT OpenAPI Specification for both the Notification 2.0 Subscription and Token API.

Once subscribed, notifications are persisted and available to be consumed using a new WebSocket-based protocol. This protocol implements a reliable delivery with at-least-once semantics. The underlying Messaging Service will repeatedly attempt to deliver a notification until that notification is acknowledged as being received and processed by the consuming application or microservice. Notification order is preserved from the point of view of a device sending in REST and MQTT API requests. The protocol is text-based and described in detail in the next section.

Tenant context

The tenant context (“tenant”) is used for subscribing to and receiving notifications, in addition to the managed object context (“mo”) mentioned above. Creations of managed objects, which generate a new object identifier that can act as a source for notifications are reported in the tenant context. This allows an application to discover a new managed object, which can then choose to subscribe to in the managed object context. It is also possible to subscribe to all alarms that are generated in the tenant context.

See the Cumulocity IoT OpenAPI Specification on how to subscribe to these notifications, additionally filtering the notification of interest.

For the protocol consumer, both managed object creations and alarms subscribed under the tenant context are reported in the same way. There is no distinction between the two contexts for consumers, and notification ordering is maintained between the two contexts.

For Java developers, the API and the protocol have been wrapped up as an open Java API and a sample WebSocket client application.

There is a sample microservice available in the cumulocity-examples repository, so Java developers do not need to code to the following protocol specification directly.

Token expiration

When creating a token, an expiration time must be given in minutes of validity from when the token was created. This security feature limits the potential damage due to leaking of a token. It requires tokens to be re-created or refreshed periodically. This can be done by calling the token create request with the same parameters as originally. If the parameters used are not available they can be extracted from the token. As the token string is a JWT (JSON Web Token), it can be decoded to extract the original information used to create the token by splitting it into 3 parts (on “.") and doing a base64 decode on the first substring. This way, information like the subscription name can be extracted and the create token REST point can be called again, all on the client side. The Cumulocity IoT microservice Java SDK TokenApi class contains a public refresh method which is implemented purely on the client side.

Unsubscribing a subscriber

Once a subscription is made, notifications will be kept until consumed by all subscribers who have previously connected to the subscription. For non-volatile subscriptions, this can result in notifications remaining in storage if never consumed by the application. They will be deleted if a tenant is deleted but otherwise can take up considerable space in permanent storage for high frequency notification sources. It is therefore advisable to unsubscribe a subscriber that will never run again. A separate REST endpoint is available for this: /notification2/unsubscribe. It has a mandatory query parameter token. The token is the same as you would use to connect to the WebSocket endpoint to consume notifications. Note that there is no explicit “subscribe a subscriber” operation using a token. Instead this happens when you first connect a WebSocket with a token for the subscription name and subscriber. However, unsubscribing an application is an explicit act using the original or a similar token. Unsubscribing should be infrequent, for example when deleting an application or during development when testing completes, as typically one wants messages to persist even when no consumer is running. Only if no consumer will ever run again should unsubscribing a subscriber be necessary.

It is also possible to unsubscribe a subscriber on an open consumer WebSocket connection. To do so, send unsubscribe_subscriber instead of a message acknowledgement identifier from your WebSocket client to the service. The service will then unsubscribe the subscriber and close the connection. It’s not possible to check if the unsubscribe operation succeeded as the connection always closes so this way of unsubscribing is mostly for testing.

It is always important to delete subscriptions (Delete operations on /notification2/subscriptions) even having unsubscribed, as otherwise notifications will be generated even if no subscriptions remain. While they would not persist, load and network traffic would still be incurred.

Consumer protocol

The Cumulocity IoT Notifications 2.0 API uses a secure WebSocket to consume notifications generated by the Cumulocity IoT API.

The new endpoint is accessible only using the external Cumulocity IoT fully qualified domain name of your Cumulocity IoT environment and the standard SSL port 443 using a secure WebSocket connection.

The URI scheme therefore is “wss” and consumers use URLs starting with “wss://” followed by the fully qualified domain name of the Cumulocity IoT environment, followed by a fixed URL path and a query string.

The fixed URL path is /notification2/consumer/ and there are only two query string arguments:

In summary, the URLs used by consumers follow the following patterns:

wss://your.cumulocity.environment.fullqualifieddomainname/notification2/consumer/?token=yourJwtTokenRequestedFromNotification2TokenService

or

wss://your.cumulocity.environment.fullqualifieddomainname/notification2/consumer/?token=yourJwtTokenRequestedFromNotification2TokenService&consumer=aUniqueNameForThisConsumer

There is a timeout of 5 minutes set on idle WebSocket connections after which the connection will be closed by the server side. Therefore the consumer must be prepared to handle closed connections which is required for fault tolerant operation in any case.

The WebSocket established with such an URL is a textual bi-directional connection using UTF-8 encoding. The WebSocket service sends a sequence of notifications to the consumer and the consumer should send back a short acknowledgement over this connection for each notification received. This acknowledgement is for a particular notification and the server will periodically resend a notification until it is acknowledged. This will cease once the server has received and processed an acknowledgement for a particular notification.

A notification (transmitted service to client) consists of a header and a body (similar to an HTTP request). The header is one or more (in practice at least 3) lines of text, separated by a \n (newline) character. The end of the header is demarcated by a double new line \n\n.

The notification body follows the header. This also consists of UTF text - for example a JSON document. If the notification is binary data or includes binary data then it will be Base64 encoded.

The header lines for a notification are as follows (separated by \n newlines):

Depending on the second header there may be further headers to follow but currently notifications only use the above three. In order to be future proof and forward compatible, we encourage consumer code to cope with more headers by parsing them out but ignoring them.

See the hello-world-notification-microservice example in the cumulocity-examples repository on how to do this.

After the headers, the notification body follows as UTF-8 text. Typically a JSON document is carried in this text.

Notification description header

The second header line is the notification description string in the form of a /-separated path. For API notifications descriptions have three parts: tenantId, type and sourceId.

Some examples are provided in Traces and backwards compatibility to real-time notifications is provided for.

Also see the rest of the documentation, examples and experiment to get values for events that you are interested in.

Notification acknowledgement

The first header line in each notification consists of an opaque, encoded binary identifier that must be returned as is in a reply to the Notification 2.0 service in a message acknowledgement.

See the hello-world-notification-microservice example in the cumulocity-examples repository on how to do this. It consists of sending the identifier back to the service in a self-contained WebSocket text message, that means, send back the first header without the training \n to the server.

Dealing with notification duplication

Until a notification is acknowledged, the WebSocket service will attempt to re-deliver it. It is therefore desirable that a notification is acknowledged as soon as possible (to help avoid duplicates). However, this should not be done until the notification has been successfully processed to make use of the at-least-once semantics that the Notification 2.0 service provides.

Simply process the message and only return the acknowledgement identifier when that processing completes successfully. If processing is longer than a minute or so, the service will resend the notification so the WebSocket client application must be prepared to deal with duplicates. Duplicates can also occur due to underlying network failures or perceived failures (slow transmissions) and subsequent failure masking re-transmission attempts.

A duplicate can be delivered out of order if several notifications are unacknowledged but only after follow-on notifications so should be easy to deal with.

For example, in the logical sequence 1,2,3,4, the notification number 2 can be duplicated after 3 or even 4 as in 1,2,3,2,4 or even several times, as in 1,2,3,4,2,3..2.

The notifications don’t contain any unique identifier or timestamps to aid in de-duplication. Some events are easy to de-duplicate, such as inventory events where a unique source object is first CREATED and then DELETED. But inventory UPDATES or logically sequenced events such as alarms and measurements require application-specific sequencing.

This can be achieved by including unique identifiers, sequence numbers or timestamps in the notification JSON (body) as required. An alternative is to look up the current value in the Cumulocity IoT database, treating the notification as a signal only and ignoring the value carried.

As can be seen from the notification Traces, some notifications do carry timestamps. If the timestamps are not generated by the device client, then they may only be loosely synchronized between notifications and therefore should be used carefully or not at all for de-duplication.

Parallel consumers

The protocol can deliver notifications to a scaled-out subscriber. It is important to give each instance of the scaled-out application a unique consumer identifier (passing on connect in the query string) as well as specifying “shared=true” when requesting a notification token. Ideally this identifier should persist between re-starts of instances. With this, notifications from a particular device or source are delivered to the same instance of the scaled-out application as long as that instance is running. This makes processing successive notifications about the same device (or source) easier.

Traces

The following is a sample of a trace of messages without showing the first header line which consists of the acknowledgement (encoded string that must be returned to acknowledge the notification).

------------------------
header /tenant-a170/managedobjects/111
header CREATE
notification {"additionParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/additionParents"},"owner":"admin","childDevices":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childDevices"},"childAssets":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childAssets"},"creationTime":"2021-09-03T12:28:58.692Z","lastUpdated":"2021-09-03T12:28:58.692Z","childAdditions":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childAdditions"},"name":"a switch","assetParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/assetParents"},"deviceParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/deviceParents"},"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111","com_cumulocity_model_BinarySwitch":{"state":"ON"}}
------------------------
header /tenant-a170/measurements/111
header CREATE
notification {"self":"http://cumulocity.default.svc.cluster.local/measurement/measurements/117","time":"2021-09-03T12:29:01.664Z","id":"117","source":{"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"c8y_TenantUsageStatisticsMeasurement","resourcesUsage":{"memory":0,"cpu":0,"usedBy":[]}}
------------------------
header /tenant-a170/events/111
header CREATE
notification {"creationTime":"2021-09-03T12:29:01.932Z","source":{"name":"a switch","self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"com_cumulocity_model_DoorSensorEvent","self":"http://cumulocity.default.svc.cluster.local/event/events/118","time":"2021-09-03T12:29:01.664Z","id":"118","text":"Door sensor was triggered"}
------------------------
header /tenant-a170/eventsWithChildren/111
header CREATE
notification {"creationTime":"2021-09-03T12:29:01.932Z","source":{"name":"a switch","self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"com_cumulocity_model_DoorSensorEvent","self":"http://cumulocity.default.svc.cluster.local/event/events/118","time":"2021-09-03T12:29:01.664Z","id":"118","text":"Door sensor was triggered"}
------------------------
header /tenant-a170/managedobjects/111
header UPDATE
notification {"additionParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/additionParents"},"owner":"admin","childDevices":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childDevices"},"childAssets":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childAssets"},"creationTime":"2021-09-03T12:28:58.692Z","lastUpdated":"2021-09-03T12:28:58.692Z","childAdditions":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/childAdditions"},"name":"a switch","assetParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/assetParents"},"deviceParents":{"references":[],"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111/deviceParents"},"self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111","c8y_ActiveAlarmsStatus":{"major":1},"com_cumulocity_model_BinarySwitch":{"state":"ON"}}
------------------------
header /tenant-a170/alarms/111
header CREATE
notification {"severity":"MAJOR","creationTime":"2021-09-03T12:29:02.092Z","count":1,"history":{"auditRecords":[],"self":"http://cumulocity.default.svc.cluster.local/audit/auditRecords"},"source":{"name":"a switch","self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"com_cumulocity_events_TamperEvent","self":"http://cumulocity.default.svc.cluster.local/alarm/alarms/119","time":"2021-09-03T12:29:01.664Z","id":"119","text":"Tamper sensor triggered","status":"ACTIVE","com_mycorp_MyProp":{"key1":"value1"}}
------------------------
header /tenant-a170/alarms/111
header CREATE
notification {"severity":"MAJOR","creationTime":"2021-09-03T12:29:02.092Z","count":1,"history":{"auditRecords":[],"self":"http://cumulocity.default.svc.cluster.local/audit/auditRecords"},"source":{"name":"a switch","self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"com_cumulocity_events_TamperEvent","self":"http://cumulocity.default.svc.cluster.local/alarm/alarms/119","time":"2021-09-03T12:29:01.664Z","id":"119","text":"Tamper sensor triggered","status":"ACTIVE","com_mycorp_MyProp":{"key1":"value1"}}
------------------------
header /tenant-a170/alarmsWithChildren/111
header CREATE
notification {"severity":"MAJOR","creationTime":"2021-09-03T12:29:02.092Z","count":1,"history":{"auditRecords":[],"self":"http://cumulocity.default.svc.cluster.local/audit/auditRecords"},"source":{"name":"a switch","self":"http://cumulocity.default.svc.cluster.local/inventory/managedObjects/111","id":"111"},"type":"com_cumulocity_events_TamperEvent","self":"http://cumulocity.default.svc.cluster.local/alarm/alarms/119","time":"2021-09-03T12:29:01.664Z","id":"119","text":"Tamper sensor triggered","status":"ACTIVE","com_mycorp_MyProp":{"key1":"value1"}}