SmartREST 1.0 has been superseded by SmartREST 2.0.
SmartREST 1.0 will be maintained by Cumulocity but no longer actively developed. We highly recommend you to use SmartREST 2.0 for new device integrations.
This section describes how you can use your existing SmartREST 1.0 templates with MQTT.
Note that SmartREST 1.0 was designed for HTTP request/response and does not support the ID-less communication with MQTT. It only uses the MQTT connection to send exactly the same request as you would send using HTTP. Therefore, it comes with some limitations as MQTT is not request/response.
The support for SmartREST 1.0 was added to ease transition if you have an existing implementation using it.
Although you must send the IDs in the body of each message with SmartREST 1.0, it is still important to connect with the correct MQTT ClientId.
The MQTT ClientId must match the externalId with type c8y_Serial of your device. It is used to assign the correct operations and responses.
Sending and receiving SmartREST 1.0
In general, the following holds for SmartREST requests and responses via MQTT:
All request rows will be sent as single MQTT messages. A single request message always yields a single response message instead of being split up to several response messages.
All SmartREST response templates will be applied to the JSON response of a single request.
Every matching response template will yield one row in the response.
Response lines are separated by \n.
Sending SmartREST 1.0
To send data to the server you can publish the same content as you would POST to the SmartREST endpoint /s.
The X-ID header is part of the topic the client must publish on.
s/ul/<X-ID>
An X-ID acts as a protocol identifier and identifies the template collection.
An X-ID should be immutable: it always identifies exactly the same template collection.
If the template collection changes, the X-ID must also change.
An X-ID should also be globally unique: select an X-ID that is not used by anyone else.
To make sure, we recommend you to use reverse domain names, for example:
com.acme.gw801-v1
com.acme.gw801-v2
We also recommend you to add the protocol version as a postfix in the X-ID.
Processing mode
Since the Cumulocity IoT SmartREST protocol supports TRANSIENT processing mode for avoiding storage of sent data in the database, publishing on MQTT t/ topic instead of s/ topic will only pass the data to real-time processing.
t/ul/<X-ID>
The [Cumulocity IoT SmartREST protocol also supports QUIESCENT processing mode for avoiding real-time notifications by publishing on MQTT q/ topic instead of s/ topic. Currently, the QUIESCENT processing mode is applicable for measurements and events only.
q/ul/<X-ID>
The Cumulocity IoT SmartREST protocol also supports CEP processing mode to ensure that data is only sent to the real-time event processing engine with real-time notifications, disabled by publishing on MQTT c/ topic instead of s/ topic. Currently, the CEP processing mode is applicable for measurements and events only.
c/ul/<X-ID>
Receiving SmartREST 1.0
If a template triggers a response template, the returning message will be published by the server on the following topic.
s/dl/<X-ID>
This topic can be subscribed by the client.
Receiving operations
SmartREST 1.0 via HTTP offers the /notification/operations endpoint to listen to realtime operations. You can receive the same content on the following MQTT topic.
s/ol/<X-ID>
Info
To get notifications running, the platform device must have an external ID set which matches the MQTT client ID, otherwise it will not receive notifications.
Limitations
MQTT currently does not support request/response. Therefore, if you send a request on the publish topic and receive a response on the subscribe topic, the client cannot securely match that they belong together.
You can counter this limitation by designing the templates in a way that you never need to know what request triggered the response, and the client automatically knows it by the messageId.
The protocol
SmartREST is built upon the well-established HTTP protocol making it work everywhere since most popular platforms provide an HTTP client through which SmartREST can be accessed. SmartREST communicates exclusively through the /s resource using the HTTP POST method for bidirectional communication. The payload data format in CSV (comma-separated values).
The following example shows the communication between a client and the SmartREST endpoint. Note the Authorization header and the custom X-Id header in the request which specifies the SmartREST template to use for this request.
Each SmartREST request is represented by one row having a unique unsigned integer as its first value determining the action and subsequent values for parameters.
The SmartREST endpoint yields the following response. Note that the HTTP response code is always 200 unless the SmartREST endpoint is not available.
HTTP/1.1 200 OK
Transfer-Encoding: chunked
200,1,123456,Request result
Each row yielded by the SmartREST endpoint represents a set of extracted values from the result of a SmartREST request containing a unique unsigned integer, the SmartREST request line number and the extracted data values, respectively.
Info
If the response from Cumulocity IoT REST API was empty (for example like after deleting a Managed Object) then the response from SmartREST would be empty as well, regardless of registered response templates.
Info
Inventory access via SmartREST is limited to inventory objects which are global or for which the client is the owner. Role assignments are not evaluated.
Data format
The CSV (comma-separated values) format is used for communication with the SmartREST endpoint. The following rules must be followed to ensure a frictionless communication.
Every linebreak must be encoded by the character sequence \n.
Values are always separated by a comma (,).
If a value contains double-quotes ("), commas (,), leading or trailing whitespaces, line-breaks (\n), carriage returns (\r) or tab stops, it must be surrounded by quotes ("). Contained double-quotes (") must be escaped by prepending another double-quote ("").
The same escaping rules apply to messages that will be sent from the server to the client.
Examples
The following examples illustrate the rules stated above:
100,Hello world!
101," I have leading whitespace!"
102,"I have trailing whitespace!"
103,"I contain a line
break!"
104,"I have ""quotes""!"
105,I also have 'quotes'!
Processing mode
Similar to Cumulocity IoT REST implementation every communication in SmartREST which can lead to data update (i. e., POST, PUT, DELETE) supports four processing modes, PERSISTENT, TRANSIENT, QUIESCENT or CEP.
If the data sent to the SmartREST endpoint must be both stored in Cumulocity IoT database and be transferred to real-time processing, then PERSISTENT mode should be set. It is also enabled by default and does not require additional configuration.
In case when it is only needed to communicate data to real-time processing, the TRANSIENT processing mode should be specified by adding it to the header of HTTP request:
With TRANSIENT mode in the header the body data is not persisted in Cumulocity IoT database.
During real-time processing, CEP scripts can be used to define if updates should be stored in the database or not.
The QUIESCENT processing mode should be specified if data sent to the SmartREST endpoint must be both stored in the Cumulocity IoT database and be transferred to real-time processing but real-time notifications must be disabled. Currently, the QUIESCENT processing mode is applicable for measurements and events only.
The CEP processing mode should be specified if data sent to the SmartREST endpoint must only be transiently sent to real-time processing engine with real-time notifications disabled. Currently, the CEP processing mode is applicable for measurements and events only.
Info
Events are always delivered to CEP/Apama for all processing modes. This is independent from real-time notifications.
Templates
SmartREST templates are a collection of request and response templates used for the conversion of CSV data and Cumulocity IoT REST API calls. Additionally, SmartREST templates contain a template identifier which is compared to the custom X-Id header field to identify the SmartREST template used for processing.
Each request and response template has a unique numeric identifier called the message identifier which is referenced by the first value of each SmartREST request or response row. To avoid collision with one of the default message identifiers, developers are advised to select message identifiers starting at 100.
Request templates
A request template contains all necessary information to convert a SmartREST request into a corresponding REST API call which is then sent to the platform.
A request template contains the following information:
A unique unsigned integer as a message identifier
The request method, for example, GET or POST.
The resource URI, for instance /inventory/managedObjects
The Content-Type and Accept header values of the sent and received data
A placeholder such as %%
The expected request parameters such as STRINGs, NUMBERs, UNSIGNED integers and DATEs
The template string with placeholders for each parameter
Response templates
A response template contains the necessary information to extract data values from a platform REST API call response which are then sent back to the client in the CSV data format.
The following information is contained within a response template:
A unique unsigned integer as a message identifier
A JSON path referencing a base object or object list to extract data from, for example, $ or $.managedObjects. If the JSON path points to a list of objects, one row of extracted data for each object in the list is yielded.
A JSON path which must exist within the base object or base object list in order to extract values, for example, $.id. The value is not added to the response.
A variable number of JSON paths for each value to extract, for example, $.id, $.name or $.type. Values are added to the response in the order they were defined in the template.
Registration process
This reference guide solely focusses on the registration of SmartREST templates using the SmartREST /s endpoint. Alternatively, templates can also be registered using the platform inventory API.
Before a SmartREST template can be registered, its existence must be checked. If the template already exists, a registration is not necessary and yields an error message.
The existence of a SmartREST template can be checked by making an empty request:
POST /s HTTP/1.0
Authorization: Basic ...
X-Id: ...
Transfer-Encoding: chunked
If the template exists, the following response is yielded where the message identifier 20 indicates that the template exists in the inventory and the parameter 123456 indicates the managed object GId of the template:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
20,12345
If the template does not exist, a response containing an error message is yielded:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
40,"No template for this X-ID."
If the template does not exist, a template registration request can be issued using the previously checked X-Id.
Templates can be registered with one single request containing SmartREST template in the form of CSV data. The difference between a template registration request and a normal SmartREST request is that rows are not processed individually during template registration.
If the template registration is successful, a response like below will be returned.
HTTP/1.1 200 OK
Transfer-Encoding: chunked
20,12345
Syntax
Each request and response template is contained within one row of the template data. Request templates are indicated by the message identifier 10 and response templates by the identifier 11. Should one of those message identifiers occour in a SmartREST request, the entire request is treated as a template. Thus any other message identifier besides 10 and 11 will yield an error.
<ID> is the message identifier of the request template.
<METHOD> is the HTTP method used for the request. GET, POST, PUT and DELETE are supported.
<URI> is the resource identifier.
<CONTENT> is the Content-Type header field value.
<ACCEPT> is the Accept header field value. This is mostly equal to <CONTENT>.
<PLACEHOLDER> is the placeholder string which is replaced by parameters in the request URI and template string.
<PARAMS> is a space-separated list of parameter types which are required for the template. The number of occourances of the placeholder string in the request URI and template string must be equal to the number of parameters specified.
STRING parameters will only yield an error if no value is specified.
UNSIGNED parameters will yield an error if the supplied parameter is not an unsigned integer.
INTEGER parameters will yield an error if the supplied parameter is not a signed integer.
NUMBER parameters will yield an error if the supplied parameter is not a floating-point number.
DATE parameters will yield an error if the parameter cannot be parsed as a date.
NOW parameters will never yield an error. No request parameter is required.
<TEMPLATE> is the actual template string which gets sent as payload to the platform after the placeholders have been replaced with the parameter values.
Here is a set of example requests:
10,100,POST,/alarm/alarms,application/vnd.com.nsn.cumulocity.alarm+json,application/vnd.com.nsn.cumulocity.alarm+json,&&,UNSIGNED NOW,"{""source"":{""id"":""&&""},""type"":""c8y_MyAlarmFromSmartREST"",""text"":""This alarm was created by using SmartREST"",""severity"":""MAJOR"",""status"":""ACTIVE"",""time"":""&&""}"
10,200,POST,/measurement/measurements,application/vnd.com.nsn.cumulocity.measurement+json,application/vnd.com.nsn.cumulocity.measurement+json,&&,UNSIGNED UNSIGNED NOW UNSIGNED,"{""c8y_SmartMeasurement"":{""temp1"":{""value"":&&,""unit"":""C""},""temp2"":{""value"":&&,""unit"":""F""}},""time"":""&&"",""source"":{""id"":""&&""},""type"":""c8y_SmartMeasurement""}"
10,300,PUT,/inventory/managedObjects/&&,application/vnd.com.nsn.cumulocity.managedObject+json,,&&,,"{""c8y_Hardware"":{""model"":""&&"",""revision"":""&&""}}"
10,301,PUT,/alarm/alarms/&&,application/vnd.com.nsn.cumulocity.alarm+json,,&&,,"{""status"":""CLEARED""}"
10,302,PUT,/devicecontrol/operations/&&,application/vnd.com.nsn.cumulocity.operation+json,,&&,,"{""status"":""SUCCESSFUL""}"
10,600,GET,/identity/externalIds/c8y_Serial/&&,,,&&,,
10,601,GET,/devicecontrol/operations?deviceId=##&status=PENDING,,,##,,
10,602,GET,/inventory/managedObjects/&&,,,&&,,
The example requests are also included in our Postman collection.
See Using Postman to learn how to import the collection into Postman.
In the Postman collection, the set of requests is located in SmartREST > Register Request Templates.
<ID> is the message identifier of the response template.
<BASE> is the base JSON path pointing to an object or object list from which the values are extracted.
<COND> is a conditional JSON path which gets checked for existance. Only if the path exists, values are extracted.
<VALUE> is a JSON path pointing to a value to extract within the base object or object in the base object list. An unlimited number of <VALUE>s can be specified.
Example
11,201,,"$.c8y_IsDevice","$.id"
Explanation:
11 describes a response template.
201 is the message identifier of the response template.
The base object JSON path is empty, thus $ is assumed.
$.c8y_IsDevice specifies that values are only extracted if the object has a fragment called c8y_IsDevice.
$.id is the value extracted, namely the device ID.
Using SmartREST with multiple X-Ids
SmartREST supports sending of messages for different X-Ids within the same request. In this case the X-Id header mustn’t be used but instead the body will contain additional information about which lines belong to which X-Id.
Sending messages
To indicate the X-Id in the body it is possible to include the following line
15,myxid
All following lines will be handled with the given X-Id until you enter the next X-Id line.
15,myxid1
...
...
15,myxid2
...
Receiving messages
When sending with multiple X-Ids the response also can contain responses from multiple X-Ids. The response will contain an additional line that will indicate which X-Id the following lines are from.
The second value in this line indicates how many lines are following from this X-Id.
87,2,myxid1
...
...
87,1,myxid2
...
Checking if templates are registered
You can check if templates are already existing by just include X-Id lines in the body.
15,myxid1
15,myxid2
15,myxid3
15,myxid4
You will get the same response like described in the registration process but for every line.
20,12345
20,12346
40,"No template for this X-ID."
20,12347
Registering templates
Template registration also supports the use of the X-Id in the body. Therefore you can create multiple in a single request.
All available real-time notification endpoints and channels of the Cumulocity IoT platform are also available in a SmartREST syntax.
See the Real-time notification API to understand the general functionality of the Bayeux protocol and to get an overview of our available endpoints and channels for real-time notifications.
Using Real-time Notifications with SmartREST
To tell the Cumulocity IoT platform that the real-time notifications should use SmartREST all requests send to the URL must contain the X-Id header.
Message identifiers
Message identifier
Message parameters
Description
80
None
Initial handshake that will return a unique bayeux clientId.
81
clientId,channel
Subscribe for the given channel.
82
clientId,channel
Unsubscribe for the given channel.
83
clientId
Establish conntection for receiving the notifications (long-polling).
84
clientId
Disconnect the client from the server.
Handshake
Example request:
80
Example response:
Un1q31d3nt1f13r
Subscribe
Example request:
81,Un1q31d3nt1f13r,/mychannel
Example response:
Unless there is an error there is no specific response for the subscribe
Unsubscribe
Example request:
82,Un1q31d3nt1f13r,/mychannel
Example response:
Unless there is an error there is no specific response for the unsubscribe
Connect
Example request:
83,Un1q31d3nt1f13r
Example response:
The response is formed by the response templates registered via SmartREST for the X-Id. Every received notification via real-time will be parsed with the available templates and every matching template will be returned as response for the connect request.
Keep-Alive:
The Cumulocity IoT platform will send every 10 minutes a space character through an open long-polling connection to detect connection loss. A response for a connect that has been open for a longer time could contain leading space characters in the first line of the response.
Disconnect
Example request:
84,Un1q31d3nt1f13r
Example response:
Unless there is an error there is no specific response for the disconnect
The advice response
The bayeux protocol has a special fragment to tell the client about the recommended settings for timeout of a connection, interval between connect requests and the policy for the follow up after a response for a connect. The advice will be communicated via SmartREST also as a seperate line in the response and can be contained in any response of the above requests.
Response Structure:
86,<timeout>,<interval>,<reconnect policy>
Timeout and interval will be numbers defining the time in millisecons. The reconnect policy can be one of three values:
none: do not reconnect after the response from a connect.
retry: do reconnect after the response from a connect.
handshake: start with a new handshake (for example because the clientId is invalid / server has closed session).
An advice response line does not need to have every value filled
Example:
86,,10000,retry
Subscribing with multiple templates
If your device uses multiple templates (for example, child devices have a different template than the parent) it is possible to add these templates to your subscribe request. The server will than use all templates (from header and subscribe statement) to parse the responses.
Example request:
POST /notification/operations HTTP/1.0
Authorization: Basic ...
X-Id: mytemplate1
81,Un1q31d3nt1f13r,/mychannel,mytemplate2,mytemplate3
In this case multiple templates are used and the response will contain an additional line that indicates which lines are parsed with which templates:
87,{number of parsed rows},{X-Id used to parse the following rows}
Example response:
HTTP/1.0 200 OK
87,2,mytemplate1
100,myvalue
101,myvalue2
87,1,mytemplate3
100,myvalue3
Built-in messages
SmartREST has a variety of built-in messages.
Request messages
Message identifier
Message parameters
Description
10
Template message identifier Method Resource identifier Content MIME type Accept MIME type Placeholder Request parameters Template string
Represents a request template. If this message occours in the body, the whole body is treated as a SmartREST template and thus, all messages besides 10 and 11 will yield an error.
11
Template message identifier Base JSON path Conditional JSON ath Value JSON paths
Represents a response template. If this message occours in the body, the whole body is treated as a SmartREST template and thus, all messages besides 10 and 11 will yield an error.
15
X-Id
Defines which X-Id to use for the following lines. You must not use the X-Id header when using this line.
61
Device MO GId
Poll device credentials during device bootstrapping process. No X-Id header must be present and the device bootstrap authorization must be used.
80
None
Initial handshake that will return a unique bayeux clientId. SmartREST real-time notifications.
81
clientId,channel
Subscribe for the given channel. SmartREST real-time notifications.
82
clientId,channel
Unsubscribe for the given channel. SmartREST real-time notifications.
83
clientId
Establish connection for receiving the notifications (long-polling). SmartREST real-time notifications.
84
clientId
Disconnect the client from the server. SmartREST real-time notifications.
Response messages
Message identifier
Message parameters
Description
20
SmartREST Template MO GId
Echo response message. Template was found or has been created and everything is OK.
40
None
Template not found.
41
Line number (optional)
Template creation error.
42
Line number
Malformed request line
43
Line number
Invalid message identifier.
45
Line number
Invalid message arguments.
50
Line number HTTP response code
Server error. This message occurs when an error happened between the SmartREST proxy and the platform.
70
Line number Unique device identifier Tenant ID Username Password
Device bootstrap polling response with credentials.
86
timeout,interval,reconnect policy
Settings advice for the client using SmartREST real-time notifications.
87
amount of lines, X-Id
Indicates which X-Id was used to create the amount of following response lines.
Error messages
Message identifier
Error message
41
Cannot create templates for already existing template object
41
Duplicate message identifiers are not allowed
41
Bad request template definition
41
Bad response template definition
41
Bad value type: …
41
Bad pattern
41
Not a valid message identifier for template creation
41
Invalid JsonPath
41
Using JsonPath to refer to a list of objects is not allowed for SmartRest
41
Using Filters (?) in JsonPath is not allowed for SmartRest
41
No content type supported for {GET or DELETE} templates.
41
No template string supported for {GET or DELETE} templates.
41
No content type found for {POST or PUT} templates.
41
No template string found for {POST or PUT} templates.
41
Values are only supported for templates with placeholder.
