Device integration using MQTT

Overview

The MQTT implementation of Cumulocity IoT provides the following benefits:

The MQTT section is structured as follows:

This section does not describe the basics of MQTT communication. If you are unfamiliar with MQTT, we recommend you to consult one of the numerous introductions in the Internet. Some references can be found on the MQTT website.

Info: For all MQTT connections to the platform, the maximum accepted payload size is 16184 bytes (16KiB), which includes both message header and body. The header size varies, but its minimum is 2 bytes.

MQTT implementation

This section will list the implementation details for the MQTT protocol. The Cumulocity IoT implementation supports MQTT Version 3.1.1.

Connecting via MQTT

Cumulocity IoT supports MQTT both via TCP and WebSockets. As URL you can use the domain of the instance in the format mqtt.<instance_domain> (e.g. mqtt.cumulocity.com) or your tenant domain (e.g. mytenant.cumulocity.com/mqtt).

Available ports:

  TCP WebSockets
SSL 8883 443
no SSL 1883 80

Info: Port 80 is deactivated in cloud systems.

Port 8883 supports two types of SSL: two-way SSL using certificates for client authorization and one-way SSL using username and password for client authorization. The two-way SSL support is enabled by default since version 10.7.0. To disable it please contact product support.

Info: To use WebSockets you need to connect to the path /mqtt and follow the MQTT standard for WebSocket communication.

SmartREST payload

The Cumulocity IoT MQTT implementation uses SmartREST as a payload. SmartREST is a CSV-like message protocol that uses templates on the server side to create data in Cumulocity IoT. It incorporates the highly expressive strength of the REST API but replaces JSON with comma-separated values (CSV) to avoid the complexity of JSON parsing for embedded devices. Additionally, the simple and compact syntax of CSV renders it highly efficient for IoT communication via mobile networks. It can save up to 80% of mobile traffic compared to other HTTP APIs.

Info: For all MQTT connections to the platform, the maximum accepted payload size is 16184 bytes, which includes both message header and body. The header size varies, but its minimum is 2 bytes.

SmartREST basics

A SmartREST message is a single row in which each parameter is separated by comma. The first parameter is an ID that defines the message. You can send multiple messages in a single publish by using a line break between messages.

SmartREST escaping

The CSV (comma-separated values) format is used for communication with the SmartREST endpoint. The following rules must be followed to ensure a frictionless communication.

The same escaping rules apply to messages that will be sent from the server to the client.

Publish example:

100,"This value, needs escaping",This value does not need escaping

Subscribe example:

511,myDeviceSerial,"execute this\nand this\nand \"this\""

Info: \n does not create a new line in the output (e.g. console, UI); to achieve this, a new line character (ASCII 0A) needs to be used.

Device hierarchies

MQTT sessions are linked to a single device, but this device can have a freely configurable device hierarchy below it.

All children require a unique ID defined when creating the device. We recommend you to use a combination of the unique ID of the root device and a unique ID within the hierarchy.

To create data for a child instead of the root device, the unique ID of the child is added as another section in the topic (e.g. s/us/myChildDeviceIdentifier).

The client will automatically receive operations for every child in the hierarchy by subscribing to the respective topic. It is not required to subscribe for each child.

Every operation received will contain the template ID followed by the ID of the device/child for which the operation was created (followed by other operation parameters).

MQTT features

MQTT authentication

The communication with Cumulocity IoT employing MQTT supports authentication in two ways:

Troubleshooting

A device sends correct username and password, but incorrect certificate at the same time

If the platform is configured to support two-way SSL, your devices have a configured keystore with invalid certificates and you want to use basic authorization, we recommend you to turn off sending certificates during connection. Certificates may be invalid because they expired or the root certificate is not uploaded to the platform. Turn off certificate sending in the device’s software. If that is not possible, make sure of the following to make the connection work:

MQTT ClientId

The MQTT ClientId is a field to uniquely identify each connected client. The Cumulocity IoT implementation also uses the ClientId to link the client directly to a device. Therefore, the following format should be used for the ClientId:

connectionType:deviceIdentifier:defaultTemplateIdentifier

Field Mandatory Description
connectionType NO Indication of connection type
default: d (device)
deviceIdentifier YES A unique identifier for your device, e.g. IMEI, Serial number
defaultTemplateIdentifier NO Check the SmartREST section for more information about template identifiers

For the simplest version of a client, the MQTT clientId can just be the deviceIdentfier. It will automatically be interpreted as device connection.

Important: The colon character has a special meaning in Cumulocity IoT. Hence, it must not be used in the deviceIdentifier.

Examples of ClientIds:

mySerialNumber
d:mySerialNumber
d:mySerialNumber:myDefaultTemplate

The uniqueness of the MQTT ClientId is determined only by the deviceIdentifier. Therefore, from the above examples only one client can be connected at the same time.

During a SSL connection with certificates, the deviceIdentifier has to match the Common Name of the used certificate (first certificate in the chain, which is provided by the device).

MQTT Quality of Service

The Cumulocity IoT implementation supports all 3 levels of MQTT QoS:

For subscriptions to the operation or error topics, we will deliver all messages in the QoS which the client defined when subscribing to the topic.

MQTT clean session

MQTT clients can set the clean session flag to “0” (false). This will ensure that in case the client disconnects, your subscription will still work and when you reconnect the client will receive the missed messages.

Info: Cumulocity IoT requires clean session to be set to “1” (true). Currently we cannot guarantee that disabling clean session will work reliably, hence we recommend you to always enable clean session.

MQTT retained flag

In the current Cumulocity IoT implementation, subscriptions to topics where devices publish data are not allowed. Publishing data with the retained flag on this topic is allowed but has no practical difference to sending it without the flag. Messages published by Cumulocity IoT like operations and errors do not contain the retained flag.

MQTT last will

In MQTT, the “last will” is a message that is specified at connection time and that is executed when the client loses the connection. For example, using 400,c8y_ConnectionEvent,"Device connection was lost." as last will message and s/us as last will topic, raises an event whenever the device loses the connection.

Info: The execution of the “last will” updates the device availability.

Debugging

To support developers during development, it is possible to subscribe to the topic s/e. On this topic the device can retrieve debug and error messages that occur during a publish from the device.

Info: This topic is purely designed to support the development of clients. It is not recommended to always subscribe to this channel as the messages are verbose and can significantly increase the data usage. Also, you should not use this topic to trigger actions of the device based on what you receive on the topic. It is not a response channel.

MQTT broker certificates

MQTT broker uses the certificates which are assigned to the main environment domain. MQTT broker always sends these certificates during TLS handshake to devices. Moreover, Enterprise tenants are not able to customize MQTT broker certificates via the SSL Management feature.

Device certificates

Overview

Devices can authenticate against the Cumulocity IoT platform using X.509 client certificates.

Devices can communicate using the MQTT interface of the platform, but MQTT over WebSocket is not supported. The Cumulocity IoT platform expects devices to connect using SSL on port 8883.

Each tenant individually defines whom it trusts by uploading the base CA certificate.

Devices connecting to the platform with certificates do not need to provide the tenant ID, username and password. Authentication information will be obtained from the certificates.

General requirements for connecting devices with certificates

Registering devices using certificates

Cumulocity IoT supports two ways to register devices which will be able to connect using certificates:

Auto registration

The user for the device will be created during the first MQTT call, if a device certificate is derived from a trusted certificate which was uploaded to the Cumulocity IoT platform with a flag autoRegistrationEnabled with a value of true. Auto-registration needs to be activated for the uploaded certificate. If auto-registration is not activated it is required to use the bulk registration (see below). To manage the auto registration field of uploaded certificates in the UI refer to Device Management > Managing device data > Managing trusted certificates.

Bulk registration

The user for the device can also be created via the standard bulk registration in Device Management.

The CSV file used in bulk registration should meet the requirements described in Bulk device credentials in the Reference guide. Moreover, it is required that the CSV file has an additional column “AUTH_TYPE” with value “CERTIFICATES”, and that the column “CREDENTIALS” is either not present or has an empty value.

Single registration

Single registration is not supported for devices which are going to use certificates for authentication.

Info: During device registration, the device user is created, which is necessary for device communication with the platform.

JWT Token retrieval

A device which is authenticated by certificates and connected to the Cumulocity IoT platform can receive a token which can later be used to authenticate HTTP requests.

71,<<Base64 encoded JWT token>>

A device token lifetime can be configured using tenant options: oauth.internal.device-token.lifespan.seconds. The default value is 1 hour. The minimum allowed value is 5 minutes.

A device can fetch a new device token before the old one expires, if it request a JWT token after half of the token’s lifetime has passed.

Introduction to X.509 certificates

X.509 is a standard that defines public key certificates, which are commonly used in the SSL protocol to provide secure connection and data transfer. After the initial release of X.509, it was modified two times. This resulted in version 3 being up-to-date since 1995.

The general purpose of an X.509 certificate is to bind an identity to a pair of keys: the public key is publicly known as a part of the certificate while the private key is known only by the certificate owner. Such pairs of keys should be created with an asymmetric-key algorithm, which is considered secure nowdays. An example of such an algorithm is RSA with at least 2048 bits key size. A size of 1024 bits or lower is no longer considered safe. A private key can be used in two ways:

Every certificate can be self-signed or can be signed by another certificate. To tell if the certificate is self-signed you can look at the “Issuer Name” and “Subject Name” fields of the certificate. If they are the same then it means that it is a self-signed certificate, otherwise the certificate claims to be signed by the issuer. To verify if the issuer really signed the certificate, you have to check the “Signature” field of the certificate. After decryption with the issuer’s public key, the signature should match the data of the signed certificate. Signing a certificate by another, means that if the issuer’s certificate is trusted then the signed certificate also can be trusted. For example if a platform trusts the customer certificate and that customer has 20 devices with individual certificates, then he does not have to upload each one of them. If these devices certificates are signed by the customer certificate, then the platform should trust them too. In this case, every device should send not only its own certificate, but the whole chain of certificates (so-called chain of trust) during the SSL handshake. The chain of certificates starts with the one belonging to the device, through all used intermediate certificates until it reaches the CA certificate trusted by the platform. Usually the chain of certificates does not have to contain the trusted CA certificate, so it can end with the certificate signed directly by the CA. However, in the Cumulocity IoT platform it is also required to provide the trusted CA certificate in the chain of certificates. Providing the chain of certificates lets the platform verify the signatures of every certificate in the chain to make sure that the device certificate is signed directly or indirectly by the trusted certificate. The chain of the certificates can differ in length, so if the platform trusts certificate A and certificate B is signed by A, and certificate C is signed by B, then certificate C will also be trusted. However, there are a few things to keep in mind:

The structure of an X.509 certificate in version 3 looks like this:

To show how the authentication with X.509 certificates works, there is an example of the simplified mutual SSL handshake:

Simplified mutual SSL handshake

A server knows that the client is the owner of the certificate it sent, after the server has decrypted the encrypted copy of the message using the client’s public key. The client knows that the server is the owner of the certificate it sent, if the server uses the correct session key, because that means that the server decrypted it with its private key. In basic authentication with username and password, the password has to be sent over the network to authenticate the user. When certificates are used, the private key is never sent, which makes the use of the certificates much more secure than the basic authentication with username and password.

There are 3 important terms related to X.509 certificates, which everyone should know who wants to start working with them:

Both keystore and truststore can be stored in the same file, but for security reasons we recommended you to keep them separately. The truststore contains public data and the keystore contains the private key, which only to the owner should know. The most popular formats for these files are:

Generating and signing certificates

To generate certificates we will use the OpenSSL toolkit. If you do not have it already installed, then you can download if from the website: https://www.openssl.org/source/

Creating a self-signed CA certificate

  1. Create a directory for the root certificate and the signing configuration, for example: mkdir /home/user/Desktop/caCertificate
  2. Go to the created directory and create a configuration file for your CA certificate: touch caConfig.cnf
  3. Create a database file for keeping the history of certificates signed by the CA: touch database.txt
  4. Create a serial file with initial serial number, which will be used to identify signed certificates. After assigning this serial to the signed certificate, the value in this file will be automatically incremented: echo 1000 > serial
  5. Create subdirectories for signed certificates and the certificate revocation list: mkdir deviceCertificates crl
  6. Fill in the configuration file. This is the example configuration, which can be used for tests after changing the directory dir to your own. If you want to use it in the production environment then please consult it first with some security specialist:
[ ca ]
default_ca = CA_default
[ CA_default ]
# Directory and file locations.
dir               = /home/user/Desktop/caCertificate
certs             = $dir # directory where the CA certificate will be stored.
crl_dir           = $dir/crl # directory where the certificate revocation list will be stored.
new_certs_dir     = $dir/deviceCertificates # directory where certificates signed by CA certificate will be stored.
database          = $dir/database.txt # database file, where the history of the certificates signing operations will be stored.
serial            = $dir/serial # directory to the file, which stores next value that will be assigned to signed certificate.

# The CA key and CA certificate for signing other certificates.
private_key       = $dir/caKey.pem # CA private key which will be used for signing certificates.
certificate       = $dir/caCert.pem # CA certificate, which will be the issuer of signed certificate.

default_md        = sha256 # hash function
default_days      = 375 # default number of days for which the certificate will be valid since the date of its generation.
preserve          = no # if set to 'no' then it will determine the same order of the distinguished name in every signed certificate.
policy            = signing_policy # the name of the tag in this file that specifies the fields of the certificate. The fields have to be filled in or even match the CA certificate values to be signed.

# For certificate revocation lists.
crl               = $crl_dir/caCrl.pem # CA certificate revocation list
crlnumber         = $crl_dir/crlnumber # serial, but for the certificate revocation list
crl_extensions    = crl_ext # the name of the tag in this file, which specifies certificates revocation list extensions, which will be added to the certificate revocation by default.
default_crl_days  = 30 # default number of days for which the certificate revocation list will be valid since the date of its generation. After that date it should be updated to see if there are new entries on the list.

[ req ]
default_bits        = 4096 # default key size in bits.
distinguished_name  = req_distinguished_name # the name of the tag in this file, which specifies certificates fields description during certificate creation and eventually set some default values.
string_mask         = utf8only # permitted string type mask.
default_md          = sha256 # hash function.
x509_extensions     = v3_ca # the name of the tag in this file, which specifies certificates extensions, which will be added to the created certificate by default.

# descriptions and default values of the created certificate fields.
[ req_distinguished_name ]
countryName                     = Country Name (2 letter code)
stateOrProvinceName             = State or Province Name
localityName                    = Locality Name
organizationName                = Organization Name
organizationalUnitName          = Organizational Unit Name
commonName                      = Common Name
emailAddress                    = Email Address

# A default value for each field can be set by adding an extra line with field name and postfix "_default". For example: "countryName_default = PL". If you add this line here, then leaving country name empty during certificate creation will result in the value "PL" being used. If the default value was specified there, but during certificate creation you do not want to use this value, then instead use "." as the value. It will leave the value empty and not use the default.

# default extensions for the CA certificate.
[ v3_ca ]
subjectKeyIdentifier = hash # subject key value will be calculated using hash funtion. It's the recommended setting by PKIX.
authorityKeyIdentifier = keyid:always,issuer # The subject key identifier will be copied from the parent certificate. It's the recommended setting by PKIX.
basicConstraints = critical, CA:true, pathlen:10 # "critical" specifies that the extension is important and has to be read by the platform. CA says if it is the CA certificate so it can be used to sign different certificates. "pathlen" specifies the maximum path length between this certificate and the device certificate in the chain of certificates during authentication. Path length is set here only to show how it is done. If you do not want to specify max path length, you can keep only the "basicConstraints = critical, CA:true" part here.
keyUsage = digitalSignature, cRLSign, keyCertSign # specifies permitted key usages.

# Default extensions for the device certificate. This tag is not used directly anywhere in this file, but will be used from the command line to create signed certificate with "-extensions v3_signed" parameter.
[ v3_signed ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints = critical, CA:false
keyUsage = nonRepudiation, digitalSignature, keyEncipherment

# default extensions for certificate revocation list
[ crl_ext ]
authorityKeyIdentifier=keyid:always

# Policy of certificates signing. It specifies which certificate fields have to be filled in during certificate creation. There are three possible values here:
# "optional" - field value can be empty
# "supplied" - field value must be filled in
# "match" - signed certificate field value must match the CA certificate value to be created
[ signing_policy ]
countryName             = optional
stateOrProvinceName     = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = supplied # every certificate should have a unique common name, so this value should not be changed.
emailAddress            = optional
  1. Create a private key with aes256 encryption and a length of at least 2048 bits. You will also be asked to set the password for the key during its creation: openssl genrsa -aes256 -out caKey.pem 4096
  2. Create a self-signed certificate using specifications from the configuration file. The “days” parameter says how long this certificate will be valid since the generation, so set it as you prefer: openssl req -config caConfig.cnf -key caKey.pem -new -x509 -days 7300 -sha256 -extensions v3_ca -out caCert.pem
  3. You can print the created certificate with the command: openssl x509 -noout -text -in caCert.pem

Creating an intermediate certificate

The intermediate certificate is signed by the CA certificate, but will also be used to sign device certificates. This step is optional. If you are fine with signing all the device certificates with one common CA certificate, then you can skip this step. However, if you need some certificates between the CA certificate and the device certificate then it is the way to go. Keep in mind that in the Cumulocity IoT cloud the maximum length of the chain of certificates is currently restricted to 2 for security reasons, so you cannot use any intermediate certificate between your CA certificate and the device certificate there. However, this behaviour can be changed for dedicated installations by changing a platform wide configuration setting and increasing the allowed maximum length of the chain of certificates to more than 2. To create the intermediate certificate:

  1. Create a new directory for intermediate certificates inside the caCertificate path: mkdir intermediateCertificate
  2. Go to this directory and create a configuration file for your intermediate certificate: touch intermediateConfig.cnf
  3. Create a database file for keeping the history of certificates signed by the intermediate certificate: touch database.txt
  4. Create a serial file with initial serial number, which will be used to identify signed certificates: echo 1000 > serial
  5. Create subdirectories for the signed certificates and the certificate revocation list: mkdir deviceCertificates crl
  6. Fill in the configuration file as in the CA configuration, but remember to change the general directory (“dir”) to your intermediateCertificate folder. Also don’t forget to change private_key, certificate and crl names from current with “ca” prefix to “intermediate” prefix (for example: caKey.pem -> intermediateKey.pem), because files with this prefix will be generated in the next steps.
  7. Generate a private key for the intermediate certificate: openssl genrsa -aes256 -out intermediateKey.pem 4096
  8. Generate a certificate signing request: openssl req -config intermediateConfig.cnf -new -sha256 -key intermediateKey.pem -out intermediateCsr.pem
  9. Go to the caCertificate directory and generate a signed intermediate certificate. You have to use the CA configuration here, because the private key specified in the configuration file will be used to sign the certificate: openssl ca -config caConfig.cnf -extensions v3_ca -days 3650 -notext -md sha256 -in intermediateCertificate/intermediateCsr.pem -out intermediateCertificate/intermediateCert.pem
  10. Verify if the generated certificate is correctly signed by CA: openssl verify -CAfile caCert.pem intermediateCertificate/intermediateCert.pem

Creating a device certificate signed by CA or intermediate

  1. Go to the directory of your caCertificate or intermediateCertificate depending on which one is used to sign the device certificate.
  2. Generate the private key for the new certificate: openssl genrsa -aes256 -out deviceCertificates/deviceKey.pem 4096
  3. Generate the certificate signing request (change “caConfig.cnf” to “intermediateConfig.cnf” if you are in the intermediateCertificate directory): openssl req -config caConfig.cnf -new -sha256 -key deviceCertificates/deviceKey.pem -out deviceCertificates/deviceCsr.pem Remember that the commonName of the device certificate, which you will be asked to provide in the console, has to match the ClientId of the device during the connection.
  4. Generate the certificate signed by the CA or intermediate (change “caConfig.cnf” to “intermediateConfig.cnf” if you are in the intermediateCertificate directory): openssl ca -config caConfig.cnf -extensions v3_signed -days 365 -notext -md sha256 -in deviceCertificates/deviceCsr.pem -out deviceCertificates/deviceCert.pem
  5. Verify if the generated certificate is correctly signed by CA or intermediate (change “caCert.pem” to “intermediateCert.pem” if you are in the intermediateCertificate directory): openssl verify -partial_chain -CAfile caCert.pem deviceCertificates/deviceCert.pem

Creating the chain of certificates

Go into your caCertificate directory.

Creating keystore and truststore

  1. Go into your deviceCertificates directory with the device’s private key and the generated chain of certificates. If you are using an intermediate certificate between the CA certificate and the device certificate then it will be the caCertificate/intermediateCertificate/deviceCertificates path, otherwise it will be caCertificate/deviceCertificates. Create keystore using the generated chain of certificates and the private key of the device: openssl pkcs12 -export -name devicekeyentry -inkey deviceKey.pem -in deviceCertChain.pem -out deviceKeystore.pkcs12
  2. If you want to convert your keystore to JKS format then you would need the Java Keytool which is usually downloaded together with Java Development Kit: keytool -importkeystore -srckeystore deviceKeystore.pkcs12 -srcstoretype PKCS12 -destkeystore deviceKeystore.jks -deststoretype JKS
  3. If you do not have the server certificate, get it by the command: openssl s_client -showcerts -connect <cumulocity url>:<mqtt mutual ssl port (currently 8883, but that can be changed in the future)> | openssl x509 -outform PEM > serverCertificate.pem
  4. Now you can create a truststore, which will contain the server certificate. It has to be created with Java Keytool (openssl does not support creating truststore, so if you don’t want to use Java keytool then you will have to keep every trusted certificate in a separate pem file). Remember that alias is the unique identifier for every keystore or truststore entry. It means that if you want to add a second trusted certificate to the same truststore then you will have to change the alias from servercertificate in the command below to some other name:
    • In PKCS12 format: keytool -importcert -noprompt -keystore deviceTruststore.pkcs12 -alias servercertificate -file serverCertificate.pem
    • In JKS format: keytool -import -file serverCertificate.pem -alias servercertificate -keystore deviceTruststore.jks
  5. Optionally, instead of creating a new file for the truststore, you can add the trusted certificates to your created keystore and store everything in one file, which is not the recommended solution:
    • If your keystore is in the PKCS12 format: keytool -importcert -noprompt -keystore deviceKeystore.pkcs12 -alias servercertificate -file serverCertificate.pem
    • If your keystore is in the JKS format: keytool -import -file serverCertificate.pem -alias servercertificate -keystore deviceKeystore.jks
  6. You can check the content of your keystore (or truststore) with the command: keytool -list -v -keystore deviceKeystore.jks

How to test created certificates with MQTT.fx client

  1. Generate a keystore and a truststore as described in Generating and signing certificates if you didn’t do it yet.

  2. Upload your CA (or intermediate) certificate to the platform. This operation will add your uploaded certificate to the server’s truststore. It can be done in two ways, both of which have a role requirement of either ROLE_TENANT_ADMIN or ROLE_TENANT_MANAGEMENT_ADMIN:

    • Via UI:

      1. You have to open the device management application, then navigate to the Management tab and select Trusted certificates.
      2. Drop your caCert.pem (or intermediateCert.pem).
      3. Check the auto-registration field.
      4. Click on the certificate status to set it to “Enabled”.
      5. Insert some custom name.
      6. Click Add certificate.

      After completing all the steps except adding the certificate, the form should look like this:

      Trusted certificate addition

      Then the added certificate should be visible:

      Trusted certificate added

    • Via REST:

      1. Display your CA (or intermediate) certificate, which you want to upload to the Cumulocity IoT platform and copy its PEM value, which starts with “—–BEGIN CERTIFICATE—–” and ends with “—–END CERTIFICATE—–” (including the hyphens). Remove new line symbols (\n) if they were added automatically at the end of each line: openssl x509 -in caCert.pem -text
      2. Send it to the platform via POST request:
          POST /tenant/tenants/<TENANT_ID>/trusted-certificates
          Host: https://<TENANT_DOMAIN>/
          Authorization: Basic <YOUR_AUTHENTICATION>
          Content-Type: application/json
          {
          	"status" :  "ENABLED",
          	"name" : "certificateName",
          	"autoRegistrationEnabled" : "true",
          	"certInPemFormat" : "<CERT_PEM_VALUE>"
          }
      
  3. Download and install the newest MQTT.fx client from: http://www.jensd.de/apps/mqttfx/

  4. In MQTT.fx click Extras at the top and then Edit Connection Profiles

  5. Insert the Cumulocity URL in the “Broker address” line.

  6. Insert the SSL port in the “Broker port” line.

  7. In the “Client ID” field, insert the common name of your device certificate.

  8. Select SSL/TLS as the authentication type.

  9. Click Enable SSL/TLS.

  10. Select “SSLv3 protocol”.

  11. Select “Self signed certificates in keystores”

  12. In “Keystore File” insert the path to your deviceTruststore file with either JKS or PKCS12 format.

  13. In “Trusted Keystore Alias” insert “servercertificate” or a different value if you provided a different alias in step 3 above.

  14. In “Trusted Keystore Password” insert the password, which you created during the deviceTruststore file creation.

  15. In “Client Keystore” insert the path to your deviceKeystore file with either JKS or PKCS12 format.

  16. In “Client Keystore Password” insert the password you created during the deviceKeystore creation.

  17. In “Client KeyPair Alias” insert “devicekeyentry” or a different value if you provided a different alias in the “-name” parameter during the step about keystore creation in Generating and signing certificates.

  18. In “Client KeyPair Password” insert the password, which you created during the deviceKey.pem creation.

  19. The “PEM formatted” field should be checked.

  20. Save and close the settings.

  21. Select the edited profile and click connect.

  22. You should be succesfully connected and the buttons “Disconnect”, “Publish” and “Subscribe” should be active now. This means that your connection with the certificates work correctly.

The connection settings should look like this:

MQTT.fx configuration

Cumulocity MQTT Example Client

The code of our example MQTT client implemented in Java, which connects to the platform using x.509 certificates, is available here: https://github.com/SoftwareAG/cumulocity-examples/tree/develop/mqtt-client. This example client uses the implementation of Eclipse Paho, which is described in detail on their website: https://www.eclipse.org/paho/index.php?page=documentation.php.

Here is an example that shows how to add the needed dependency in Maven to use Eclipse Paho client:

<dependency>
    <groupId>org.eclipse.paho</groupId>
    <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
    <version>${paho.version}</version>
</dependency>

Then the instance of the MQTT client can be created with a single line:

MqttClient mqttClient = new MqttClient(BROKER_URL, "d:" + CLIENT_ID, new MemoryPersistence());

The BROKER_URL should contain protocol, url and port, which the client will connect to, like this: ssl://<cumulocity url>:8883. The CLIENT_ID value has to match the value of the Common Name of the device certificate that will be used. The “d:” prefix is used in Cumulocity IoT for device connections and it should not be removed or changed. Now the only thing that needs to be configured to establish the SSL connection is to fill paths in the code fragment:

sslProperties.put(SSLSocketFactoryFactory.KEYSTORE, getClass().getClassLoader().getResource(KEYSTORE_NAME).getPath());
sslProperties.put(SSLSocketFactoryFactory.KEYSTOREPWD, KEYSTORE_PASSWORD);
sslProperties.put(SSLSocketFactoryFactory.KEYSTORETYPE, KEYSTORE_FORMAT);
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTORE, getClass().getClassLoader().getResource(TRUSTSTORE_NAME).getPath());
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTOREPWD, TRUSTSTORE_PASSWORD);
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTORETYPE, TRUSTSTORE_FORMAT);

After filling in this data, the example client will use the provided data to connect to the specified platform using certificates. The example also shows how to create the callback for the connection. First thing is to create the class which implements the interface MqttCallbackExtended. Then such a class can be created and an instance of it can be provided to the MQTT client: mqttClient.setCallback(this);.

In general, the MQTT Eclipse Paho Client uses the Java Secure Socket Extension, which is part of the Java Development Kit, to provide secure connections via SSL. JSSE provides the Java implementation of the SSL and TLS protocol, which can be configured by developers using its classes. The documentation of the Java Secure Socket Extension shows how the SSL connection is established and provides some examples of customizing the implementation. The full document is available on the official Oracle website.

Device integration

Overview

The basic life cycle for integrating devices into Cumulocity IoT is discussed in Interfacing devices.

In this section, we will show how this life cycle can be managed using the MQTT implementation.

The life cycle consists of two phases, a startup phase and a cycle phase.

The startup phase can be as short as just checking the credentials:

The cycle phase consists of two kinds of actions:

MQTT phases

Startup phase

Step 0: Request device credentials

In Cumulocity IoT, every MQTT connection needs to be authenticated. You can use the device credentials topics in the MQTT implementation to generate new credentials for a device.

Once the device retrieved the credentials, it needs to store them locally for further connections.

To establish a connection you need to configure the following parameters:

For more information, refer to the Hello MQTT section.

The process works as follows:

After receiving the credentials, the device can close the MQTT connection and create a new one with the received credentials.

Step 1: Verify device

As MQTT supports an automatic device creation if the client sends data and there is no device present, this step is only required if you want to create the device manually.

The device creation can be achieved by employing the static template 100. This template can be blindly used on every boot of the device as it will only create the device if it is not already present.

The device will be linked automatically to the ID the client uses with its MQTT ClientId.

100,Device Name,Device Type

Info: The topic used for Cumulocity IoT’s pre-provided static templates is s/us.

Step 2: Verify children

Like the root device, also children of it are covered by the automatic device creation.

To handle this step manually you can send the static template 101 for creating a child device. The template will only create the child if it does not already exist.

101,Unique Child ID,Child Name,Child Type

Step 3: Subscribe topics

If the device supports operations, it should subscribe to all required topics (static templates and SmartREST 2.0).

Cycle phase

Step A: Send CSV data

While the device holds an active MQTT connection, it can publish either on the topics for static templates or on the topics for a SmartREST template to send data to the server.

Based on the MQTT ClientId, the physical device is directly connected to the device object in Cumulocity IoT. Therefore, the data you send is automatically connected to the device.

To send data to a child device, publish the data to the topics described in Device hierarchies.

Step B: Receive CSV operations

By subscribing to a topic the device automatically tells Cumulocity IoT that it wants to receive operations. Any operation created will be automatically parsed using either the static templates or the templates the device defines.

SmartREST 1.0

This section describes how you can use your existing SmartREST 1.0 templates with MQTT.

Note that SmartREST 1.0 was designed for HTTP request/response and does not support the ID-less communication with MQTT. It only uses the MQTT connection to send exactly the same request as you would send using HTTP. Therefore, it comes with some limitations as MQTT is not request/response.

The support for SmartREST 1.0 was added to ease transition if you have an existing implementation using it.

Info: If you start a new device integration, we highly recommend to use SmartREST 2.0.

For general information on SmartREST 1.0, refer to Using the REST interface > Using SmartREST in the Microservice SDK guide.

MQTT ClientId

Although you need to send the IDs in the body of each message with SmartREST 1.0, it is still important to connect with the correct MQTT ClientId.

The MQTT ClientId needs to match the externalId with type c8y_Serial of your device. It is used to assign the correct operations and responses.

Sending SmartREST 1.0

To send data to the server you can publish the same content as you would POST to the SmartREST endpoint /s.

The X-ID header is part of the topic the client needs to publish on.

s/ul/<X-ID>

An X-ID acts as a protocol identifier and identifies the template collection. An X-ID should be immutable: it always identifies exactly the same template collection. If the template collection changes, the X-ID must also change. An X-ID should also be globally unique: select an X-ID that is not used by anyone else. To make sure, we recommend you to use reverse domain names, e.g.:

com.acme.gw801-v1
com.acme.gw801-v2

We also recommend you to add the protocol version as a postfix in the X-ID.

Processing mode

Since the Cumulocity IoT SmartREST protocol supports TRANSIENT processing mode for avoiding storage of sent data in the database, publishing on MQTT t/ topic instead of s/ topic will only pass the data to real-time processing.

t/ul/<X-ID>

The Cumulocity IoT SmartREST protocol also supports QUIESCENT processing mode for avoiding real-time notifications by publishing on MQTT q/ topic instead of s/ topic. Currently, the QUIESCENT processing mode is applicable for measurements and events only.

q/ul/<X-ID>

The Cumulocity IoT SmartREST protocol also supports CEP processing mode to ensure that data is only sent to the real-time event processing engine with real-time notifications, disabled by publishing on MQTT c/ topic instead of s/ topic. Currently, the CEP processing mode is applicable for measurements and events only.

c/ul/<X-ID>

Receiving SmartREST 1.0

If a template triggers a response template, the returning message will be published by the server on the following topic.

s/dl/<X-ID>

This topic can be subscribed by the client.

Receiving operations

SmartREST 1.0 via HTTP offers the /devicecontrol/notifications endpoint to listen to realtime operations. You can receive the same content on the following MQTT topic.

s/ol/<X-ID>

Info: To get notifications running, the platform device must have an external ID set which matches the MQTT client ID, otherwise it will not receive notifications.

Limitations

MQTT currently does not support request/response. Therefore, if you send a request on the publish topic and receive a response on the subscribe topic, the client cannot securely match that they belong together.

You can counter this limitation by designing the templates in a way that you never need to know what request triggered the response, and the client automatically knows it by the messageId.

SmartREST 2.0

Overview

This section describes the SmartREST 2.0 payload format that can be used with the Cumulocity IoT MQTT implementation.

SmartREST 2.0 was designed to make use of the MQTT protocol, so it may reduce the payload even more than the SmartREST 1.0 via HTTP.

SmartREST 2.0 is only available via MQTT and it offers the following MQTT topics for the main communication:

To publish messages:

s/uc/<X-ID>

To publish messages in transient mode:

t/uc/<X-ID>

To publish messages in quiescent mode:

q/uc/<X-ID>

To publish messages in CEP mode:

c/uc/<X-ID>

Refer to SmartREST > Processing mode in the Reference guide for more information about transient, quiescent & CEP data processing.

To subscribe for responses:

s/dc/<X-ID>

The topics for creating templates are described in Creating templates via MQTT.

Changes from SmartREST 1.0

In its base, SmartREST 2.0 is like the previous version: a CSV-like payload format that is backed by previously created templates to finally create the targeted JSON structure.

Several changes in the functionality have been made:

Supported templates

SmartREST 2.0 lets you create templates for the following matching HTTP methods:

API GET POST PUT
Inventory x x x
Alarm   x x
Event   x  
Measurement   x  
Operation     x

Additionally, you can create templates to return certain values from responses and operations.

Template collections

A template collection is a set of request and response templates that specifies a device communication protocol. Each collection is referenced by a unique ID (called X-ID).

Creating templates via MQTT

Like in SmartREST 1.0, you need to pass all templates in a collection in one message. After the creation of a template collection, it can no longer be modified through MQTT.

When creating templates, the client needs to publish to the following topic:

s/ut/<X-ID>

To verify if a template collection exists, the client can subscribe to the topic:

s/dt

When subscribed, the client can send an empty message to the creation topic which will trigger a new message about the creation status of this X-ID.

Examples

Empty publish to s/ut/myExistingTemplateCollection

20,myExistingTemplateCollection,<ID of collection>

Empty publish to s/ut/myNotExistingTemplateCollection

41,myNotExistingTemplateCollection

Request templates

A request template contains the following basic fields:

Field Data type Possible values Mandatory Description
messageId String   Y Unique ID to reference the template within the collection
method String GET
PUT
POST
Y Whether to get, update or create data
api String INVENTORY
MEASUREMENT
ALARM
EVENT
OPERATION
Y Cumulocity IoT API to be used
response Boolean true
false
N Whether the request should trigger response templates. For GET templates by default true otherwise by default false
mandatoryValues List<String>   Y Values for the mandatory fields on the API. The values depend on the API and method the template uses
customValues List<CustomValue>   N Custom values that should be added to the object

A request template lists all the fragments in the object structure (mandatory and custom) that should be added when creating or updating the data. It can set fixed values in the template that will then be replaced by the server. If it does not set the value in the template, the value needs to be included in the publish message (this includes mandatoryValues).

Info: If the message rate limit per second is exceeded, the requests are delayed and kept in queue. If the queue limit number is exceeded, the client messages are rejected and the client is disconnected.

Example

We create a template to create a measurement like this (measurements have two mandatory values: type and time)

# 10,msgId,method,api,response,type,time,custom1.path,custom1.type,custom1.value
10,999,POST,MEASUREMENT,,c8y_MyMeasurement,,c8y_MyMeasurement.M.value,NUMBER,

This template defines one additional custom property for the measurement. It leaves two fields empty in the template declaration (time and the custom property), so to use the template the client needs to send these two values:

999,2016-06-22T17:03:14.000+02:00,25
# We can also use server time by leaving the time empty
999,,25

The following sections will get into more detail of how to create and use different templates.

GET templates

GET templates for the inventory do not need any mandatory or custom values. Instead, they use two different fields.

With SmartREST 2.0 you have the option to either get an object from inventory by its ID or by an external ID directly. Therefore, instead of the fields mandatoryValues and customValues, the following two fields are used:

Field Data type Possible values Mandatory Description
byId Boolean true
false
Y Whether the GET should be executed by Cumulocity IoT ID (=true) or externalId (=false)
externalIdType String   N Sets a fixed externalIdType if the template calls by externalId

This enables you to query inventory in three different ways:

By Cumulocity IoT ID

# Creation:
10,999,GET,INVENTORY,,true
# Usage:
999,123456

By external ID with a fixed type in the template

# Creation:
10,999,GET,INVENTORY,,false,c8y_Serial
# Usage:
999,myDeviceImei

By external ID without fixed type in the template

# Creation:
10,999,GET,INVENTORY,,false
# Usage:
999,c8y_Serial,myDeviceImei
POST templates

POST templates require a different set of mandatory values based on the API:

API mandatory values
MEASUREMENT type, time
EVENT type, text, time
ALARM type, text, status, severity, time
INVENTORY externalIdType

This results in the following minimal template creations:

# Creation:
10,100,POST,MEASUREMENT,false,c8y_MyMeasurement,,c8y_MyMeasurement.M.value,NUMBER,
10,101,POST,EVENT,,c8y_CustomEvent,mytext,,
10,102,POST,ALARM,,c8y_CustomAlarm,mytext,ACTIVE,MAJOR,

# Usage:
100
101
102

Creating data on the inventory optionally includes the creation of an externalId for that object. This is controlled by the mandatory value externalIdType.

Important: POST Inventory templates start with the value of the externalId after the msgId. Leaving this column empty will result in not creating an external ID.

# Creation:
10,100,POST,INVENTORY,,c8y_MySerial
# Usage:
# Create object with externalId c8y_MySerial/myImei
100,myImei
# Create object with externalId c8y_MySerial/myImei
101,myImei,c8y_MySerial
# This message will result in not creating an external ID
101,,c8y_MySerial
PUT templates

PUT templates for inventory follow the same logic as the GET templates, and with the addition that you can also use custom values for PUT.

# Creation:
# 10,msgId,method,api,response,byId,externalIdTyoe,custom1.path,custom1.type,custom1.value
10,999,PUT,INVENTORY,,false,c8y_Serial,c8y_MyCustomValue,STRING,
# Usage:
999,myDeviceImei,myValue

The PUT template for alarms uses the type of the alarm to find the alarm to update. It will first check the ACTIVE alarms and, if there is no ACTIVE alarm, it will check the ACKNOWLEDGED alarms.

# Creation:
# 10,msgId,method,api,response,type,custom1.path,custom1.type,custom1.value
10,999,PUT,ALARM,,c8y_MyCustomAlarm,status,ALARMSTATUS
# Usage:
999,CLEARED

PUT templates for operations use the fragment of the operation to find the operation. It will first check the EXECUTING operations and, if there is no EXECUTING operation, it will check the PENDING operations.

# Creation:
# 10,msgId,method,api,response,fragment,custom1.path,custom1.type,custom1.value
10,999,PUT,OPERATION,,c8y_MyOperation,status,OPERATIONSTATUS,SUCCESSFUL,c8y_Fragment.val,NUMBER,
# Usage:
999,24
Adding custom properties

All POST and PUT values enable you to add custom properties to the results of the templates.

A single custom property requires you to add the following three values to your template creation:

Field Description
path A JsonPath for the value that should be set
type An optional data type of the value. Default: STRING
value The value to be set. Leaving this field empty requires the client to send the value when using the template
Type Description
STRING The default type. No additional verification of the value
DATE A time stamp in the ISO 8601 format. Using date and not sending a time stamp results in the use of server time
NUMBER An integer or number with decimal places
INTEGER An integer
UNSIGNED An integer (only positive)
FLAG An empty map (e.g. c8y_IsDevice: {}). The client does not need to send anything for this value
SEVERITY A severity of an alarm. Used to update the severity field of alarms
ALARMSTATUS A status of an alarm. Used to update the status field of alarms
OPERATIONSTATUS A status of an operation. Used to update the status field of operations
Examples

Template for clearing an alarm with an additional custom property

# Creation:
10,999,PUT,ALARM,,c8y_MyCustomALarm,status,ALARMSTATUS,CLEARED,c8y_CustomFragment.reason,STRING,
# Usage:
999,Device resolved alarm on its own

Template for creating a custom measurement

# Creation:
10,999,POST,MEASUREMENT,,c8y_CustomMeasurement,,c8y_CustomMeasurement.custom.value,NUMBER,,c8y_CustomMeasurement.custom.unit,STRING,X
# Usage:
999,30.6

Template for updating a property in the device

# Creation:
10,999,PUT,INVENTORY,,false,c8y_Serial,c8y_MyCustomValue,STRING,
# Usage:
999,myDeviceImei,updatedValue

Response templates

The SmartREST 2.0 response templates use the same structure as in SmartREST 1.0.

Field Data type Mandatory Description
messageId String Y Unique ID to reference the template within the collection
base String N A JsonPath prefix that all patterns will use
condition String N A JsonPath that needs to exist in the object to use the pattern
pattern List<String> Y A list of JsonPath that will be extracted from the object and returned to the device

Response templates will be used for every operation and for any request template that defines the response field with true. In each case, the server will try every registered response template, so there might be multiple response lines for a single operation or request.

SmartREST 2.0 will always return a response template if the condition is true (or no condition was defined). Patterns that did not resolve will be returned as empty string.

You should make use of the condition field to control when response templates should be returned.

In the examples below, you can see how to query data and parse custom operations.

Querying data from the device object

Device object:

{
  "id": "12345",
  "type": "myMqttDevice",
  "c8y_IsDevice": {},
  "c8y_Configuration": "val1=1\nval2=2"
}

Template creation:

10,999,GET,INVENTORY,,true
11,888,,c8y_IsDevice,type,c8y_Test,c8y_Configuration

Client publishes:

999,12345

Client receives:

888,myMqttDevice,,"val1=1\nval2=2"
Parsing custom operations

Operation object:

{
  "id": "12345",
  "deviceId": "67890",
  "agentId": "67890",
  "status": "PENDING",
  "c8y_CustomConfiguration": {
    "val1": "1",
    "val2": "2",
    "customValues": ["a", "b", "c"]
  },
  "description": "Send custom configuration"
}

Template creation:

11,111,c8y_CustomConfiguration,deviceId,val1,val2,customValues[*]
11,222,,deviceId,c8y_CustomConfiguration.val1,c8y_CustomConfiguration.val2
11,333,,deviceId,val1,val2,customValues[*]
11,444,c8y_CustomConfiguration,c8y_CustomConfiguration.val3,val1,val2,customValues[*]

Client receives (assuming the ClientId is “myMqttTestDevice”):

111,myMqttTestDevice,1,2,a,b,c
222,myMqttTestDevice,1,2
333,myMqttTestDevice,,,

The template 444 is not returned as the condition does not match the operation.

Querying data from the device object containing key with multiple objects

Device object:

{
  "id": "12345",
  "name": "test",
  "type": "c8y_MQTTdevice",
  "c8y_IsDevice": {},
  "myList": [
      {
          "pid": 12345,
          "type": "test"
      },
      {
          "pid": 123456,
          "type": "test2"
      }
  ]
}

Template creation:

10,999,GET,INVENTORY,,true
11,888,,,"$.myList[*].type"

Client publishes:

999,12345

Client receives:

888,test,test2

Using a default collection

Having the X-ID as part of the topic gives you the freedom to easily use multiple template collections, but adds additional bytes for every message. If anyway the device uses mostly (or completely) a single collection, it makes sense to specify this collection as your default collection.

With a default collection specified, the client can use special topics which do not require the X-ID, and instead the server will use the X-ID previously specified. The topics are s/ud for publishing and s/dd for subscribing.

You can specify one X-ID within your MQTT ClientId (see MQTT implementation).

Your MQTT ClientId could look like this:

d:myDeviceSerial:myDefaultTemplateXID

Info: If you use a default X-ID, you need to include in the ClientId the d: at the beginning to specify that the client is a device.

It is not required that the default template exists at the time of establishing the MQTT connection (it will be verified once the client uses it).

MQTT Static templates

Overview

To ease device integration Cumulocity IoT already supports a number of static templates that can be used by any client without the need to create your own templates. These templates focus on the most commonly used messages for device management purposes.

To use the templates listed below, you need to publish the messages to the topic s/us (t/us for transient processing of published content, q/us for quiescent processing of published content or c/us for CEP processing of published content. Refer to SmartREST > Processing mode in the Reference guide for further information.

You need to subscribe to the topic s/ds to receive operations with the static templates.

Quick reference

Click the commands below to see more information on the respective template. If a parameter is in square brackets, it is optional.

Inventory templates

Measurement templates

Alarm templates

Event templates

Operation templates

Automatic device creation

The topic for static templates supports an automatic creation of devices. Whenever there is no child associated with your MQTT ClientID and you send data, Cumulocity IoT will automatically create a device for the MQTT ClientID. If you want to create the device on your own, your first message must be the device creation. In this case Cumulocity IoT will create the device from the template.

The automatic creation of devices is also supported for 1st level child devices. For child devices on a deeper level, you must use the template for creating a child device by sending it to the topic of the child device under the one you want to place the new child.

Handling non-mandatory parameters

If a parameter is not declared as mandatory, the client can send an empty string in that place.

100,,myType

Tailing commas is not required. The two lines below result in the same message.

100,,
100

Publish templates

The following templates can be used to publish data on the topics s/us as well as t/us. Refer to SmartRest > Processing mode in the Reference guide for more information about the t/ topic for transient data processing.

Inventory templates (1xx)

Device creation (100)

Create a new device for the serial number in the inventory if not yet existing. An externalId for the device with type c8y_Serial and the device identifier of the MQTT clientId as value will be created.

Position Parameter Mandatory Default value
1 device name NO MQTT Device <serialNumber>
2 device type NO c8y_MQTTDevice

Example

100,myDevice,myType
Child device creation (101)

Create a new child device for the current device. The newly created object will be added as child device. Additionally, an externaId for the child will be created with type c8y_Serial and the value a combination of the serial of the root device and the unique child ID.

Position Parameter Mandatory Default value
1 unique child ID YES  
2 device name NO MQTT Device <serialNumber>
3 device type NO c8y_MQTTChildDevice

Example

101,uniqueChildId,myChildDevice,myChildType
Get child devices (105)

Trigger the sending of child devices of the device.

Example

105
Clear device’s fragment (107)

Remove one or more fragments from a device.

Position Parameter Mandatory
1… fragmentName YES

Example

107,c8y_Position,c8y_Configuration
Configure Hardware (110)

Update the hardware properties of the device.

Position Parameter Mandatory
1 serialNumber NO
2 model NO
3 revision NO

Example

110,1234567890,myModel,1.2.3
Configure Mobile (111)

Update the mobile properties of the device.

Position Parameter Mandatory
1 imei NO
2 iccid NO
3 imsi NO
4 mcc NO
5 mnc NO
6 lac NO
7 cellId NO

Example

111,1234567890,,54353
Configure Position (112)

Update the position properties of the device.

Position Parameter Mandatory
1 latitude NO
2 longitude NO
3 altitude NO
4 accuracy NO

Example

112,50.323423,6.423423
Set Configuration (113)

Update the configuration properties of the device.

Position Parameter Mandatory
1 configuration NO

Example

113,"val1=1\nval2=2"
Set supported operations (114)

Set the supported operations of the device.

Position Parameter Mandatory
1… List of supported operations NO

Example

114,c8y_Restart,c8y_Configuration,c8y_SoftwareList

>Info: If you want to remove an item from the supported operations list, send a new 114 request with the updated list, e.g. 114, c8y_Restart,c8y_Configuration in order to remove c8y_SoftwareList after the request from the example above.

Set firmware (115)

Set the firmware installed on the device.

Position Parameter Mandatory
1 name NO
2 version NO
3 url NO

Example

115,firmwareName,firmwareVersion,firmwareUrl
Set software list (116)

Set the list of software installed on the device.

Position Parameter Mandatory
1… List of 3 values per software NO
1.1 name NO
1.2 version NO
1.3 url NO

Example

116,software1,version1,url1,software2,,url2,software3,version3
Set required availability (117)

Set the required interval for availability monitoring as an integer value representing minutes. For more information, see Device management library > Device availability > c8y_RequiredAvailability in the Reference guide. This will only set the value if it does not exist. Values entered, e.g. through the UI, are not overwritten.

Position Parameter Mandatory
1 Required interval NO

Example

117,60
Set supported logs (118)

Set the supported logs of the device.

Position Parameter Mandatory
1… List of supported logs NO

Example

118,ntcagent,dmesg,logread
Set supported configurations (119)

Set the supported configurations of the device.

Position Parameter Mandatory
1… List of supported configurations NO

Example

119,modbus,system
Set currently installed configuration (120)

Set currently installed configuration of the device.

Position Parameter Mandatory Default value
1 Configuration type YES  
2 Configuration file download URL YES  
3 File name NO Configuration type
4 Date and time when the configuration was applied NO Current date and time

Example

120,myType,http://www.my.url,config.bin,2020-07-22T17:03:14.000+02:00
Set device profile that is being applied (121)

Set device profile that is being applied to the device.

Position Parameter Mandatory Default value
1 Profile executed YES  
2 Profile ID NO Profile ID from the oldest EXECUTING device profile operation

Example

121,true,8473

Measurement templates (2xx)

Create custom measurement (200)

Create a measurement with a given fragment and series.

Position Parameter Mandatory Default value
1 fragment YES  
2 series YES  
3 value YES  
4 unit NO  
5 time NO Current server time

Example

200,c8y_Temperature,T,25
Create signal strength measurement (210)

Create a measurement of type c8y_SignalStrength.

Position Parameter Mandatory Default value
1 rssi value YES, if 2 not set  
2 ber value YES, if 1 not set  
3 time NO Current server time

Example

210,-90,23,2016-06-22T17:03:14.000+02:00
Create temperature measurement (211)

Create a measurement of type c8y_TemperatureMeasurement.

Position Parameter Mandatory Default value
1 temperature value YES  
2 time NO Current server time

Example

211,25,2016-06-22T17:03:14.000+02:00
Create battery measurement (212)

Create a measurement of type c8y_Battery.

Position Parameter Mandatory Default value
1 battery value YES  
2 time NO Current server time

Example

212,95,2016-06-22T17:03:14.000+02:00

Alarm templates (3xx)

Create CRITICAL alarm (301)

Create a CRITICAL alarm.

Position Parameter Mandatory Default value
1 type YES  
2 text NO Alarm of type alarmType raised
3 time NO Current server time

Example

301,c8y_TemperatureAlarm
Create MAJOR alarm (302)

Create a MAJOR alarm.

Position Parameter Mandatory Default value
1 type YES  
2 text NO Alarm of type alarmType raised
3 time NO Current server time

Example

302,c8y_TemperatureAlarm,"This is an alarm"
Create MINOR alarm (303)

Create a MINOR alarm.

Position Parameter Mandatory Default value
1 type YES  
2 text NO Alarm of type alarmType raised
3 time NO Current server time

Example

303,c8y_TemperatureAlarm
Create WARNING alarm (304)

Create a WARNING alarm.

Position Parameter Mandatory Default value
1 type YES  
2 text NO Alarm of type alarmType raised
3 time NO Current server time

Example

304,c8y_TemperatureAlarm,,2013-06-22T17:03:14.000+02:00
Update severity of existing alarm (305)

Change the severity of an existing alarm.

Position Parameter Mandatory
1 type YES
2 severity YES

Example

305,c8y_TemperatureAlarm,CRITICAL
Clear existing alarm (306)

Clear an existing alarm.

Position Parameter Mandatory
1 type YES

Example

306,c8y_TemperatureAlarm
Clear alarm’s fragment (307)

Remove one or more fragments from an alarm of a specific type.

Position Parameter Mandatory
1 alarmType YES
2… fragmentName YES

Example

307,c8y_TemperatureAlarm,c8y_Position,c8y_Configuration

Event templates (4xx)

Create basic event (400)

Create an event of given type and text.

Position Parameter Mandatory Default value
1 type YES  
2 text YES  
3 time NO Current server time

Example

400,c8y_MyEvent,"Something was triggered"
Create location update event (401)

Create typical location update event containing c8y_Position.

Position Parameter Mandatory Default value
1 latitude NO  
2 longitude NO  
3 altitude NO  
4 accuracy NO  
5 time NO Current server time

Example

401,51.151977,6.95173,67
Create location update event with device update (402)

Create typical location update event containing c8y_Position. Additionally the device will be updated with the same c8y_Position fragment.

Position Parameter Mandatory Default value
1 latitude NO  
2 longitude NO  
3 altitude NO  
4 accuracy NO  
5 time NO Current server time

Example

402,51.151977,6.95173,67
Clear event’s fragment (407)

Remove one or more fragments from an event of a specific type.

Position Parameter Mandatory
1 eventType YES
2… fragmentName NO

Example

407,c8y_MyEvent,c8y_Position,c8y_Configuration

Operation templates (5xx)

Get PENDING operations (500)

Trigger the sending of all PENDING operations for the agent.

Example

500
Set operation to EXECUTING (501)

Set the oldest PENDING operation with given fragment to EXECUTING.

Position Parameter Mandatory
1 fragment YES

Example

501,c8y_Restart
Set operation to FAILED (502)

Set the oldest EXECUTING operation with given fragment to FAILED.

Position Parameter Mandatory
1 fragment YES
2 failureReason NO

Example

502,c8y_Restart,"Could not restart"
Set operation to SUCCESSFUL (503)

Set the oldest EXECUTING operation with given fragment to SUCCESSFUL.

It enables the device to send additional parameters that trigger additional steps based on the type of operation sent as fragment (see Section Updating operations).

Position Parameter Mandatory
1 fragment YES
2… parameters NO

Example

503,c8y_Restart

Subscribe templates

The client can receive the following templates when subscribing to s/ds.

Inventory template

Operations template

Inventory templates (1xx)

Get children of device (106)

List all children of the device.

Position Parameter
1… child

Example

106,child1,child2,child3

Operation templates (5xx)

All operation responses have the same base structure, leading with the message ID and followed by the ID of either the root device or a child which should handle the operation.

Restart (510)

Restart a device.

Example

510,DeviceSerial
Command (511)

Run the command being sent in the operation.

Position Parameter
1 Command text

Example

511,DeviceSerial,execute this
Configuration (513)

Set the configuration being sent in the operation.

Position Parameter
1 configuration

Example

513,DeviceSerial,"val1=1\nval2=2"
Firmware (515)

Install the firmware from the url.

Position Parameter
1 firmware name
2 firmware version
3 url

Example

515,DeviceSerial,myFimrware,1.0,http://www.my.url
Software list (516)

Install the software sent in the operation.

Position Parameter
1… List of 3 values per software
1.1 name
1.2 version
1.3 url

Example

516,DeviceSerial,softwareA,1.0,url1,softwareB,2.0,url2
Measurement request operation (517)

Send the measurements specified by the request name.

Position Parameter
1 request name

Example

517,DeviceSerial,LOGA
Relay (518)

Open or close the relay.

Position Parameter
1 Relay state

Example

518,DeviceSerial,OPEN
RelayArray (519)

Open or close the relays in the array.

Position Parameter
1… List of relay state

Example

519,DeviceSerial,OPEN,CLOSE,CLOSE,OPEN
Upload configuration file (520)

The current configuration is uploaded from Cumulocity IoT to the device.

Example

520,DeviceSerial
Download configuration file (521)

Download a configuration file from the URL.

Position Parameter
1 url

Example

521,DeviceSerial,http://www.my.url
Logfile request (522)

Upload a log file for the given parameters.

Position Parameter
1 Log file name
2 Start date
3 End date
4 Search text
5 Maximum lines

Example

522,DeviceSerial,logfileA,2013-06-22T17:03:14.000+02:00,2013-06-22T18:03:14.000+02:00,ERROR,1000
Communication mode (523)

Change the communication mode.

Position Parameter
1 mode

Example

523,DeviceSerial,SMS
Download configuration file with type (524)

Download a configuration file from the URL with type.

Position Parameter
1 URL
2 configuration type

Example

524,DeviceSerial,http://www.my.url,type
Firmware from patch (525)

Install the firmware from the patch.

Position Parameter
1 firmware name
2 firmware version
3 URL
4 dependency

Example

525,DeviceSerial,firmwareName,1.0,http://www.my.url,dependency
Upload configuration file with type (526)

Configuration is uploaded from Cumulocity IoT to the device with type.

Position Parameter
1 configuration type

Example

526,DeviceSerial,type
Set device profiles (527)

Set the device profiles

Position Parameter
1 firmware marker
1… 5 values of firmware
1.1 firmware name
1.2 firmware version
1.3 firmware URL
1.4 firmware isPatch
1.5 firmware dependency
2 software marker
2… List of 4 values per software
2.1 software name
2.2 software version
2.3 software URL
2.4 software action
3 configuration marker
3… List of 2 values per configuration
3.1 configuration URL
3.2 configuration type

Example

527,DeviceSerial,$FW,firmwareName,1.0,http://www.my.url,true,dependency,$SW,softwareA,1.0,http://www.my.url1,action1,softwareB,2.0,http://www.my.url2,action2,$CONF,http://www.my.url1,type1,http://www.my.url2,type2
Update Software (528)

Update the software installed on the device.

Position Parameter
1… List of 4 values per software
1.1 name
1.2 version
1.3 URL
1.4 action

Example

528,DeviceSerial,softwareA,1.0,url1,install,softwareB,2.0,url2,install

Info: The action can either be install or delete.

When the install action is received, the device agent ensures that the software will appear in the c8y_SoftwareList fragment of the device after it has completed the installation. The agent will also determine if there is a previous version of the software and replace it with the new version, resulting in an update.

When the delete action is received, the device agent ensures that the software will no longer appear in the c8y_SoftwareList fragment of the device after the software update operation has completed.

Cloud Remote Access Connect (530)

Establish tunneling by Remote Access device agent.

Position Parameter
1 hostname
2 port
3 connection key

Example

530,DeviceSerial,10.0.0.67,22,eb5e9d13-1caa-486b-bdda-130ca0d87df8

Updating operations

When using the template to set an operation to status SUCCESSFUL, it supports sending additional parameters to trigger additional calls on the server. The table below shows the operations supporting this feature and what will be done with the parameters.

Fragment Parameters Action triggered
c8y_Command result Result will be added to operation
c8y_RelayArray relay states Device object will be updated with the states
c8y_CommunicationMode no parameter needed Device object will be updated with the mode
c8y_LogfileRequest file url File url will be added to operation
c8y_DownloadConfigFile (optional) timestamp Device object will be updated with the ID of the configuration dump and the timestamp (or server time)

Handling IDs

Concept of ID-less communication

The MQTT implementation of Cumulocity IoT is specifically designed for device communication. Therefore, it tries to remove as much unnecessary logic from the client side as possible.

Using the REST (or SmartREST) protocol requires to know the ID of every object, alarm and operation you want to update. Hence, a client needs to keep a state of these IDs. For example, if it creates an alarm, it needs to know the ID of the alarm so it can clear it afterwards.

With the MQTT implementation we want to reduce the logic required on the device to do such actions and move the logic to the server.

Example 1: ID of the device

You need to know the ID of the device object to update data in it via REST. Also, this ID is required for every other data that needs to be associated with the device, e.g. the source in a measurement, alarm or event.

To remove the necessity of persisting the ID on the device, Cumulocity IoT offers the identity API where you can link external IDs (e.g. a serial number) to the object, so you can query the ID any time.

A typical device start looks like this:

REST Receiving Device ID

With MQTT, we automatically use the identity API with the MQTT clientId. This removes the necessity to tell the ID to the device, and because the client sends also the other data on this connection, we can associate every measurement, alarm, event, etc. with the correct device.

MQTT Automatically Resolve ID

Example 2: ID of alarms

When a client creates an alarm using the REST API, it needs to ensure that it gets the ID of the alarm that was generated by Cumulocity IoT in return.

The client will need this ID to later update the alarm, e.g. to status CLEARED, if the alarm is not active anymore.

REST Handling Alarms

In Cumulocity IoT, a device can only have a single alarm per type in status ACTIVE. If it creates another alarm with the same type, it will get de-duplicated. Therefore, the MQTT implementation uses the type of an alarm as identifier. The client only needs to send which type of alarm has been resolved, and the server will find the correct alarm object.

MQTT Handling Alarms

JSON via MQTT

This section describes the JSON payload format that can be used with the Cumulocity IoT MQTT implementation.

Compared to SmartREST 2.0 – which only works with fixed templates – JSON’s support for MQTT was designed to combine the payload flexibility of our REST API with the low protocol overhead of MQTT.

The SmartREST way should still be the preferred way if it is important to reduce your payload to the minimum (mobile traffic, low capability device).

Topic structure

The topic structure in JSON MQTT is quite similar to the REST endpoints. The main difference is in the additional action part which is included in the topic.

To publish messages:

 <api>/<resource>/<action>/<resource_id>

To publish messages in transient mode:

t/<api>/<resource>/<action>/<resource_id>

To publish messages in quiescent mode:

q/<api>/<resource>/<action>/<resource_id>

To publish messages in CEP mode:

c/<api>/<resource>/<action>/<resource_id>

Info: <resource_id> is not required for every <action>. See the examples below.

Refer to Processing mode for more information about transient, quiescent and CEP data processing.

Topic actions

The action in the topic corresponds to the HTTP methods combined with the content-type header.

The following actions are available:

Supported endpoint

The current JSON MQTT implementation does not cover all SmartREST 2.0 operations, so for example the whole device bootstrap process has to be done using SmartREST 2.0.

The following endpoints and actions are supported:

Endpoint create createBulk update delete
event/events x x x x
alarm/alarms x x x  
measurement/measurements x x   x
inventory/managedObjects x   x  
inventory/managedObjects/<DeviceID>/childDevices x      

If the operation is not supported, a proper error message will be sent to the error topic.

For all of the above endpoints, you can use the same payload like in the REST API. The only difference is in the “source” field - in REST this field is mandatory while for JSON MQTT there is no need to set the device ID here. The source device ID will automatically be resolved based on the MQTT client ID. This value will always be used no matter if something is already defined there.

Examples

Create new event

Publish a message on topic /event/events/create with payload:

{
  "type": "TestEvent",
  "text": "sensor was triggered",
  "time": "2014-03-03T12:03:27.845Z"
}

Create many events

Publish a message on topic /event/events/createBulk with payload:

{
  "events": [
    {
      "type": "TestEvent1",
      "text": "sensor was triggered",
      "time": "2014-03-03T12:03:27.845Z"
    },
    {
      "type": "TestEvent2",
      "text": "sensor was triggered",
      "time": "2014-03-04T12:03:27.845Z"
    }
  ]
}

Update event

Publish a message on topic /event/events/update/<event_id> with payload:

{
  "text": "new text"
}

Delete event

Publish a message on topic /event/events/delete/<event_id> with empty payload.

Create a measurement data point

Publish a message on topic measurement/measurements/create with payload:

{
  "type": "c8y_TemperatureMeasurement",
  "time": "2021-09-06T17:35:14.000+02:00",
  "c8y_TemperatureMeasurement": {
  	"T": {
      	"value": 20,
          "unit": "C"
    }
  }
}

Error handling

Use the error topic to subscribe for errors related to the JSON MQTT implementation. In case of invalid payload, wrong topic or any other exception, a notification will be published on this topic. The payload is in JSON format. Besides a standard error message, it also contains a message ID which helps the client in finding out which exact message was failing.

Example payload:

{
  "error": "undefined/validationError",
  "message": "Following mandatory fields should be included: severity,text,time",
  "messageId": 3
}

Receiving operations

A notification client can subscribe to the devicecontrol/notifications topic to receive notifications of newly created operations. Initially upon subscription, all operations which are not yet forwarded will be published.

Additionally, it contains an External ID, so the client can identify for which child the operation is executed.

Example notification:

{
  "agentId": "1",
  "creationTime": "2018-05-17T07:33:15.555Z",
  "delivery": {
    "log": [

    ],
    "status": "PENDING",
    "time": "2018-05-17T07:33:15.575Z"
  },
  "deviceId": "2",
  "id": "123",
  "status": "PENDING",
  "c8y_Command": {
    "text": "Do something"
  },
  "description": "Execute shell command",
  "externalSource": {
    "externalId": "3",
    "type": "c8y_Serial"
  }
}