Device integration using MQTT

Overview

The MQTT implementation of Cumulocity IoT provides the following benefits:

The MQTT section is structured as follows:

Also see our SmartREST documentation.

This section does not describe the basics of MQTT communication. If you are unfamiliar with MQTT, we recommend you to consult one of the numerous introductions in the Internet. Some references can be found on the MQTT website.

Info
For all MQTT connections to the platform, the maximum accepted payload size is 16184 bytes (16KiB), which includes both message header and body. The header size varies, but its minimum is 2 bytes.

MQTT implementation

This section will list the implementation details for the MQTT protocol. The Cumulocity IoT implementation supports MQTT Version 3.1.1.

Connecting via MQTT

Cumulocity IoT supports MQTT both via TCP and WebSockets. As URL you can use the domain of the instance in the format mqtt.<instance_domain> (for example mqtt.cumulocity.com) or your tenant domain (for example mytenant.cumulocity.com/mqtt).

Available ports:

  TCP WebSockets
SSL 8883 443
no SSL 1883 80
Info
Port 80 is deactivated in cloud systems.

Port 8883 supports two types of SSL: two-way SSL using certificates for client authorization and one-way SSL using username and password for client authorization. The two-way SSL support is enabled by default. To disable it please contact product support.

Info
To use WebSockets you must connect to the path /mqtt and follow the MQTT standard for WebSocket communication.

SmartREST payload

The Cumulocity IoT MQTT implementation uses SmartREST as a payload. SmartREST is a CSV-like message protocol that uses templates on the server side to create data in Cumulocity IoT. It incorporates the highly expressive strength of the REST API but replaces JSON with comma-separated values (CSV) to avoid the complexity of JSON parsing for embedded devices. Additionally, the simple and compact syntax of CSV renders it highly efficient for IoT communication via mobile networks. It can save up to 80% of mobile traffic compared to other HTTP APIs.

Info
For all MQTT connections to the platform, the maximum accepted payload size is 16184 bytes, which includes both message header and body. The header size varies, but its minimum is 2 bytes.

SmartREST basics

A SmartREST message is a single row in which each parameter is separated by comma. The first parameter is an ID that defines the message. You can send multiple messages in a single publish by using a line break between messages.

SmartREST escaping

The CSV (comma-separated values) format is used for communication with the SmartREST endpoint. The following rules must be followed to ensure a frictionless communication.

The same escaping rules apply to messages that will be sent from the server to the client.

Publish example:

100,"This value, needs escaping",This value does not need escaping

Subscribe example:

511,myDeviceSerial,"execute this\nand this\nand \"this\""
Info
\n does not create a new line in the output (for example console, UI); to achieve this, a new line character (ASCII 0A) needs to be used.

Device hierarchies

MQTT sessions are linked to a single device, but this device can have a freely configurable device hierarchy below it.

All children require a unique ID defined when creating the device. We recommend you to use a combination of the unique ID of the root device and a unique ID within the hierarchy.

To create data for a child instead of the root device, the unique ID of the child is added as another section in the topic (for example s/us/myChildDeviceIdentifier).

The client will automatically receive operations for every child in the hierarchy by subscribing to the respective topic. It is not required to subscribe for each child.

Every operation received will contain the template ID followed by the ID of the device/child for which the operation was created (followed by other operation parameters).

MQTT features

MQTT authentication

The communication with Cumulocity IoT employing MQTT supports authentication in two ways:

Troubleshooting

A device sends correct username and password, but incorrect certificate at the same time

If the platform is configured to support two-way SSL, your devices have a configured keystore with invalid certificates, and you want to use basic authorization, we recommend you to turn off sending certificates during connection. Certificates may be invalid because they expired or the root certificate is not uploaded to the platform. Turn off certificate sending in the device’s software. If that is not possible, to make the connection work, check the following:

MQTT ClientId

The MQTT ClientId is a field to uniquely identify each connected client. The Cumulocity IoT implementation also uses the ClientId to link the client directly to a device. Therefore, the following format should be used for the ClientId:

connectionType:deviceIdentifier:defaultTemplateIdentifier

Field Mandatory Description
connectionType NO Indication of connection type
default: d (device)
deviceIdentifier YES A unique identifier for your device, for example, IMEI, Serial number
defaultTemplateIdentifier NO Check SmartREST 2.0 > MQTT static templates for more information about template identifiers

For the simplest version of a client, the MQTT clientId can just be the deviceIdentfier. It will automatically be interpreted as device connection.

Important
The colon character has a special meaning in Cumulocity IoT. Hence, it must not be used in the deviceIdentifier.

Examples of ClientIds:

mySerialNumber
d:mySerialNumber
d:mySerialNumber:myDefaultTemplate

The uniqueness of the MQTT ClientId is determined only by the deviceIdentifier. Therefore, from the above examples only one client can be connected at the same time.

During an SSL connection with certificates, the deviceIdentifier must match the ‘Common Name’ of the used certificate (first certificate in the chain, which is provided by the device).

MQTT Quality of Service (QoS)

The Cumulocity IoT implementation supports all 3 levels of MQTT QoS:

For subscriptions to the operation or error topics, we will deliver all messages in the QoS which the client defined when subscribing to the topic.

MQTT clean session

MQTT clients can set the clean session flag to “0” (false). This will ensure that in case the client disconnects, your subscription will still work and when you reconnect the client will receive the missed messages.

Info
Cumulocity IoT requires clean session to be set to “1” (true). Currently we cannot guarantee that disabling clean session will work reliably, hence we recommend you to always enable clean session.

MQTT retained flag

In the current Cumulocity IoT implementation, subscriptions to topics where devices publish data are not allowed. Publishing data with the retained flag on this topic is allowed but has no practical difference to sending it without the flag. Messages published by Cumulocity IoT like operations and errors do not contain the retained flag.

MQTT last will

In MQTT, the “last will” is a message that is specified at connection time and that is executed when the client loses the connection. For example, using 400,c8y_ConnectionEvent,"Device connection was lost." as last will message and s/us as last will topic, raises an event whenever the device loses the connection.

Info
The execution of the “last will” updates the device availability.

MQTT return codes

When there is an MQTT error, the platform responds with a CONNACK message with a non-zero return code. This message is the first clue that there is a problem. Such a return code can be treated similarly to REST API HTTP codes, such as 401. They can be returned because of an unexpected error, lack of permissions, and so on.

CONNACK is not only a response to a CONNECT message, but also a way to signal errors that occurred in the platform. Therefore, it is possible to receive this message a second time during a normal connection, and without a direct action. It is also a way to signal a closing connection, as most MQTT clients treat CONNACK with a code other than 0 like the connection needs to be closed. See the details below.

The table below shows the list of errors returned by Cumulocity IoT:

Code Canonical message Troubleshooting
0 Connection accepted No issue, connection is working.
1 Connection refused, unacceptable protocol version Unsupported version of the MQTT protocol. Currently, Cumulocity IoT only allows 3.1 and 3.1.1.
2 Connection refused, identifier rejected ClientId is not accepted by the platform.
3 Connection refused, Server unavailable General platform side error, used on internal errors and unknown authorization problems.
Can be received on network issues.
The error should be temporary and independent of device state, therefore the usual solution to this is to try again later.
4 Connection refused, bad username or password Incorrect credentials (wrong username and/or password, but not on empty password). This error is never returned when authenticating with certificates.
5 Connection refused, not authorized Mostly a device side related problem, used when the device doesn’t have permissions or is doing something forbidden. For example, if the client sends malformed messages or tries to execute an operation without authenticating first, such as publishing a message.
Thrown on any issue with certificate authentication (for example, wrong common name, failed auto registration).
Also thrown on general issues with receiving device data or some other authorization problem related to the device state on the platform. For example, device managed object problems, or the sudden removal of permissions. In this situation it may be required to take action on the platform to investigate and apply a fix.
When clientId is too long the user can receive this error when using 3.1 version of MQTT. This can happen if clientId has 24 characters or more.
Lastly, it can also be thrown on unexpected exceptions like performance issues, especially during connection. Therefore it is a good approach to repeat the connection a few times to overcome temporary performance issues.

Refer to MQTT Version 3.1.1 > 3.2 CONNACK - Acknowledge connection request for details on the official MQTT connection return codes.

Debugging

To support developers during development, it is possible to subscribe to the topic s/e. On this topic the device can retrieve debug and error messages that occur during a publish from the device.

Info
This topic is purely designed to support the development of clients. It is not recommended to always subscribe to this channel as the messages are verbose and can significantly increase the data usage. Also, you should not use this topic to trigger actions of the device based on what you receive on the topic. It is not a response channel.

MQTT broker certificates

MQTT broker uses the certificates which are assigned to the main environment domain. MQTT broker always sends these certificates during TLS handshake to devices. Moreover, Enterprise tenants are not able to customize MQTT broker certificates via the SSL Management feature.

Device certificates

Overview

Devices can authenticate against the Cumulocity IoT platform using X.509 client certificates.

Devices can communicate using the MQTT interface of the platform, but MQTT over WebSocket is not supported. The Cumulocity IoT platform expects devices to connect using SSL on port 8883.

Each tenant individually defines whom it trusts by uploading the base CA certificate.

Devices connecting to the platform with certificates do not need to provide the tenant ID, username and password. Authentication information will be obtained from the certificates.

General requirements for connecting devices with certificates

Registering devices using certificates

Cumulocity IoT supports two ways to register devices which will be able to connect using certificates:

Auto registration

The user for the device will be created during the first MQTT call, if a device certificate is derived from a trusted certificate which was uploaded to the Cumulocity IoT platform with a flag autoRegistrationEnabled with a value of true. Auto-registration needs to be activated for the uploaded certificate. If auto-registration is not activated it is required to use the bulk registration (see below). To manage the auto registration field of uploaded certificates in the UI refer to Device Management > Managing device data > Managing trusted certificates.

Bulk registration

The user for the device can also be created via the standard bulk registration in Device Management.

The CSV file used in bulk registration should meet the requirements described in Create a bulk device credentials request in the Cumulocity IoT OpenAPI Specification. Moreover, it is required that the CSV file has an additional column AUTH_TYPE with value CERTIFICATES, and that the column CREDENTIALS is either not present or has an empty value.

Single registration

Single registration is not supported for devices which are going to use certificates for authentication.

Info
During device registration, the device user is created, which is necessary for device communication with the platform.

JWT token retrieval

A device which is authenticated by certificates and connected to the Cumulocity IoT platform can receive a token which can later be used to authenticate HTTP requests.

71,<<Base64 encoded JWT token>>

A device token lifetime can be configured using tenant options with a category of oauth.internal and a key of device-token.lifespan.seconds. The default value is 1 hour. The minimum allowed value is 5 minutes. Refer to the Tenant API in the Cumulocity IoT OpenAPI Specification for more details.

A device can fetch a new device token before the old one expires, if it request a JWT token after half of the token’s lifetime has passed.

Introduction to X.509 certificates

X.509 is a standard that defines public key certificates, which are commonly used in the SSL protocol to provide secure connection and data transfer. Version 3 is up-to-date since 1995.

The general purpose of an X.509 certificate is to bind an identity to a pair of keys: the public key is publicly known as a part of the certificate while the private key is known only by the certificate owner. Such pairs of keys should be created with an asymmetric-key algorithm, which is considered secure nowadays. An example of such an algorithm is RSA with at least 2048 bits key size. A size of 1024 bits or lower is no longer considered safe. A private key can be used in two ways:

Every certificate can be self-signed or can be signed by another certificate. To tell if the certificate is self-signed you can look at the “Issuer Name” and “Subject Name” fields of the certificate. If they are the same then it means that it is a self-signed certificate, otherwise the certificate claims to be signed by the issuer. To verify if the issuer really signed the certificate, you must check the “Signature” field of the certificate. After decryption with the issuer’s public key, the signature should match the data of the signed certificate. Signing a certificate by another, means that if the issuer’s certificate is trusted then the signed certificate also can be trusted.

For example if a platform trusts the customer certificate and that customer has 20 devices with individual certificates, then he does not have to upload each one of them.

If these device certificates are signed by the customer certificate, then the platform should trust them too. In this case, every device should send not only its own certificate, but the whole chain of certificates (so-called chain of trust) during the SSL handshake.

The chain of certificates starts with the one belonging to the device, through all used intermediate certificates until it reaches the CA certificate trusted by the platform. Usually the chain of certificates does not have to contain the trusted CA certificate, so it can end with the certificate signed directly by the CA. However, in the Cumulocity IoT platform it is also required to provide the trusted CA certificate in the chain of certificates.

Providing the chain of certificates lets the platform verify the signatures of every certificate in the chain to make sure that the device certificate is signed directly or indirectly by the trusted certificate. The chain of the certificates can differ in length, so if the platform trusts certificate A and certificate B is signed by A, and certificate C is signed by B, then certificate C will also be trusted. However, there are a few things to keep in mind:

The structure of an X.509 certificate in version 3 looks like this:

To show how the authentication with X.509 certificates works, there is an example of the simplified mutual SSL handshake:

Simplified mutual SSL handshake

A server knows that the client is the owner of the certificate it sent, after the server has decrypted the encrypted copy of the message using the client’s public key. The client knows that the server is the owner of the certificate it sent, if the server uses the correct session key, because that means that the server decrypted it with its private key. In basic authentication with username and password, the password must be sent over the network to authenticate the user. When certificates are used, the private key is never sent, which makes the use of the certificates much more secure than the basic authentication with username and password.

There are 3 important terms related to X.509 certificates, which everyone should know who wants to start working with them:

Both keystore and truststore can be stored in the same file, but for security reasons we recommended you to keep them separately. The truststore contains public data and the keystore contains the private key, which only to the owner should know. The most popular formats for these files are:

Generating and signing certificates

To generate certificates we use the OpenSSL toolkit. If you do not have it already installed, then you can download if from the website: https://www.openssl.org/source/

Creating a self-signed CA certificate

  1. Create a directory for the root certificate and the signing configuration, for example:

    mkdir /home/user/Desktop/caCertificate

  2. Go to the created directory and create a configuration file for your CA certificate:

    touch caConfig.cnf

  3. Create a database file for keeping the history of certificates signed by the CA:

    touch database.txt

  4. Create a serial file with initial serial number, which will be used to identify signed certificates. After assigning this serial to the signed certificate, the value in this file will be automatically incremented:

    echo 1000 > serial

  5. Create subdirectories for signed certificates and the certificate revocation list:

    mkdir deviceCertificates crl

  6. Fill in the configuration file. This is the example configuration, which can be used for tests after changing the directory dir to your own. If you want to use it in the production environment then please consult it first with some security specialist:

    [ ca ]
    default_ca = CA_default
    [ CA_default ]
    # Directory and file locations.
    dir               = /home/user/Desktop/caCertificate
    certs             = $dir # directory where the CA certificate will be stored.
    crl_dir           = $dir/crl # directory where the certificate revocation list will be stored.
    new_certs_dir     = $dir/deviceCertificates # directory where certificates signed by CA certificate will be stored.
    database          = $dir/database.txt # database file, where the history of the certificates signing operations will be stored.
    serial            = $dir/serial # directory to the file, which stores next value that will be assigned to signed certificate.
    
    # The CA key and CA certificate for signing other certificates.
    private_key       = $dir/caKey.pem # CA private key which will be used for signing certificates.
    certificate       = $dir/caCert.pem # CA certificate, which will be the issuer of signed certificate.
    
    default_md        = sha256 # hash function
    default_days      = 375 # default number of days for which the certificate will be valid since the date of its generation.
    preserve          = no # if set to 'no' then it will determine the same order of the distinguished name in every signed certificate.
    policy            = signing_policy # the name of the tag in this file that specifies the fields of the certificate. The fields must be filled in or even match the CA certificate values to be signed.
    
    # For certificate revocation lists.
    crl               = $crl_dir/caCrl.pem # CA certificate revocation list
    crlnumber         = $crl_dir/crlnumber # serial, but for the certificate revocation list
    crl_extensions    = crl_ext # the name of the tag in this file, which specifies certificates revocation list extensions, which will be added to the certificate revocation by default.
    default_crl_days  = 30 # default number of days for which the certificate revocation list will be valid since the date of its generation. After that date it should be updated to see if there are new entries on the list.
    
    [ req ]
    default_bits        = 4096 # default key size in bits.
    distinguished_name  = req_distinguished_name # the name of the tag in this file, which specifies certificates fields description during certificate creation and eventually set some default values.
    string_mask         = utf8only # permitted string type mask.
    default_md          = sha256 # hash function.
    x509_extensions     = v3_ca # the name of the tag in this file, which specifies certificates extensions, which will be added to the created certificate by default.
    
    # descriptions and default values of the created certificate fields.
    [ req_distinguished_name ]
    countryName                     = Country Name (2 letter code)
    stateOrProvinceName             = State or Province Name
    localityName                    = Locality Name
    organizationName                = Organization Name
    organizationalUnitName          = Organizational Unit Name
    commonName                      = Common Name
    emailAddress                    = Email Address
    
    # A default value for each field can be set by adding an extra line with field name and postfix "_default". For example: "countryName_default = PL". If you add this line here, then leaving country name empty during certificate creation will result in the value "PL" being used. If the default value was specified there, but during certificate creation you do not want to use this value, then instead use "." as the value. It will leave the value empty and not use the default.
    
    # default extensions for the CA certificate.
    [ v3_ca ]
    subjectKeyIdentifier = hash # subject key value will be calculated using hash funtion. It's the recommended setting by PKIX.
    authorityKeyIdentifier = keyid:always,issuer # The subject key identifier will be copied from the parent certificate. It's the recommended setting by PKIX.
    basicConstraints = critical, CA:true, pathlen:10 # "critical" specifies that the extension is important and must be read by the platform. CA says if it is the CA certificate so it can be used to sign different certificates. "pathlen" specifies the maximum path length between this certificate and the device certificate in the chain of certificates during authentication. Path length is set here only to show how it is done. If you do not want to specify max path length, you can keep only the "basicConstraints = critical, CA:true" part here.
    keyUsage = digitalSignature, cRLSign, keyCertSign # specifies permitted key usages.
    
    # Default extensions for the device certificate. This tag is not used directly anywhere in this file, but will be used from the command line to create signed certificate with "-extensions v3_signed" parameter.
    [ v3_signed ]
    subjectKeyIdentifier = hash
    authorityKeyIdentifier = keyid,issuer
    basicConstraints = critical, CA:false
    keyUsage = nonRepudiation, digitalSignature, keyEncipherment
    
    # default extensions for certificate revocation list
    [ crl_ext ]
    authorityKeyIdentifier=keyid:always
    
    # Policy of certificates signing. It specifies which certificate fields must be filled in during certificate creation. There are three possible values here:
    # "optional" - field value can be empty
    # "supplied" - field value must be filled in
    # "match" - signed certificate field value must match the CA certificate value to be created
    [ signing_policy ]
    countryName             = optional
    stateOrProvinceName     = optional
    organizationName        = optional
    organizationalUnitName  = optional
    commonName              = supplied # every certificate should have a unique common name, so this value should not be changed.
    emailAddress            = optional
    
  7. Create a private key with aes256 encryption and a length of at least 2048 bits. You will also be asked to set the password for the key during its creation:

    openssl genrsa -aes256 -out caKey.pem 4096

  8. Create a self-signed certificate using specifications from the configuration file. The “days” parameter says how long this certificate will be valid since the generation, so set it as you prefer:

    openssl req -config caConfig.cnf -key caKey.pem -new -x509 -days 7300 -sha256 -extensions v3_ca -out caCert.pem

  9. You can print the created certificate with the command:

    openssl x509 -noout -text -in caCert.pem

Creating an intermediate certificate

The intermediate certificate is signed by the CA certificate, but will also be used to sign device certificates. This step is optional. If you are fine with signing all the device certificates with one common CA certificate, then you can skip this step. However, if you need some certificates between the CA certificate and the device certificate then it is the way to go. Keep in mind that in the Cumulocity IoT cloud the maximum length of the chain of certificates is currently restricted to 2 for security reasons, so you cannot use any intermediate certificate between your CA certificate and the device certificate there. However, this behaviour can be changed for dedicated installations by changing a platform wide configuration setting and increasing the allowed maximum length of the chain of certificates to more than 2. To create the intermediate certificate:

  1. Create a new directory for intermediate certificates inside the caCertificate path: mkdir intermediateCertificate
  2. Go to this directory and create a configuration file for your intermediate certificate: touch intermediateConfig.cnf
  3. Create a database file for keeping the history of certificates signed by the intermediate certificate: touch database.txt
  4. Create a serial file with initial serial number, which will be used to identify signed certificates: echo 1000 > serial
  5. Create subdirectories for the signed certificates and the certificate revocation list: mkdir deviceCertificates crl
  6. Fill in the configuration file as in the CA configuration, but remember to change the general directory (“dir”) to your intermediateCertificate folder. Also don’t forget to change private_key, certificate and crl names from current with “ca” prefix to “intermediate” prefix (for example: caKey.pem -> intermediateKey.pem), because files with this prefix will be generated in the next steps.
  7. Generate a private key for the intermediate certificate: openssl genrsa -aes256 -out intermediateKey.pem 4096
  8. Generate a certificate signing request: openssl req -config intermediateConfig.cnf -new -sha256 -key intermediateKey.pem -out intermediateCsr.pem
  9. Go to the caCertificate directory and generate a signed intermediate certificate. You must use the CA configuration here, because the private key specified in the configuration file will be used to sign the certificate: openssl ca -config caConfig.cnf -extensions v3_ca -days 3650 -notext -md sha256 -in intermediateCertificate/intermediateCsr.pem -out intermediateCertificate/intermediateCert.pem
  10. Verify if the generated certificate is correctly signed by CA: openssl verify -CAfile caCert.pem intermediateCertificate/intermediateCert.pem

Creating a device certificate signed by CA or intermediate

  1. Go to the directory of your caCertificate or intermediateCertificate depending on which one is used to sign the device certificate.
  2. Generate the private key for the new certificate: openssl genrsa -aes256 -out deviceCertificates/deviceKey.pem 4096
  3. Generate the certificate signing request (change “caConfig.cnf” to “intermediateConfig.cnf” if you are in the intermediateCertificate directory): openssl req -config caConfig.cnf -new -sha256 -key deviceCertificates/deviceKey.pem -out deviceCertificates/deviceCsr.pem Remember that the commonName of the device certificate, which you will be asked to provide in the console, must match the ClientId of the device during the connection.
  4. Generate the certificate signed by the CA or intermediate (change “caConfig.cnf” to “intermediateConfig.cnf” if you are in the intermediateCertificate directory): openssl ca -config caConfig.cnf -extensions v3_signed -days 365 -notext -md sha256 -in deviceCertificates/deviceCsr.pem -out deviceCertificates/deviceCert.pem
  5. Verify if the generated certificate is correctly signed by CA or intermediate (change “caCert.pem” to “intermediateCert.pem” if you are in the intermediateCertificate directory): openssl verify -partial_chain -CAfile caCert.pem deviceCertificates/deviceCert.pem

Creating the chain of certificates

Go into your caCertificate directory.

If you created the CA certificate, which was used to sign the intermediate certificate and then the intermediate certificate was used to sign the device certificate, then you create your chain with the command:

cat intermediateCertificate/deviceCertificates/deviceCert.pem intermediateCertificate/intermediateCert.pem caCert.pem > intermediateCertificate/deviceCertificates/deviceCertChain.pem

If you are not using the intermediate certificate then the command is:

cat deviceCertificates/deviceCert.pem caCert.pem > deviceCertificates/deviceCertChain.pem

If you are using multiple intermediate certificates between the CA certificate and the device certificate, then remember that you must keep the correct order during the chain creation (Every certificate must be followed by the certificate, which it is signed by).

Creating keystore and truststore

  1. Go into your deviceCertificates directory with the device’s private key and the generated chain of certificates. If you are using an intermediate certificate between the CA certificate and the device certificate then it will be the caCertificate/intermediateCertificate/deviceCertificates path, otherwise it will be caCertificate/deviceCertificates. Create keystore using the generated chain of certificates and the private key of the device: openssl pkcs12 -export -name devicekeyentry -inkey deviceKey.pem -in deviceCertChain.pem -out deviceKeystore.pkcs12
  2. If you want to convert your keystore to JKS format then you would need the Java Keytool which is usually downloaded together with Java Development Kit: keytool -importkeystore -srckeystore deviceKeystore.pkcs12 -srcstoretype PKCS12 -destkeystore deviceKeystore.jks -deststoretype JKS
  3. If you do not have the server certificate, get it by the command: openssl s_client -showcerts -connect <cumulocity url>:<mqtt mutual ssl port (currently 8883, but that can be changed in the future)> | openssl x509 -outform PEM > serverCertificate.pem
  4. Now you can create a truststore, which will contain the server certificate. It must be created with Java Keytool (openssl does not support creating truststore, so if you don’t want to use Java keytool then you must keep every trusted certificate in a separate PEM file). Remember that alias is the unique identifier for every keystore or truststore entry. It means that if you want to add a second trusted certificate to the same truststore then you must change the alias from servercertificate in the command below to some other name:
    • In PKCS12 format: keytool -importcert -noprompt -keystore deviceTruststore.pkcs12 -alias servercertificate -file serverCertificate.pem
    • In JKS format: keytool -import -file serverCertificate.pem -alias servercertificate -keystore deviceTruststore.jks
  5. Optionally, instead of creating a new file for the truststore, you can add the trusted certificates to your created keystore and store everything in one file, which is not the recommended solution:
    • If your keystore is in the PKCS12 format: keytool -importcert -noprompt -keystore deviceKeystore.pkcs12 -alias servercertificate -file serverCertificate.pem
    • If your keystore is in the JKS format: keytool -import -file serverCertificate.pem -alias servercertificate -keystore deviceKeystore.jks
  6. You can check the content of your keystore (or truststore) with the command: keytool -list -v -keystore deviceKeystore.jks

How to test created certificates with MQTT.fx client

Keystore and truststore

Generate a keystore and a truststore as described in Generating and signing certificates if you didn’t do it yet.

Upload your CA certificate

Upload your CA (or intermediate) certificate to the platform. This operation will add your uploaded certificate to the server’s truststore. It can be done in two ways, both of which have a role requirement of either ROLE_TENANT_ADMIN or ROLE_TENANT_MANAGEMENT_ADMIN:

Via UI:

  1. Open the Device Management application, then navigate to the Management tab and select Trusted certificates.
  2. Drop your caCert.pem (or intermediateCert.pem).
  3. Check the auto-registration field.
  4. Click on the certificate status to set it to Enabled.
  5. Insert some custom name.
  6. Click Add certificate.

After completing all the steps except adding the certificate, the form should look like this:

Trusted certificate addition

Then the added certificate should be visible:

Trusted certificate added

Via REST:

  1. Display your CA (or intermediate) certificate, which you want to upload to the Cumulocity IoT platform and copy its PEM value, which starts with “—–BEGIN CERTIFICATE—–” and ends with “—–END CERTIFICATE—–” (including the hyphens). Remove new line symbols (\n) if they were added automatically at the end of each line: openssl x509 -in caCert.pem -text
  2. Send it to the platform via POST request:
    POST /tenant/tenants/<TENANT_ID>/trusted-certificates
    Host: https://<TENANT_DOMAIN>/
    Authorization: Basic <YOUR_AUTHENTICATION>
    Content-Type: application/json
    {
    	"status" :  "ENABLED",
    	"name" : "certificateName",
    	"autoRegistrationEnabled" : "true",
    	"certInPemFormat" : "<CERT_PEM_VALUE>"
    }

Install and configure the MQTT client

  1. Download and install the newest MQTT.fx client from: http://www.jensd.de/apps/mqttfx/
  2. In MQTT.fx click Extras at the top and then Edit Connection Profiles.
  3. Edit the connection profiles like so:
    • Insert the Cumulocity URL in the Broker address line.
    • Insert the SSL port in the Broker port line.
    • In the Client ID field, insert the common name of your device certificate.
    • Select SSL/TLS as the authentication type.
    • Click Enable SSL/TLS.
    • Select TSL v1.2 or TSL v1.3.
    • Select Self signed certificates in keystores
    • In Keystore File insert the path to your deviceTruststore file with either JKS or PKCS12 format.
    • In Trusted Keystore Alias insert servercertificate or a different value if you provided a different alias in step 3 above.
    • In Trusted Keystore Password insert the password, which you created during the deviceTruststore file creation.
    • In Client Keystore insert the path to your deviceKeystore file with either JKS or PKCS12 format.
    • In Client Keystore Password insert the password you created during the deviceKeystore creation.
    • In Client KeyPair Alias insert devicekeyentry or a different value if you provided a different alias in the “-name” parameter during the step about keystore creation in Generating and signing certificates.
    • In Client KeyPair Password insert the password, which you created during the deviceKey.pem creation.
    • The PEM formatted field should be checked.
  4. Save and close the settings.
  5. Select the edited profile and click connect.
  6. You should be succesfully connected and the buttons Disconnect, Publish and Subscribe should be active now. This means that your connection with the certificates work correctly.

The connection settings should look like this:

MQTT.fx configuration

MQTT example client

The code of the Cumulocity IoT MQTT example client implemented in Java, which connects to the platform using x.509 certificates, is available here: https://github.com/SoftwareAG/cumulocity-examples/tree/develop/mqtt-client. This example client uses the implementation of Eclipse Paho, which is described in detail on their website: https://www.eclipse.org/paho/index.php?page=documentation.php.

Here is an example that shows how to add the needed dependency in Maven to use Eclipse Paho client:

<dependency>
    <groupId>org.eclipse.paho</groupId>
    <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
    <version>${paho.version}</version>
</dependency>

Then the instance of the MQTT client can be created with a single line:

MqttClient mqttClient = new MqttClient(BROKER_URL, "d:" + CLIENT_ID, new MemoryPersistence());

The BROKER_URL should contain protocol, url and port, which the client will connect to, like this: ssl://<cumulocity url>:8883. The CLIENT_ID value must match the value of the common name of the device certificate that will be used. The “d:” prefix is used in Cumulocity IoT for device connections and it should not be removed or changed. Now the only thing that needs to be configured to establish the SSL connection is to fill paths in the code fragment:

sslProperties.put(SSLSocketFactoryFactory.KEYSTORE, getClass().getClassLoader().getResource(KEYSTORE_NAME).getPath());
sslProperties.put(SSLSocketFactoryFactory.KEYSTOREPWD, KEYSTORE_PASSWORD);
sslProperties.put(SSLSocketFactoryFactory.KEYSTORETYPE, KEYSTORE_FORMAT);
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTORE, getClass().getClassLoader().getResource(TRUSTSTORE_NAME).getPath());
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTOREPWD, TRUSTSTORE_PASSWORD);
sslProperties.put(SSLSocketFactoryFactory.TRUSTSTORETYPE, TRUSTSTORE_FORMAT);

After filling in this data, the example client will use the provided data to connect to the specified platform using certificates. The example also shows how to create the callback for the connection. First thing is to create the class which implements the interface MqttCallbackExtended. Then such a class can be created and an instance of it can be provided to the MQTT client: mqttClient.setCallback(this);.

In general, the MQTT Eclipse Paho Client uses the Java Secure Socket Extension, which is part of the Java Development Kit, to provide secure connections via SSL. JSSE provides the Java implementation of the SSL and TLS protocol, which can be configured by developers using its classes. The documentation of the Java Secure Socket Extension shows how the SSL connection is established and provides some examples of customizing the implementation. The full document is available on the official Oracle website.

Device integration

Overview

The basic life cycle for integrating devices into Cumulocity IoT is discussed in Interfacing devices.

In this section, we will show how this life cycle can be managed using the MQTT implementation.

The life cycle consists of two phases, a startup phase and a cycle phase.

The startup phase can be as short as just checking the credentials:

The cycle phase consists of two kinds of actions:

MQTT phases

Startup phase

Step 0: Request device credentials

In Cumulocity IoT, every MQTT connection needs to be authenticated. You can use the device credentials topics in the MQTT implementation to generate new credentials for a device.

Once the device retrieved the credentials, it needs to store them locally for further connections.

To establish a connection you must configure the following parameters:

For more information, refer to the Hello MQTT section.

The process works as follows:

After receiving the credentials, the device can close the MQTT connection and create a new one with the received credentials.

Step 1: Verify device

As MQTT supports an automatic device creation if the client sends data and there is no device present, this step is only required if you want to create the device manually.

The device creation can be achieved by employing the static template 100. This template can be blindly used on every boot of the device as it will only create the device if it is not already present.

The device will be linked automatically to the ID the client uses with its MQTT ClientId.

100,Device Name,Device Type
Info
The topic used for Cumulocity IoT’s pre-provided static templates is s/us.

Step 2: Verify children

Like the root device, also its children are covered by the automatic device creation.

To handle this step manually you can send the static template 101 for creating a child device. The template will only create the child if it does not already exist.

101,Unique Child ID,Child Name,Child Type

Step 3: Subscribe topics

If the device supports operations, it should subscribe to all required topics (static templates and SmartREST 2.0).

Cycle phase

Step A: Send CSV data

While the device holds an active MQTT connection, it can publish either on the topics for static templates or on the topics for a SmartREST template to send data to the server.

Based on the MQTT ClientId, the physical device is directly connected to the device object in Cumulocity IoT. Therefore, the data you send is automatically connected to the device.

To send data to a child device, publish the data to the topics described in Device hierarchies.

Step B: Receive CSV operations

By subscribing to a topic the device automatically tells Cumulocity IoT that it wants to receive operations. Any operation created will be automatically parsed using either the static templates or the templates the device defines.