Introduction

About this guide

The CEL analytics guide consists of the following sections:

• Introduction
• Data model
• Functions
• Advanced uses cases
• Best practices and troubleshooting
• Examples
• Study: Circular geofence alarms

Overview

Using the Cumulocity real-time event processing, you can add your own logic to your IoT solution. This includes data analytics logic but it is not limited to it. To define new analytics, you can use the Cumulocity Event Language. The language allows analyzing incoming data. It is using a powerful pattern and time window based query language. You can create, update and delete your data in real-time.

Typical real-time analytics use cases include:

• Remote control: Turn a device off if it’s temperature rises over 40 degrees.
• Validation: Discard negative meter readings or meter readings that are lower than the previous.
• Derived data: Calculate the volume of sales transactions per vending machine per day.
• Aggregation: Sum up the sales of vending machines for a customer per day.
• Notifications: Send me an email if there’s a power outage in one of my machines.
• Compression: Store location updates of all cars only once every five minutes (but still send real-time data for the car that I am looking at to the user interface).

In the following sections, we describe the basics for understanding how the Cumulocity Event Language works and how you can create your own analytics or other server-side business logic and automation.

CEP application variants

In Cumulocity, there are two deployment scenarios for using CEP rules:

• MULTI_TENANT: This scope provides access to a shared instance of CEP container. All subscribed tenants share the resources of the same CEP instance. It is available if you are subscribed to the “Cep” application, a built-in application which comes with Cumulocity.

• PER_TENANT: Each subscribed tenant has at least one own instance of CEP container. The container is isolated from other tenants, hence high CPU load or memory issues on other containers do not have any impact on the own one. This feature is available with the application “Cep-small” which is an optional service. Also, you need to be subscribed to the application" Feature-cep-custom-rules" to be able to upload your own CEP rules.

For details on application subscription refer to Administration > Managing tenants > Subscribing to applications in the User guide.

Using Cumulocity Event Language (CEL)

Cumulocity Event Language has a syntax similar to SQL language. In SQL a statement is run against a logically fixed database, produces a result and completes the task. In Cumulocity, a statement is continuously running against a stream of input data (input events) and is continuously calculating its output (output events).

As an example, the following statement continuously retrieves new temperature sensor readings ranging above a particular temperature:

select *
from MeasurementCreated e
where getNumber(e, "c8y_TemperatureMeasurement.T.value") > 100

Here, MeasurementCreated is a stream containing an event for each measurement that is created in the system. Selecting a subset of these events is done using where, similar to SQL. getNumber() is a function to read out a numeric value from an event. In this example, “e” is the “MeasurementCreated” event and the property is “c8y_TemperatureMeasurement”. “T.value”, is a value in degrees Celsius of a temperature sensor (see the sensor library).

How can I create derived data from CEL?

There are special streams provided by the system to perform predefined operations (such as data storage or sending data by email). One stream is CreateAlarm, which can be used to store an alarm in Cumulocity. Assume that an alarm should be generated immediately if the temperature of a sensor exceeds a defined value. This is done with the following statement:

insert into CreateAlarm
select
 e.measurement.time as time,
 e.measurement.source.value as source,
 "c8y_TemperatureAlert" as type,
 "Temperature too high" as text,
 "ACTIVE" as status,
 "CRITICAL" as severity
from MeasurementCreated e
where getNumber(e, "c8y_TemperatureMeasurement.T.value") > 100

Technically, this statement produces a new “AlarmCreated” event each time a temperature sensor reads more than 100 degrees Celsius and puts it into the “CreateAlarm” output stream. The property names in the selected clause have to match the properties of “AlarmCreated” (see the Cumulocity Event Language reference).

How can I control devices from CEL?

Remote control in CEL is just another type of derived data. Remote operations are targeted to a specific device. The following example illustrates switching a relay based on temperature readings:

insert into CreateOperation
select
"PENDING" as status,
<<heating ID>> as deviceId,
{
"c8y_Relay.relayState", "CLOSED"
} as fragments
from MeasurementCreated e
where getNumber(e, "c8y_TemperatureMeasurement.T.value") > 100

• heating ID is a placeholder for the ID of the heating that should be triggered.
• fragments defines the nested content of the operation a “c8y\Relay” that is “CLOSED”.

The syntax of the fragments part is a list of pairs of property names and values surrounded by curly braces: {?key1?, ?value1?, ?key2?, ?value2?, …}.

How can I query data from CEL?

It may be required to query information from the Cumulocity database as part of the ongoing event processing. This is supported by a set of querying methods. Here is an example that shows how to summarize total sales for vending machines every hour. The sales report data created after a purchase is retrieved from the Cumulocity database.

create window SalesReport.win:time_batch(1 hour)  
(
    event com.cumulocity.model.event.Event,
    customer com.cumulocity.model.ManagedObject
)

insert into SalesReport
select
    e.event as event,
    findOneManagedObjectParent(e.event.source.value) as customer
from EventCreated as e

insert into CreateMeasurement
select
    "total_cust_trx",
    "customer_trx_counter",
    {
        "total", count(*),
        "customer_id", sales_report.customer.id.value
    }
from SalesReport as sales_report
group by sales_report.customer.id.value

Above we create a batch window first, which keeps data for one hour in order to calculate a total in this time frame. We store the prepared data into this window: Incoming events along with the parent managed object of the event source. This corresponds to the data model of our vending application: Sales reports are represented as events in Cumulocity with a vending machine as source. Customers are represented as parent managed objects of vending machines.

The collection of sales reports is calculated through “insert into CreateMeasurement…” using a SQL-like syntax and is stored as a measurement. The difference to SQL is: In SQL, you calculate a result over a fixed, current content of a database. In Cumulocity Event Language, statements run endlessly and the process time has to be limited by the time window.

Event streams

In the Cumulocity Event Language data flows in streams. You can create events in streams and listen to events created in streams.

Predefined streams

There are some predefined streams to interact with several Cumulocity APIs. For each input stream, Cumulocity will automatically create a new event when the respective API call was made. If a measurement was created via REST API there will be a new event in the MeasurementCreated stream. For interacting with the Cumulocity backend you can create an event on the respective output stream and Cumulocity will automatically execute either the database query or create the API calls necessary for sending mails, sms, or similar. To create a new alarm in the database you can create a new event in the CreateAlarm stream.

Look at the data model to see how the events for each stream are structured.

Creating events in a stream

Creating an event is done by the keywords insert into and select. First, you need to specify the “insert into” followed by the stream name for which stream you want to create an event. After that you can use the “select” clause to specify the parameters of the event. A parameter gets specified by the following syntax: value as parameter. You can specify multiple parameters by separating them by commas. The order of the parameters does not matter. Please notice that streams can have mandatory parameters you need to specify in the “select” clause.

Listening to events in a stream

The most common way to trigger the creation of an event in a stream is when something happens on another stream. Therefore you can listen to events from other streams. This is done by the keyword from followed by the name of the stream and (optional) followed by a variable name to reference the event in your statement at a later point.

Conditions

Adding conditions can be done with the keyword where to not trigger your event creation for every incoming event but only if these conditions are met. The where keyword is followed by an expression that results either in true or false. You can also have multiple expressions connected with and or or.

Example

As an example, we create a statement. It should listen to a stream and create a new event in another stream whenever the specified condition applies. As example we want to create an alarm for each temperature measurement that is created.

1. To create an alarm we need to insert into the stream CreateAlarm.
2. We need to specify all parameters for the event in the select clause.
3. We want the alarm to be created when an event from the stream MeasurementCreated is received.
4. We want the alarm only be created under certain conditions of the event from the MeasurementCreated stream which we specify in the where clause.

The resulting statement can look like this:

insert into CreateAlarm
select
  measurementEvent.measurement.time as time,
  measurementEvent.measurement.source.value as source,
  "c8y_TemperatureAlarm" as type,
  "Temperature measurement was created" as text,
  "ACTIVE" as status,
  "CRITICAL" as severity
from MeasurementCreated measurementEvent
where measurementEvent.measurement.type = "c8y_TemperatureMeasurement";

Troubleshooting

Error message

Real-time event processing is currently overloaded and may stop processing your events. Please contact [product support](/about-doc/contacting-support).

Description

The CEP queue for the respective tenant is full. This might for example happen when more events are created than currently can be handled.

In this case, an alarm will be raised. To avoid losing incoming new events, the oldest events will be deleted, i.e. an incoming new event triggers the deletion of the queue head event.