Overview

Streaming analytics with Apama EPL

Using Apama streaming analytics, you can add your own logic to your IoT solution for immediate processing of incoming data from devices or other data sources. These user-defined operations can, for example, alert applications of new incoming data, create new operations based on the received data (such as sending an alarm when a threshold for a sensor is exceeded), or trigger operations on devices.

The operation logic is based on Apama’s Event Processing Language (Apama EPL).

Important: Support for streaming analytics using CEL (Esper) is deprecated. All new Cumulocity IoT subscriptions use the Apama CEP engine. While using the Esper CEP engine is still supported for older installations, this will no longer be provided for new subscriptions and will not be invested into in the future.

For documentation on using the deprecated CEL functionality based on Esper, refer to the CEL analytics guide.

For details on migration, see Migrating from CEL (Esper) to Apama.

Typical real-time analytics use cases include:

Remote control: Turn a device off if its temperature rises over 40 degrees.
Validation: Discard negative meter readings or meter readings that are lower than the previous.
Derived data: Calculate the volume of sales transactions per vending machine per day.
Aggregation: Sum up the sales of vending machines for a customer per day.
Notifications: Send me an email if there is a power outage in one of my machines.
Compression: Store location updates of all cars only once every five minutes (but still send real-time data for the car that I am looking at to the user interface).

The following sections describe the basics for understanding how Apama EPL works and how you can create your own analytics or other server-side business logic and automation.

Info: This documentation assumes basic familiarity with Apama application development. Refer to the Apama documentation for further details.

Apama EPL Apps

Apama EPL Apps is a web application which is available from the application switcher. It allows you to develop Apama applications directly within Cumulocity IoT, written in Apama EPL. You can also use it to import existing Apama applications (*.mon files) into Cumulocity IoT. When you then activate an Apama application, you deploy it to Cumulocity IoT.

See Basic functionality for further information.

Apama Analytics Builder

Apama Analytics Builder is a web application which is available from the application switcher. It allows you to build analytic models that transform or analyze streaming data in order to generate new data or output events. The models are capable of processing data in real time.

You build the models in a graphical environment by combining pre-built blocks into models. The blocks in a model package up small bits of logic, and have a number of inputs, outputs and parameters. Each block implements a specific piece of functionality, such as receiving data from a sensor, performing a calculation, detecting a condition, or generating an output signal. You define the configuration of the blocks and connect the blocks using wires. You can edit the models, simulate deployment with historic data, or run them against live systems.

See the documentation for Apama Analytics Builder for Cumulocity IoT for detailed information.

Microservice runtime and applications

Custom Apama applications (individual *.mon files), Apama Analytics Builder models and Smart Rules are executed in an Apama-ctrl microservice. This has a per-tenant isolation scope, that is, each subscribed tenant has its own instance of an Apama container with dedicated resources (that is, memory and CPU usage). The container is isolated from other tenants, hence high CPU load or memory issues on other containers are tracked and resourced independently.

You can use predefined rules (see Smart Rules), define your own custom rules with Apama EPL Apps, or use Apama Analytics Builder to build analytic models. This requires the following microservices and/or applications:

To do this	you need the following
Use predefined rules	Apama-ctrl microservice and Smartrule microservice (included in Cumulocity IoT’s Standard Tenant).
Define custom rules	Apama-ctrl microservice and Apama EPL Apps web application.
Use Apama Analytics Builder	Apama-ctrl microservice and Apama Analytics Builder web application.

See also the tables listing the available applications under Managing applications in the User guide.

If your tenant is subscribed to the Apama Starter microservice (instead of other Apama-ctrl microservices), then the following applies for Apama:

Unlimited number of Smart Rules.
Apama Analytics Builder is limited to at most 3 active models.
Apama EPL Apps is not available in the application switcher.

Contact Software AG Support to discuss adding more capabilities.

Migrating from CEL (Esper) to Apama

Migrating from CEL when only using Smart Rules

If a tenant is only using Smart Rules and not any custom rules, you can simply unsubscribe the tenant from CEL and subscribe to Apama instead (or ask your operations team to do this for your tenant).

Any previously configured Smart Rules will be restarted. For Smart Rules which are stateful, this will, as with any restart of the microservice hosting the Smart Rules, lose state within the Smart Rule. In this case, the input (measurements, alarms, etc.) for the devices in question will need to be sent again before the Smart Rule will be functioning as before.

Smart Rules will only work correctly if moving from CEL to Apama, not in the opposite direction.

The Smart Rules listed in the following table are stateful:

For this Smart Rule	Check the following
On geofence create alarm	Which devices are currently in or out of the geofence?
On geofence send email	Which devices are currently in or out of the geofence?
On alarm escalate it	Which alarms have been created?
On alarm duration increase severity	Which alarms have been created?
Calculate energy consumption	What are the last meter readings?
On missing measurements create alarm	What is the previous time of measurement and which devices have sent the measurement before?

The following Smart Rules are stateless:

On alarm send email
On alarm send SMS
On alarm execute operation
On measurement threshold create alarm
On measurement explicit threshold create alarm

Migrating from CEL when also using custom rules

Migrating from custom rules written in CEL to Apama EPL requires rewriting and retesting the custom rules. As with any scripting or programming, you should thoroughly test significant changes before deploying into a production environment. Thus, the recommended approach is to create a separate tenant for hosting Apama rules as they are developed, and replicate any input data required in that tenant. To do this, follow these steps:

Lock down the CEP custom rules on the existing tenant to prevent change.
Make available a new tenant on which Apama has been enabled.
Manually convert all CEP custom rules from the existing tenant into equivalent Apama EPL applications on the new tenant.
Manually recreate all Smart Rules from the existing tenant on the new tenant.
Manually recreate any scheduled exports from the existing tenant on the new tenant.

Info: The “scheduled exports” feature separate from CEP (Esper) requires Cumulocity IoT 10.6 or later.
Test the behavior of the new custom rules, checking for memory leaks and performance as well as correctness.
Remove the existing tenant after all CEP custom rules, Smart Rules and scheduled exports have been moved to or recreated on the new tenant.

You can also choose to work with Software AG Professional Services to help ensure the migration is as smooth as possible. Software AG Professional Services can help migrate CEL code into Apama EPL code and they can also provide training on using Apama in Cumulocity.