Streaming Analytics

December 2023

Change Apama correlator version

This version of Cumulocity IoT Streaming Analytics includes the Apama version 10.15.4 correlator. EPL apps developers should also refer to What’s New In Apama 10.15.4 in the Apama documentation.

Change Location of diagnostics links

Previously, two links for downloading diagnostics information were available at the bottom of the Streaming Analytics application. These links have been moved. They are now available when you click the User button in the Streaming Analytics application to display the right drawer. The right drawer now displays these links in a new Diagnostics section. See also Downloading diagnostics and logs in the user documentation.

In addition, the right drawer now also provides a Documentation section with links to the Streaming Analytics documentation.

Change Basic diagnostics information

The EPL memory profiler snapshots, which were previously only included in the enhanced diagnostics information, are now also included in the basic diagnostics information. This is helpful in case a high memory usage alarm is raised when the Apama-ctrl microservice consumes 90% of the maximum memory permitted for the microservice container and you only have basic diagnostics information available. See also Downloading diagnostics and logs in the user documentation.

Change Renamed commands for downloading analytic models and EPL apps

The commands for downloading analytic models and EPL apps have been renamed from Export to Download. See also Downloading a model and Developing apps with the Streaming Analytics application.

Feature New EPL sample

A new EPL sample named “Receive update notifications” can now be accessed from the EPL editor of the Streaming Analytics application. It shows how to write an EPL app that can distinguish between notifications for creating and updating managed objects, measurements, alarms, events, and operations. See also Developing apps with the Streaming Analytics application in the user documentation.

For more detailed information, see Receiving update notifications in the Apama documentation for the Cumulocity IoT transport connectivity plug-in.

Feature Receive input from all input sources by default

The Analytics Builder input blocks can now be configured to receive inputs from all input sources. You can simplify global tasks by creating a single Analytics Builder model that works with inputs from all input sources, for example, send an email for every critical alarm of type C8Y_TemperatureAlarm that is generated, regardless of the device that generated the alarm. A new All Inputs option is available for this purpose. When you add a new block to your model or when you edit the parameters of a new template instance, the All Inputs option is now set by default. See also Editing the parameters of a block in the user documentation.

You can also use the new All Inputs option in the replace dialog box when replacing input sources. Keep in mind that the replace dialog box is used for both input blocks and output blocks. So when you replace a device with the new All Inputs option, all matching output devices are automatically changed to trigger devices. See also Replacing sources or destinations in the user documentation.

A new Analytics Builder sample named “Aggregate measurements per input source” is now available, which creates new measurements that average the measurement values for each input source that has a specified fragment and series. This is a simple sample that creates a model without template parameters, so you can activate the model directly in the model manager. See also The Samples tab in the user documentation.

Feature Additional value types for the Constant Value block

The Constant Value block in Analytics Builder now supports float and boolean value types and can produce output of these types. This enables the block’s output to be consumed by other blocks that take input of type float or boolean like the blocks in the Logic and Aggregate categories. The Type parameter is also now optional. If a type is not selected, the type of the output value is inferred from the Value parameter.

Change Removing template parameters

In Analytics Builder, an icon is now provided for removing a template parameter from the Template Parameter dialog box. The actions menu (the three vertical dots at the end of a row) has therefore been removed. See also Managing template parameters in the user documentation.

Feature Selection lists for template parameters

You can now create a selection list for an Analytics Builder template parameter. This allows the model author to provide a predefined list of values for the user to choose from, ensuring that the user only enters the values you allow. You can define selection lists for types such as string, float, source or destination, or geofence and you can also select a specific value to be the default value. The values that you define for a selection list are then available for selection when you create instances of the model. See also Adding a selection list for a template parameter in the user documentation.

Change Missing subassets in a device group or asset hierarchy

Previously, when reactivating an Analytics Builder model, an error was thrown if a subasset of a device group or asset hierarchy from which the model receives events no longer existed in the inventory, for example, because a device was deleted. As of this version, missing subassets in a device group or asset hierarchy are ignored and an error is no longer thrown. However, if the deletion of a subasset results in an empty device group or asset hierarchy, an error is still thrown.

Feature Cumulocity IoT transport in Apama 10.15.4

The Alarm and Operation events have new constants which define the valid values for their respective status and severity members. This allows more robust coding and eliminates runtime errors caused by typographical errors with literal strings.

Feature EPL enhancements in Apama 10.15.4

String concatenation operator + supports non-string operands

If only one operand expression is of type string, then string conversion is now performed on the other operand to produce a string at runtime. For more details, see Additive operators in the Apama documentation.

New built-in convenience methods on string, sequence and dictionary

The EPL language has been enhanced with some convenience methods for common operations such as checking if a string or sequence contains a specified substring or item, and determining if a string, sequence or dictionary is empty.

There is a method to get a sequence value with a fallback default if there is no value at the specified index:

mySequence.getOr(100, "default value");

There is also a powerful regular expression group search for strings that finds the first match and returns the text captured by each “(…)” group of the regular expression:

// Prints ["Bob", "Eve"]
 print "Today Bob met Eve".groupSearch("([a-zA-Z]) met ([a-zA-Z])").toString();

The following table lists all new methods:

For more information, see the API Reference for EPL (ApamaDoc).

Support for negative indexes for sequence[…] access

Instead of throwing an out of bounds exception, it is now possible to refer to items near the end of a sequence by specifying a negative index. For example, seq[-1] gives the last item in the sequence and seq[-2] gives the item before that. Negative sequence indexes can also be used in the sequence.remove() and sequence.insert() methods.

Note that negative values cannot be specified when accessing a sequence item using the any.getEntry() method.

Mixed-type sequence and dictionary literals are now treated as by default

It is no longer necessary to add an <any> cast around the first item in a sequence or dictionary literal that contains a mixture of different types. For example, the following sequence literal which produces a sequence<any>:

[<any> 12345, "a string"]

can now be written more simply as

[12345, "a string"]

This simplifies the functional example given in EPL enhancements in 10.15.3 in the Apama documentation: the <any> is no longer needed in the setFields call.

Discarding of unused return values

In EPL, it is now possible to discard the return value of a function or expression and not assign it to a variable. This is useful if you only want the side effects of the function and do not need the return value. Be careful not to discard a return value that indicates important conditions that your program may need to use.

Previously, you had to write:

any _ := Functional(newAlarms).map(Fn.getEntry("alarm")).map(allAlarms.append);

But now you can write:

Functional(newAlarms).map(Fn.getEntry("alarm")).map(allAlarms.append);

See also Using an expression as a statement in the Apama documentation.

Functional operators

The com.apama.functional.Fn and com.apama.functional.Functional events now have a new method mapKeys which can return a dictionary container with modified keys. For more details, see Functional operators in the Apama documentation and the API Reference for EPL (ApamaDoc).

Functional listeners

The functional onTimeout action now returns the wait listener it creates so that it can be quit if needed. For more details, see Functional listeners in the Apama documentation.

Handling uncaught exceptions

EPL monitors are no longer terminated by default when an uncaught exception is thrown in some cases.

Programming errors and unexpected data in incoming events can cause an uncaught exception in EPL. This causes the monitor instance to be terminated, rendering the application unusable. To provide a better experience, exceptions thrown from listeners and stream listeners which are not caught no longer terminate the monitor instance. Instead, they just stop the handling of the current event.

Developers are still encouraged to correctly catch and handle the exceptions in their EPL. If an exception is thrown and not caught, it is still possible for the events to be lost and not correctly handled.

This is a change in behavior. Some users may be relying on this previous behavior, in combination with an ondie() action. EPL with an ondie() action retains the previous behavior of always terminating the monitor and calling ondie(). If you need to retain the previous behavior, you can add an empty ondie() action to your monitor.

Updated events in com.apama.cumulocity

To avoid confusion as to whether events were being sent towards Cumulocity IoT or being received back as updates from Cumulocity IoT, the Cumulocity event definitions API was changed in a previous release (10.5.2.0) so that the existing CHANNEL constant was deprecated and replaced by either SEND_CHANNEL or SUBSCRIBE_CHANNEL constants as appropriate. Some events were omitted from these changes and this has now been rectified. Therefore, the CHANNEL constant is deprecated on the following events:

• SubscribeMeasurements
• UnsubscribeMeasurements
• FindManagedObjectResponseAck
• FindMeasurementResponseAck
• GenericResponseComplete
• Subscribe
• Unsubscribe

In addition, a new SUBSCRIBE_CHANNEL or SEND_CHANNEL has been added to the following events instead:

• SubscribeMeasurements
• UnsubscribeMeasurements
• Subscribe
• Unsubscribe

The constants on the following events are instead replaced with SUBSCRIBE_CHANNEL constants on their corresponding Response types:

• FindManagedObjectResponseAck
• FindMeasurementResponseAck
• GenericResponseComplete

The API Reference for EPL (ApamaDoc) has been amended to make it clear that acknowledgement events are always received on the same channel as the corresponding response events, and to avoid confusion the CHANNEL constant has been deprecated on those acknowledgement events where it had been defined.