Change Logs

These change logs document all relevant changes for the Apama product.

The following types of change are included:

  • Feature - New features which are generally available.
  • Preview - Features which are in Public Preview and not yet generally available.
  • Improvement - Small improvements.
  • Announcement - Deprecations, removals or important UI changes.
  • API change - Breaking changes in the APIs.

This section covers changes from 10.15 onwards. For information about earlier releases please see Pre-10.15 Release Notes.

Displaying all changes since the previous release


Apama Ant macros are now deprecated

The Apama macros for Ant (apama-macros.xml) are deprecated and will be removed in a future release.

Modern deployments of Apama can usually be performed by simply running the correlator with an appropriate YAML configuration. For more advanced cases, we recommend writing a Python script (or a shell script).

Apama images for Docker moving to Amazon ECR

The Apama images for Docker have been moved to Amazon ECR Public Gallery. They are now available at public.ecr.aws/apama and can be viewed at https://gallery.ecr.aws/apama/. Previously, they were available at softwareag/ and could be viewed at https://hub.docker.com/u/softwareag. Within any Dockerfiles that use the Apama images, you must therefore change FROM softwareag/<IMAGE>:<TAG> to FROM public.ecr.aws/apama/<IMAGE>:<TAG>.

In addition, Apama is now installed in a different directory. The location of Apama within the Docker image has been moved from /opt/softwareag to /opt/cumulocity. Existing images remain unchanged.

See also Deploying Apama applications with Docker.

JMS, Kafka and Distributed MemoryStore are now deprecated

The following connectivity features are now deprecated and will be removed in a future release:

  • Correlator-integrated JMS, including the --jmsConfig correlator command line option. If you need to connect to JMS after this is removed, write a connectivity plug-in.
  • The connectivity plug-in for Kafka. If you need to connect to Kafka, write a connectivity plug-in.
  • Distributed MemoryStore, including the Terracotta and BigMemory drivers, the com.apama.correlator.memstore Java API and the --distMemStoreConfig correlator command line option. The in-memory and file-based MemoryStore EPL APIs are unaffected.

New Download Site for Apama

The location for downloading Apama has changed to an Apama area in the Cumulocity Download Centre, available at this link: https://download.cumulocity.com/Apama/.

The download centre hosts the Full Edition and the Community Core Edition of Apama. It replaces both the Empower site previously used by commercial customers, and for community users of Apama the https://downloads.apamacommunity.com/ site. The directory layout within the site is a little different, so any scripts referring to the https://downloads.apamacommunity.com/ will need to be updated to point to the new package locations.

If you use the Apama Debian Package Repository, you need to repeat the setup steps at https://download.cumulocity.com/Apama/Debian/ to add the new repository (which will overwrite or supercede the previous one). See also Installing Apama using Debian apt.

New EPL event for caching managed objects

The new com.apama.cumulocity.ManagedObjectCache event covers the common use case of fetching a ManagedObject with a known id. It eliminates the need to implement the FindManageObject code for this scenario and reduces traffic across the REST API.

To allow more than one EPL app to share the same cached managed objects, a SharedManagedObjectCache is now also provided, which is a wrapper monitor around the ManagedObjectCache.

For detailed information, see Caching managed objects.

New location for APAMA_WORK directory and license file

The location of the APAMA_WORK directory has changed from \SoftwareAG\ApamaWork_*n.n*\ to \Apama\ApamaWork_*n.n*\. Thus, Apama now expects the license file under \Apama\ApamaWork_*n.n*\license\ (see also About Apama license files).

Various Java and C++ APIs are now deprecated

The following features are now deprecated and will be removed in a future release:

  • The Java APIs for Engine Client, Engine Management, Event Service, Scenario Service, Generic Management and com.apama.util.Logger. This includes the com.apama.engine, com.apama.event, com.apama.net, com.apama.util and com.apama.services packages. The Java APIs for connectivity plug-ins and EPL plug-ins remain unaffected. Instead of the com.apama.util.Logger class, use SLF4J for logging from within the correlator.
  • The C++ Engine Client API. This includes the com::apama::event and com::apama::engine namespaces, and engine_client_cpp.hpp. The C++ APIs for connectivity plug-ins and EPL plug-ins remain unaffected.

Windows and Eclipse are now deprecated

As part of the ongoing modernization of our developer experience the following are deprecated and likely to be removed in a future release:

  • Support for running Apama directly on Windows. Alternative workflows for Windows-based developers may be provided in future.
  • The Eclipse development environment. An alternative development environment may be provided in future.
10.15.5.3

Flushing a full HTTP client queue when using a large number of clients could potentially deadlock.

A potential deadlock has been fixed when using a large number of clients and flushing a full HTTP client queue.

10.15.5.2

Upgrade Python to 3.9.20 and setuptools module to 70.3.

Python has been upgraded to version 3.9.20 and the setuptools module has been upgraded to version 70.3 in this fix release.

10.15.5.1

ap-correlator-jms.jar was missing from the "apama-dev" package.

In 10.15.5.0, JMS support was only provided through the apama-dbp-dev package. JMS support is now also available from the apama-dev package.

apama_env.bat did not set "APAMA_JRE" in "apama_core".

In an apama_core installation of 10.15.5.0, the apama_env.bat did not correctly set the APAMA_JRE environment variable to the JAVA_HOME environment variable. This has been fixed.

SL dashboards command line tools and apama-macros.xml did not work after installation changes.

When using SL dashboard executables from Ant macros, set the APAMA_DASHBOARD_BIN environment variable to the installation path of apama-sl-dashboard/Apama/bin to ensure that the macros can find the corresponding executables. The environment variable APAMA_DASHBOARD_HOME is no longer used by Apama and is not set by apama_env[.bat].

10.15.5

Additional metrics added to the Cumulocity transport

The following additional metrics have been added to the Cumulocity transport to help provide more information when monitoring the transport:

  • received*event*EventsParsedCount
  • sent*event*EventsParsedCount
  • received*event*EventsParsedSizeEWMABytes
  • sent*event*EventsParsedSizeEWMABytes

where *event* stands for one of Alarm, Measurement, Event, Operation or ManagedObject. For more information, see Monitoring status for Cumulocity.

Apama is no longer available from the Software AG Installer

As of June 2024, Apama is no longer available from within the Software AG Installer. This option has been replaced with .zip (Windows) and .tar.gz (Linux) files that are extracted to the desired installation location.

Migration from the Software AG Installer

The installed structure of the extracted contents of the installation packages matches that created by the Software AG Installer. Therefore, the installations are equivalent. The recommended method for upgrading an installation package for a release, service pack or fix is to remove the existing installation and extract the upgraded package in its place at the same location on the disk. The same procedure can be used with an installation originally created by the Software AG Installer.

Differences between the installations

While the structure of the installations on the disk is the same, there are some other differences.

  • License file

    The Software AG Installer provides the ability to select an arbitrary license file and put in place during the installation. For a new installation, the license file must be manually copied into place, typically into the APAMA_WORK.license directory (see About Apama license files). If the installation is upgraded, this should not be necessary, as the license file is already in place and the APAMA_WORK directory is not touched, assuming it was placed outside of the installation directory.

  • Windows Start menu

    Installation by extraction does not automatically create any entries in the Windows Start menu. This can be done manually if required, and does not need to be changed if the installation is updated.

  • Apama Plugin for Eclipse

    Apama Plugin for Eclipse, which is an OEM version of Eclipse, has been replaced with a standard copy of Eclipse using the Apama plug-in. The packages that contain the Apama plug-in include a full copy of Eclipse. There is no need to install Eclipse separately or add the plug-in manually.

    The Apama demos (BackTesting, StatisticalArbitrage and Weather) have been removed.

Choosing which package to use

Several installation packages are available from the Apama downloads area, as shown in the table below. Used without a license, these packages provide the freemium Apama Community Edition, with the limitations that come with it (see Running Apama without a license file).

Feature apama_core apama-c8y-dev apama-dev apama-dbp-dev apama_core_{VERSION}_cumulocity apama_core_{VERSION}_samples apama_core_{VERSION}_doc apama-sl-dashboard
Use Case Runtime deployment Cumulocity development Standalone customers without IBM products (that is, not needing client libraries to connect to Universal Messaging and/or Terracotta) Standalone customers with IBM products (that is, needing client libraries to connect to Universal Messaging and/or Terracotta) Retained for compatibility with existing zip files Samples for EPL et al. Apama documentation Dashboard development or deployment
Notes Current apama_core.     As apama-dev with IBM components. Cumulocity components only. Overlay on apama_core or apama-dev. Current samples zip. Can be extracted separately. Current documentation zip. Can be extracted separately. Only SL components. Installs its own JVM into a separate directory.
Core        
Cumulocity            
Eclipse with Apama plug-in 1          
IAF2/ADBC3            
IBM webMethods              
Samples        
Documentation        
SL Dashboard        

For more information, see Installing Apama.

Cannot compare optional types containing any type within it.

It was not possible to compare optional types containing the any type. For example, strings such as the following could not be compared: optional<sequence<any>>, optional<dictionary<string,any>>. This issue has now been fixed.

Command Central integration has been removed

Command Central integration is no longer supported and has been removed. For installation or deployment of Apama, it is recommended to instead either install the Apama packages from the download centre, or to use modern containerization methods at runtime via Docker or Kubernetes with the Apama images. Instead of monitoring Apama from Command Central, it is recommended that you now use Prometheus. See Installing Apama, Deploying Apama applications with Docker, Deploying Apama applications with Kubernetes, and Monitoring with Prometheus. See also the announcement in Removed and deprecated features in 10.15.0.

Cumulocity Client connectivity bundle is now deprecated

The Cumulocity Client connectivity bundle is now deprecated and will be removed in a future release. It is recommended that you use the new Cumulocity Notifications 2.0 connectivity bundle instead.

Migrating from Cumulocity Client to Cumulocity Notifications 2.0

Differences between Notifications 2.0 and Cumulocity Client

The Cumulocity Notifications 2.0 bundle receives messages sent in CEP and QUIESCENT processing modes, in addition to the TRANSIENT and PERSISTENT modes received by the long-polling-based Cumulocity Client bundle. There are also some extra fields that appear within the params dictionary in the messages. These changes serve to bring the notifications received by custom Apama applications in line with those received by applications running inside the Cumulocity IoT platform using EPL apps, analytic models, and smart rules.

Converting a project from using the Cumulocity Client bundle to the Cumulocity Notifications 2.0 bundle

If you have an existing project that uses the Cumulocity Client bundle, then to start using the Cumulocity Notifications 2.0 bundle instead, you must first make a copy of the CumulocityIoT.properties file in the bundle instance in your project. Then remove the Cumulocity Client bundle and add the Cumulocity Notifications 2.0 bundle. This creates two files: CumulocityIoTREST.properties and CumulocityNotifications2.properties. Copy all relevant values from the old CumulocityIoT.properties file into the CumulocityIoTREST.properties file, and then fill out the required properties in the CumulocityNotifications2.properties file.

Removing the Cumulocity Client bundle without adding the Cumulocity Notifications 2.0 bundle

If you have an existing project which does not use notifications at all, you can add a bundle that only supports querying the REST API and does not receive any notifications. Save the CumulocityIoT.properties file, remove the Cumulocity Client bundle as described above, and then add the Cumulocity REST Support bundle instead. This bundle has a single properties file: CumulocityIoTREST.properties. Copy all relevant values from the old CumulocityIoT.properties file into the CumulocityIoTREST.properties file.

Cumulocity events now no longer throw exceptions from the isCreate() and isUpdate() actions

The Alarm, Event, ManagedObject and Operation events now no longer throw exceptions from the isCreate() and isUpdate() actions. If the event being checked was generated in EPL code, these two actions now simply return false. try... catch blocks around these calls are no longer necessary if the event being checked was generated in EPL code. In the unlikely case that the try... catch block is used to intentionally check for internally generated events, you need to change the check to the following:

if not (evt.isCreate() or evt.isUpdate()) {... }

Cumulocity IoT transport claims an ability to connect to a server with self-signed certificates that does not work.

When connecting to a Cumulocity IoT platform whose certificate is not signed by a trusted certificate authority (CA), the CUMULOCITY_AUTHORITY_FILE property in the CumulocityIoTREST.properties configuration file, which provides the TLS certificate authority file for the Cumulocity IoT connectivity bundles, was not used. This has now been fixed.

Cumulocity IoT transport suppresses all HTTP error logging in a multi-tenant environment.

HTTP error responses are now logged when Apama communicates with multiple tenants. In an Apama-ctrl microservice subscribed to multiple tenants, the Cumulocity IoT transport ignored HTTP errors rather than recording them in the correlator log. This issue has now been fixed. When an HTTP error response is received, it is now logged as an error.

Cumulocity transport goes to an infinite loop when pageSize is set to 1.

The Cumulocity IoT transport had a regression (introduced in Apama 10.15.2) that could cause REST queries to infinitely loop if pageSize was set to 1. This is now fixed.

Dashboard generation and deployment using Apama Plugin for Eclipse has been removed

Dashboard generation and deployment using Apama Plugin for Eclipse is no longer supported and has therefore been removed. It is recommended that you create and deploy dashboards from the command line instead, see [Generating a deployment package from the command line](/dashboard-deployment/preparing-dashboards-for-deployment#generating-deployment-package-from-command line) for detailed information.

Digital Event Services transport connectivity plug-in has been removed

Digital Event Services integration is no longer supported and the Digital Event Services transport connectivity plug-in has therefore been removed. It is recommended to instead use the correlator-integrated support for the Java Message Service (JMS). See Correlator-integrated support for the Java Message Service (JMS). See also the announcement in Removed and deprecated features in 10.15.2.

EPL warning needed - now makes it easier to use "=" when you mean ":=".

Since discarding the value of an expression used as a statement was permitted in 10.15.4 one of the situations for a common EPL programming error, using “=” instead of “:=”, was permitted. In order to prevent this common error, we have added a warning if this case is detected. This warning will be promoted to an error in a future release.

GenericRequest gives more detailed error when specifying an invalid source parameter

Previously, the Cumulocity IoT transport returned a nonsensical message and a success state when sending a GenericRequest with an invalid source parameter. Now an error state is returned correctly and a more helpful message is provided. Instead of the previously returned message “Incorrect type in get (you asked for map but it’s actually string)”, you will now receive the following message: “Failed to parse source id, or source was in the wrong format (should be {source: {id: id}})”.

HTTP client form encoding does not work for the Cumulocity IoT binaries API.

The HTTP client incorrectly added “Content-Length” headers to multi-part forms that were sent with chunk encoding. This is no longer the case. In addition, the HTTP client now supports adding form metadata to string type inputs.

HTTP Client transport supports digest authentication

The HTTP client transport now supports digest authentication in addition to basic authentication. This improves the security of HTTP authentication beyond just using basic authentication. To use digest authentication, specify HTTP_DIGEST for the authentication/authenticationType configuration option. For more information, see Configuring the HTTP client transport.

ICU time-zone data updated to 2024a

Apama 10.15.5 incorporates the ICU (International Components for Unicode) time-zone data update 2024a, which is the most recent update at the time of release. This will update time-zone data used by the correlator and Time Format EPL plug-in.

MATLAB plug-in is now deprecated

The plug-in providing access to MATLAB® analysis and modeling from Apama applications is now deprecated and will be removed in an upcoming release.

New Cumulocity Notifications 2.0 connectivity bundle in 10.15.5

A new Cumulocity Notifications 2.0 connectivity bundle has been added for receiving notifications from Cumulocity IoT using the Notifications 2.0 mechanism. The existing Cumulocity Client connectivity bundle that uses the legacy long-polling mechanism is deprecated in favor of this new bundle. In addition, it is now possible to select the Cumulocity REST Support connectivity bundle if you do not receive any notifications. You must only select one of the three bundles for your project.

As part of this change, the CumulocityIoT.properties configuration file provided in the Cumulocity Client bundle has been split into two files to provide uniformity between the three bundles. Existing projects will continue to work as is, but new projects will have separate properties files. For more information, see The Cumulocity transport connectivity plug-in. For migration information, see Migrating from Cumulocity Client to Cumulocity Notifications 2.0.

Note: The Cumulocity Notifications 2.0 feature is currently in private preview. If you would like to have it enabled for your tenant, please contact Cumulocity IoT Operations.

New EPL actions for creating and parsing the c8y_Position fragment

The ManagedObject event has a unique way of handling c8y_Position fragments that have the parseable float values added to their position property, and all other values added to their params property. All other events of the Cumulocity IoT transport API simply add the fragment as a whole to their params property. This has made it cumbersome to robustly copy c8y_Position to a ManagedObject from another event and vice versa. To make this easier, two new helper actions have been added to the ManagedObject event to populate ManagedObject from a c8y_Position object, and conversely create the object from the ManagedObject event.

  • action setC8yPosition(any c8y_Position)

    This new action sets the ManagedObject using the c8y_Position object. This must be of the form dictionary<string, any>, but is validated prior to use. Example:

    on all Event(type="locationUpdate") as evt
    { ManagedObject mo := new ManagedObject;
      mo.id := evt.source;
      mo.setC8yPosition(evt.params.getOrDefault("c8y_Position"));
      send mo to ManagedObject.SEND_CHANNEL; }
    
  • action getC8yPosition() returns dictionary<string, any>

    This new action produces a c8y_Position style object from the ManagedObject. Example:

    evt.params["c8y_Position"] := mo.getC8yPosition();
    

New utility functions for the TimeFormat event library

The following utility functions have been added to the TimeFormat event library to help with comparing and manipulating datetimes. As with the existing functions, the new utility functions work for the local time zone, an arbitrary time zone and, where appropriate, the UTC time zone.

  • dateComponent: Gives the datetime for the previous midnight in the time zone at the given datetime.
  • timeComponent: Gives the number of seconds since the previous midnight in the time zone at the given datetime.
  • daysSinceEpoch: Gives the number of Julian days since the beginning of the epoch at the given datetime.
  • getOffset: Gives the offset in seconds of the time zone from UTC at the given datetime, taking into account any daylight savings that may be being applied.
  • getRawOffset: Gives the base offset in seconds of the time zone, that is, without daylight savings applied.

For usage information, see the API reference for EPL (ApamaDoc).

Persistent EPL monitors are now deprecated

Persistent EPL monitors and the correlator persistence feature are now deprecated and will be removed in an upcoming release. This also includes the com.apama.memorystore.Store.prepareCorrelatorPersistent API. It is recommended to instead write EPL to explicitly store any required application state either in the Cumulocity IoT platform, or for non-Cumulocity deployments using some other mechanism such as the MemoryStore.

pywin32 updated to build 306

Apama now ships build 306 of the pywin32 library with Python 3.9.5 instead of build 300. See https://pypi.org/project/pywin32/ for detailed information on what this change means.

R plug-in is now deprecated

The R plug-in is now deprecated and will be removed in an upcoming release. It is recommended to instead write EPL Python plug-ins for statistical computing use cases previously covered by the R support, making use of the large ecosystem of third-party Python libraries.

string.toDecimal() no longer throws ParseException

Previously, performing string.toDecimal() for a string with an invalid conversion resulted in a ParseException error. This now returns “0.0”. This was done to make the experience of using toDecimal() similar to using toFloat(). If you want the previous strict behavior, use decimal.parse() instead.

Support for UBI 9

In order to keep up to date with currently active platforms that incorporate the latest security fixes, the Apama images on Docker Hub are now available in Red Hat Universal Base Image (UBI) 9. These variants have tags ending in “-ubi9”. The previous UBI 8 variants are now deprecated and will cease to be maintained in a future release. See also Deploying Apama applications with Docker.

The .NET API for Apama is now deprecated

The .NET API for Apama is now deprecated and will be removed in an upcoming release. It is recommended to use connectivity plug-ins (such as the HTTP server) for communicating with Apama from external custom clients.

UBI 8 images deprecated

The published UBI 8 variants of the Apama images are now deprecated and will cease to be maintained in a future release. It is recommended that you now use the new UBI 9 variants of the Apama images. See also Support for UBI 9.

Updated Cumulocity SDK version to get fix for vulnerable 3rd party component "json".

The Cumulocity SDK Java version has been updated to version 1015.0.518 to resolve high severity vulnerabilities affecting the previous version.

Vulnerable 3rd party component spring-core.

The Spring libraries have been upgraded from 5.3.22 to 5.3.31.

Windows for production purposes is now deprecated

Using Windows for production purposes is now deprecated and will no longer be supported in future releases. In the future, only Linux should be used to deploy production workloads.

10.15.4.1

EPL warning added since it's now easier to unintentionally use "=" when you mean ":=".

Since discarding the value of an expression used as a statement was permitted in 10.15.4, one of the situations for a common EPL programming error, using “=” instead of “:=”, was permitted. In order to prevent this common error, we have added a warning if this case is detected. This warning will be promoted to an error in a future release.

10.15.4

Alarm and Operations events have new constants for status and severity members

The Alarm and Operation events have new constants which define the valid values for their respective status and severity members. This allows more robust coding and eliminates runtime errors caused by typographical errors with literal strings.

Base64 EPL Plug-in for encoding and decoding

A new EPL plug-in has been added to encode and decode strings in Base64 format from EPL. For more details, see Using the Base64 plug-in.

CHANNEL constant deprecated on Cumulocity events

To avoid confusion as to whether events were being sent towards Cumulocity IoT or being received back as updates from Cumulocity IoT, the Cumulocity event definitions API was changed in a previous release (10.5.2.0) so that the existing CHANNEL constant was deprecated and replaced by either SEND_CHANNEL or SUBSCRIBE_CHANNEL constants as appropriate. Some events were omitted from these changes and this has now been rectified. Therefore, the CHANNEL constant is deprecated on the following events:

  • SubscribeMeasurements
  • UnsubscribeMeasurements
  • FindManagedObjectResponseAck
  • FindMeasurementResponseAck
  • GenericResponseComplete
  • Subscribe
  • Unsubscribe

In addition, a new SUBSCRIBE_CHANNEL or SEND_CHANNEL has been added to the following events instead:

  • SubscribeMeasurements
  • UnsubscribeMeasurements
  • Subscribe
  • Unsubscribe

The constants on the following events are instead replaced with SUBSCRIBE_CHANNEL constants on their corresponding Response types:

  • FindManagedObjectResponseAck
  • FindMeasurementResponseAck
  • GenericResponseComplete

The API Reference for EPL (ApamaDoc) has been amended to make it clear that acknowledgement events are always received on the same channel as the corresponding response events, and to avoid confusion the CHANNEL constant has been deprecated on those acknowledgement events where it had been defined.

com.apama.functional.Fn and com.apama.functional.Functional events now have a new method "mapKeys"

The com.apama.functional.Fn and com.apama.functional.Functional events now have a new method mapKeys which can return a dictionary container with modified keys. For more details, see Functional operators and the API reference for EPL (ApamaDoc).

EPL discards unused return values

EPL monitors are no longer terminated by default when an uncaught exception is thrown in some cases.

Programming errors and unexpected data in incoming events can cause an uncaught exception in EPL. This causes the monitor instance to be terminated, rendering the application unusable. To provide a better experience, exceptions thrown from listeners and stream listeners which are not caught no longer terminate the monitor instance. Instead, they just stop the handling of the current event.

Developers are still encouraged to correctly catch and handle the exceptions in their EPL. If an exception is thrown and not caught, it is still possible for the events to be lost and not correctly handled.

This is a change in behavior. Some users may be relying on this previous behavior, in combination with an ondie() action. EPL with an ondie() action retains the previous behavior of always terminating the monitor and calling ondie(). If you need to retain the previous behavior, you can add an empty ondie() action to your monitor.

Note:

If you have an action that is called from within a stream query (for example, as part of a where or group by clause, or a window definition) which throws an exception, this still terminates the monitor instance.

EPL monitors are no longer terminated by default when an uncaught exception is thrown in some cases

EPL monitors are no longer terminated by default when an uncaught exception is thrown in some cases.

Programming errors and unexpected data in incoming events can cause an uncaught exception in EPL. This causes the monitor instance to be terminated, rendering the application unusable. To provide a better experience, exceptions thrown from listeners and stream listeners which are not caught no longer terminate the monitor instance. Instead, they just stop the handling of the current event.

Developers are still encouraged to correctly catch and handle the exceptions in their EPL. If an exception is thrown and not caught, it is still possible for the events to be lost and not correctly handled.

This is a change in behavior. Some users may be relying on this previous behavior, in combination with an ondie() action. EPL with an ondie() action retains the previous behavior of always terminating the monitor and calling ondie(). If you need to retain the previous behavior, you can add an empty ondie() action to your monitor.

Note:

If you have an action that is called from within a stream query (for example, as part of a where or group by clause, or a window definition) which throws an exception, this still terminates the monitor instance.

Errata: GetCurrentUserResponseFailed was erroneously listed as an event of the com.apama.cumulocity package

In Miscellaneous changes in 10.15.3 affecting backwards compatibility, GetCurrentUserResponseFailed was erroneously listed as an event of the com.apama.cumulocity package which has been updated to improve consistency in error handling. It has now been removed from this list.

flushAllQueues management request can now flush queues within connectivity plug-ins

The flushAllQueues management request can now flush queues within connectivity plug-ins via a control message sent down the chain. This has been implemented for the HTTP client and the Batch Accumulator codec. For more details, see Flushing background queues in connectivity plug-ins.

HTTP Client transport supports max response limits

The HTTP client comes with two new configurable options, maxResponseKB and maxResponsePolicy, that enable the user to restrict the size of the server response. These options can be set for the entire transport, or on individual requests. Support has also been added to the predefined event definitions (see Using predefined generic event definitions to invoke HTTP services with JSON and string payloads). For more details, see Configuring the HTTP client transport.

Mixed-type sequence and dictionary literals are now treated as <any> by default

It is no longer necessary to add an <any> cast around the first item in a sequence or dictionary literal that contains a mixture of different types. For example, the following sequence literal which produces a sequence<any>

[<any> 12345, "a string"]

can now be written more simply as

[12345, "a string"]

This simplifies the functional example given in EPL enhancements in 10.15.3: the <any> is no longer needed in the setFields call.

New built-in convenience methods on string, sequence and dictionary

The EPL language has been enhanced with some convenience methods for common operations such as checking if a string or sequence contains a specified substring or item, and determining if a string, sequence or dictionary is empty.

There is a method to get a sequence value with a fallback default if there is no value at the specified index:

mySequence.getOr(100, "default value");

There is also a powerful regular expression group search for strings that finds the first match and returns the text captured by each “(…)” group of the regular expression:

// Prints ["Bob", "Eve"]
 print "Today Bob met Eve".groupSearch("([a-zA-Z]) met ([a-zA-Z])").toString();

The following table lists all new methods:

New methods Description
string.contains(), sequence.contains() Determines whether the string or sequence contains the specified substring or item. This is a convenient alternative to checking the find() or indexOf() integer value.
string.groupSearch() Finds the first regular expression match in this string, and return a list of the matched “(…)” groups.
string.startsWith(), string.endsWith() Determines whether the string has the specified prefix or suffix.
string.rfind() Locates a string within this string, starting from the right (that is, from the end) of the string.
string.substringFrom() Extracts part of this string, starting at a specified character and ending at the end of the string.
string.quote() Adds quotation marks and escaping according to the standard EPL event representation. This method may be useful for logging strings that may contain spaces and newline characters. This is the reverse of the existing string.parse() method.
string.isEmpty() Determines whether the string has a length of zero characters.
dictionary.isEmpty(), sequence.isEmpty() Determines whether the size is 0.
any.isEmpty(), optional.isEmpty(), listener.isEmpty(), Channel.isEmpty(), chunk.isEmpty() Aliases for the existing empty() methods on these types. It is recommended to use the new isEmpty() methods, but the use of empty() is not deprecated.
sequence.getOr() Gets the value at the specified index, or a specified fallback value if the index is not valid.

For more information, see the API reference for EPL (ApamaDoc).

onTimeout action now returns the wait listener it creates

The functional onTimeout action now returns the wait listener it creates so that it can be quit if needed. For more details, see Functional listeners.

PySys upgraded to 2.2

This release of Apama ships with a new version of PySys, version 2.2.

The new version improves usability of the pysys command line, gives a better experience editing and running tests from IDEs, improves Ctrl+C cancellation handling, and adds a few new methods and parameters you can use in your tests.

Some highlights in this release are:

  • Instead of running pysys print, you can use the equivalent pysys ls to save some typing. Try out the new --verbose flag which groups related tests (sorting by title), and prints the full path of each test file (for example, XXX/pysystest.py) so you can easily open up any tests of interest directly from your IDE or shell.
  • Now that pysys run accepts directory paths (not only test ids), you can use your shell’s directory tab completion to help specify the tests to run.
  • Get better code completion and avoid warnings when editing tests inside your Python-enabled IDE/editor by following the new best practices described in the PySys documentation. For example, use mix-in helper classes (not <test-plugin>) for sharing logic between tests, use import pysys.basetest, pysys.mappers not import pysys at the start of all tests, and make sure references to pysys.basetest.BaseTest in your test class inheritance list are fully qualified. To use a Python IDE with PySys, configure it to use the Python that is installed by the full version of Apama. If your project has any custom Python extensions configured with a <pythonpath> element, you should add that to your editor’s PYTHONPATH too.
  • Use the new BaseTest.assertGrepOfGrep() method if you are extracting a line of text from a file and then validating it with a regular expression. You will get much more descriptive messages if a test fails using this approach rather than doing it all in a single assertGrep.
  • See also the new BaseTest.createThreadPoolExecutor, BaseTest.listDirContents and pysys.mappers.IncludeMatches methods.

There are also several bug fixes and minor enhancements. See the PySys Change Log (PySys Test Framework > Change Log in the API reference for Python) for more information. There are no breaking changes in this release.

Range-based queries (such as FindManagedObject) attempt to retrieve all resources matching the query parameters by default.

Range-based queries (such as FindManagedObject) attempt to retrieve all resources matching the query parameters by default. Explicitly setting a value for currentPage or setting withTotalPages to false can improve the query performance by disabling paging. See the information on REST usage and query result paging in the Cumulocity IoT OpenAPI Specifications for more information.

String concatenation operator + supports non-string operands

If only one operand expression is of type string, then string conversion is now performed on the other operand to produce a string at runtime. For more details, see Additive operators.

Support for negative indexes for sequence[...] access

Instead of throwing an “out of bounds” exception, it is now possible to refer to items near the end of a sequence by specifying a negative index. For example, seq[-1] gives the last item in the sequence and seq[-2] gives the item before that. Negative sequence indexes can also be used in the sequence.remove() and sequence.insert() methods.

Note that negative values cannot be specified when accessing a sequence item using the any.getEntry() method.

Truncated message in correlator-integrated JMS.

An issue has been fixed where JMS mapping rules using convention-based mapping from XML to an Apama event could randomly truncate the contents of XML text nodes, especially in large XML documents.

Vulnerable third-party component Guava found in Apama images for Docker (designer_headless).

The Guava third-party jar has been removed from the Apama images for Docker images to address potential security issues.

10.15.3.4

Consume Cumulocity SDK with fixes for multiple events and client re-connects.

The Cumulocity Java SDK has been upgraded to v1015.0.488 to fix an issue where subscriptions for real-time notifications were lost when switching between Cumulocity core nodes.

10.15.3.3

Consume Cumulocity IoT SDK v1015.0.456+ to get fixes for subscription retry.

The Cumulocity IoT Java SDK has been upgraded to v1015.0.456 to fix an issue where subscriptions for real-time notifications were lost when switching between Cumulocity IoT core nodes.

Long-running Cumulocity query messages are misleading when sending large numbers of sequential requests.

More information has been added to Cumulocity IoT long-running query messages, as well as for distinguishing between queries and batches.

10.15.3.2

JMS convention-based mapper truncates XML text data.

An issue has been fixed where JMS mapping rules using convention-based mapping from XML to an Apama event could randomly truncate the contents of XML text nodes, especially in large XML documents.

10.15.3.1

Vulnerable third-party "woodstox-core" found in Apama images.

The version of woodstox-core has been upgraded from 6.3.0 to 6.5.0.

Vulnerable third-party components found in Apama (PAM) Docker images - JDK, C8Y SDK.

The version of Java included in the Docker images has been updated to 11.0.19 to resolve security issues.

10.15.3

A new EPL library to support programming in a functional style

A new EPL library has been added to support programming in a functional style for quicker development, fewer for loops, and more concise and readable code. For example:

// Calculate the average of the values in the sequence = 20
Fn.reduce([10, 20, 30], Fn.mean());

// Filter strings by regular expression using the string.matches action = ["Hello Alice", "Hello Al"]
Fn.filter(["Hello Alice", "Hello Bob", "Hello Al"], Fn.callAction("matches", ["Hello Al.*"]));
// Use Functional() to chain multiple operations: filter out odd numbers, then reduce the sequence to a single value containing the sum = 70
Functional([10, -5, 20, 40]).filter(Fn.even).reduce(Fn.sum);

The library provides similar capabilities to Python’s functools module. It can be added to Apama projects with a new Functional EPL Library bundle which provides two main EPL types: com.apama.functional.Fn and com.apama.functional.Functional. These provide functional operations such as map, filter and reduce in both a functional style and a fluent style. They operate on EPL sequence and dictionary containers and on a new “generator” concept that lazily calculates infinite lists.

There is support for partial binding of action arguments:

Fn.map(["Bob", "Alice"], Fn.partial(Fn.concat, "Hello ")).toString(); // = ["Hello Bob","Hello Alice"]

There are also some new actions to make event sending and listening more concise, including the following:

// Initialize an event (providing only the subset of fields you care about), send and extract the request id for use in listener
// (it is also possible to initialize event fields by name rather than by position)
on MyResponse(reqId=<integer> Fn.sendToChannel(MyEvent.SEND_CHANNEL,
        Fn.setFields(new MyEvent, [<any>createRequestId(), "/myrequest"])
        ).getEntry("reqId") ) as response
{
...
}

// Listen for events with an identifier matching each value from this sequence, and calls a completion action when all have arrived
// (similar to an EPL "and" event expression). A different callback is executed if the specified timeout expires.
Functional(sequenceIDs)
.waitForAllCompleted("MyResponse", "id", onCompleted)
.onTimeout(TIMEOUTSECS, onTimeout);

For detailed information, see Using functional operations in EPL. See also the API reference for EPL (ApamaDoc) for detailed information on com.apama.functional.

ApamaDoc and Eclipse provide incorrect action return type from action body.

Action variables declared in the body of an action could cause incorrect ApamaDoc to be generated for the outer action. This issue has now been fixed.

at() event operator in EPL for listeners can now be provided with a time zone as an optional final argument

The at() event operator in EPL for listeners can now be provided with a time zone as an optional final argument. This can be in string form or (recommended) one of the new constants in the TimeZone namespace. For more details, see Triggering event listeners at specific times and the TimeZone namespace in the API reference for EPL (ApamaDoc). The new constants can also be used with the TimeFormat event library.

Code coverage broken by wrong correlator MD5 hashes on UTF8 BOM files.

epl_coverage uses a hash comparison for locating the files in the specified source directory. An issue has been fixed so that this will now work for files in a non-UTF-8 locale, or with a UTF-8 byte order marker, which previously would have been missed.

Cumulocity events updated to improve consistency in error handling

The following events of the com.apama.cumulocity package have been updated to improve consistency in error handling:

  • FindAlarmResponseAck
  • FindEventResponseAck
  • FindManagedObjectResponseAck
  • FindMeasurementResponseAck
  • FindOperationResponseAck
  • FindTenantOptionsResponse

Note: In 10.15.3, GetCurrentUserResponseFailed was erroneously included in the above list. It was removed from this list in 10.15.4.

In all cases of a server acknowledgement, the response is now sent to the correct response channel. The above events now have the following additional members:

  • boolean error - Set to true if the find request received either an error content type or an error response HTTP return code.
  • integer status - The HTTP return code.
  • string errorDetails - Details of the error. You should now use the updated events listed above instead of the Error event since it is sent to the same channel as the updated events. For more details, see the API reference for EPL (ApamaDoc).

Cumulocity IoT transport supports handling of standard Cumulocity IoT microservice proxy environment variables

The Cumulocity IoT transport now supports handling of standard Cumulocity IoT microservice proxy environment variables such as PROXY_HTTP_HOST and PROXY_HTTP_PORT. See also Proxy variables in the Cumulocity IoT documentation.

epl_coverage tool has a new option: --source-encoding

The epl_coverage tool has a new option: --source-encoding. Previously, all source EPL files were assumed to be in UTF-8 encoding, which is also the new default. You can alternatively provide the encoding local in which case the EPL files are read in the local system encoding, unless they start with a UTF-8 byte order marker. See also Creating code coverage reports.

Extra files removes from old C/C++ EPL plug-in API.

Some files such as the header files AP_CorrelatorInterface.hpp and AP_PluginCommon.h from the old plug-in API removed in 10.15 still existed in the product. These files were not required and any plug-ins which still include them should remove the reference.

HTTP server transport now supports the creation of different chains for incoming requests on the same port

The HTTP server transport now supports the creation of different chains for incoming requests on the same port, differentiated on the HTTP request path. The use of separate chains enables separation of parsing configuration for different paths and provides performance improvements if they are delivering to different channels subscribed by different contexts. A new configuration option called matchPathPrefixes is available for this. For detailed information, see Handling requests to different paths with different chains.

ICU time-zone data updated to 2023c

Apama 10.15.3 incorporates the ICU (International Components for Unicode) time-zone data update 2023c, which is the most recent update at the time of release. This will update time-zone data used by the correlator and Time Format EPL plug-in.

Improvements to Apama helper classes for PySys

  • The CorrelatorHelper.injectEPL (and related inject* methods) now expands ${...} project properties automatically, for example, correlator.injectEPL(['${APAMA_HOME}/monitors/ManagementImpl.mon']). Additionally, if the same .mon path is listed more than once in a single call to injectEPL(), the later duplicate names are silently ignored rather than producing an injection error.
  • A new project property apamaCorrelatorLicense can be used to specify the path of the correlator license. When the property is not specified, PySys continues to look in the default directory for the license.

Listener event fields allow specifying just first few fields of interest

When using positional syntax for specifying which event fields to listen for, it is now possible to specify just the first few fields of interest, without explicitly adding “*” wildcards to cover the later fields. For example, given an event E with 4 fields, on E(1, 2, 3) is now valid and is semantically equivalent to E(1, 2, 3, *). See also Using positional syntax to listen for events with particular content.

persistenceStoreLocation created even if persistence is disabled.

The correlator does not validate the persistence storeLocation value when persistence is disabled, and will treat the empty string the same as being unset (uses the current working directory).

Support cgroups v2 and fix v1 to check /proc/self/cgroups.

Support has been added for cgroups v2 and the implementation of cgroups v1 has been fixed.

Type mismatch while trying to cast actions.

If calling anything that resulted in a call to any.getaction(), this could cause an incorrect cast exception or cause a crash. This has now been fixed.

Vulnerable third party component Snakeyaml used.

SnakeYAML has been updated to version 2.0 to resolve vulnerabilities associated with earlier versions. The Jackson project libraries have been upgraded to be compatible with the SnakeYAML version.

10.15.2.4

data_t parsing performance improvements.

Performance has been improved when parsing large events with lots of strings from connectivity plug-ins.

10.15.2.3

Invalid access to "pagesLeft" on large paged queries.

Diagnostics logging for long running queries could report errors in the log file of the form “Error while transforming message: Element not in map”. This issue has been fixed.

Unable to send email with empty "bcc" field.

Starting with release 10.16, the Cumulocity IoT platform performs more strict validations on incoming fields of email requests.

When the bcc field of the com.apama.cumulocity.SendEmail event was empty, the Apama HTTP client transport for Cumulocity IoT incorrectly forwarded the bcc field as an empty string instead of as a JSON array, causing requests to be rejected. This issue has now been resolved.

10.15.2.1

The Cumulocity measurement payload contains null keys.

A regression was introduced in 10.15 when a measurement was received containing multiple fragments/series with multiple “value and unit” entries. NULL keys were generated in the resulting representation. This issue has been fixed.

Vulnerable component OpenSSL (1.1.x and 3.0.x).

OpenSSL has been updated to version 3.0.8 to resolve high severity vulnerabilities.

10.15.2

Apama can be installed using apt

Apama 10.15 can be installed on Debian-based Linux distributions, including for thin-edge.io, using the native apt package management tool. For information about the repository, see https://download.cumulocity.com/Apama/Debian/. See also Installing Apama using Debian apt.

Apama dashboards are now deprecated and will be removed in a future release.

Apama dashboards are now deprecated and will be removed in a future release.

Dashboard Builder now supports a command line option to configure the palette style

The Dashboard Builder now supports the following command line option for the palette style: --paletteStyle flat|classic. The default value is classic. For more information, see Command-line options for the Dashboard Builder.

DashboardKeystore expires in 2024 March.

Our DashboardKeystore certificate was due to expire and so has been renewed, this means a new key has been generated. While this certificate is intended for demo purposes only and we recommend use of a unique certificate, any existing dashboard deployments, using our old DashboardKeystore, will need to be redeployed with this certificate.

Default pidfile location is not writeable on thin-edge.io, resulting in the correlator failing to start up.

Deploying projects to thin-edge.io with the default engine_deploy output will now start without errors creating a pidfile.

Digital Event Services transport connectivity plug-in has been deprecated

Digital Event Services integration is now deprecated and the Digital Event Services transport connectivity plug-in will therefore be removed in a future release. It is recommended to instead use the correlator-integrated support for the Java Message Service (JMS).

epl_coverage tool fixes potential duplicate or unmerged entries for the same .mon file

A change has been introduced for the EPL coverage tool to correct a few issues with the tool, such as potentially duplicate or unmerged entries for the same .mon file. That is, the EPL code coverage tool now uses slightly different rules for merging coverage information for each unique .mon file, and for applying the include and exclude filters. In some cases, this can change the .mon paths listed in coverage reports. The main change is that if a --source directory is specified, the tool now uses that for the path of all injected .mon files with binary contents matching a file of the same name in the source directory. Additionally, the include and exclude filters are now applied not only to the .mon file paths that were injected, but also to the searching of --source directories. For best results, it is recommended to specify a --source directory and adjust any existing --exclude patterns if needed to ensure that the right EPL files are included in the coverage reports. For those using PySys to run the coverage tool, the sample project at APAMA_HOME/samples/pysys/pysysproject.xml has been updated to demonstrate a best practice EPLCoverageWriter configuration. The PySys EPLCoverageWriter no longer implicitly excludes testRootDir/** if the testRootDir (or any subdirectory) is specified as a source directory. See also Creating code coverage reports.

IAF is now deprecated

The IAF is now deprecated, with the exception of use within Apama for Capital Markets. Its architecture is superseded by connectivity plug-ins.

The following standard IAF plug-ins are now deprecated and will be removed in a future release:

  • Apama Database Connector (ADBC)
  • File IAF adapter (JMultiFileTransport)
  • Basic File IAF adapter (FileTransport/JFileTransport)

New type conversion support

Support for the following type conversions has been added using explicit cast operations:

  • integer to float
  • any(integer) to float
  • integer to decimal
  • any(integer) to decimal
  • float to decimal
  • any(float) to decimal
  • decimal to float
  • any(decimal) to float For example:
integer i := 10;
float f := <float> i;

See also Type conversion and Handling the any type.

Performance improvement for cloning any - distinguish between methods that mutate the data of an object to those that mutate the metadata of an object.

Performance improvements to coassigning large events when you reference string or any members using .clone(). This includes Cumulocity IoT Measurement events when using the getFragments method.

The Data Player, which relies on ADBC, is now deprecated and will be removed in a future release.

The Data Player, which relies on ADBC, is now deprecated and will be removed in a future release.

Universal Messaging transport connectivity plug-in is now deprecated

The Universal Messaging transport connectivity plug-in is now deprecated and will be removed in a future release. It is recommended to instead either use the MQTT transport connectivity plug-in or the correlator-integrated support for the Java Message Service (JMS).

User status metrics with names containing labels in the form {keyA=valueA,keyB=valueB} are now converted to Prometheus metrics with those labels in the Prometheus labels

User status metrics with names containing labels in the form {keyA=valueA,keyB=valueB} are now converted to Prometheus metrics with those labels in the Prometheus labels. The HTTP server transport now uses this syntax for chains that it creates dynamically.

For example, if you have somename{key=value} as part of the user status name, this is now converted to a Prometheus label on the metric somename. Previously, a user status with a name like this did not appear in Prometheus at all.

As a result, the HTTP server transport, which used to prefix all metrics on its chains for example with httpServer_instance_5_, now prefixes them with httpServer{chain=5} instead. This means, that the resulting metrics now look like this:

sag_apama_correlator_user_httpServer_metricname{chain=5}

instead of

sag_apama_correlator_user_httpServer_instance_5_metricname

10.15.1.2

Duplicate Prometheus metrics (for example, name of slowest input queue).

Stale Prometheus metrics are now removed from the metrics endpoint to avoid it growing constantly over time. This includes both metrics with labels when old labels are no longer used and user-status metrics which have since been deleted.

String assignment/clone defeats copy coassignment optimizations.

Performance improvements have been made when coassigning large events in EPL listeners. In particular, this will improve the handling of Cumulocity IoT Measurement events for high-throughput use cases.

10.15.1

Docker images cannot locate OpenSSL cert.pem.

Apama 10.15.0 introduced a regression to the product Docker images from Docker Hub which broke the validation of public SSL certificates. This regression has now been fixed.

EPL Plug-ins using chunks no longer terminate correlator when injected in persistent monitor

EPL plug-ins written in C++ which return or use chunks would previously terminate the correlator if injected in a persistent monitor, if a chunk was ever used. Now these plug-ins will fail to inject instead. This failure to inject occurs whether or not chunks are actually used. To resolve this issue, remove use of chunks from the plug-in which is used in a persistent monitor, or only import that plug-in into a non-persistent monitor.

Improvements to Apama helper classes for PySys

The template for newly created PySys test projects and individual PySys tests has been changed to use a different approach for making Apama helper functionality available through the self.apama field. The previous approach was to use a <test-plugin> element in the pysysproject.xml file to make the ApamaTestPlugin available to tests as self.apama. However, this resulted in testcases that could not be fully parsed by Python IDEs and editors. Instead, a new pattern is now recommended in which each test explicitly subclasses a new helper class which adds the self.apama field in an IDE-friendly manner. There is no need to remove the use of <test-plugin> from existing projects, but it is no longer recommended for new projects, and in all cases, it is recommended to inherit from ApamaHelper in new testcases:

from apama.testplugin import ApamaHelper
class PySysTest(ApamaHelper, pysys.basetest.BaseTest):
...

To make use of editor code completion and navigation, configure the editor with a PYTHONPATH environment variable matching the PYTHONPATH from an Apama command prompt or apama_env invocation. You may also need to add any custom PySys extension classes of your own from your pysysproject.xml. Some editors allow environment variables like this to be configured by creating an .env file in your project directory.

Long-polling Cumulocity IoT transport raises an unhelpful error for some misconfiguration.

The Cumulocity IoT transport throws an exception when the variable C8Y_BASEURL is set and a properties file is provided. This issue is seen when the variable C8Y_BASEURL is set but username/password are not provided and the properties file is expected to be invoked. The exception is now resolved. Also, now when this variable is set, then a statement is logged saying that the transport is started in microservice mode and username/password are required.

10.15.0.2

Vulnerable third-party component OpenSSL 3.x used.

OpenSSL has been updated to version 3.0.7 to resolve critical vulnerabilities affecting the previous version.

10.15.0

Apama Community Edition Core no longer contains JMS or Software AG Suite connectivity

The core edition of the Apama Community Edition in 10.15.0 no longer contains connectivity to JMS or to Software AG Suite products such as Terracotta via the Distributed Memory Store or Universal Messaging. Connectivity to Cumulocity IoT has been moved to a separate bundle available alongside the core edition.

If you need any of the functionality that has been removed, install using either the full version of the Community Edition installer or the official Apama downloads area.

Apama queries are now deprecated and will be removed in a future release

Apama queries (that is, .qry files) are now deprecated and will be removed in a future release. Deprecation messages are now logged when Apama queries are loaded. The ATM fraud demo in Apama Plugin for Eclipse (located under samples/studio/demos/ATMFraud) and the samples under samples/queries and samples/docker/applications/Queries have already been removed in 10.15.0.

The Dashboard Generation wizard uses Apama queries to generate the dashboards. Therefore, dashboard generation is also deprecated and will also be removed in a future release.

Apama's in-process API for Java (JMon) is now deprecated and will be removed in a future release.

Apama’s in-process API for Java (JMon) is now deprecated and will be removed in a future release. Deprecation messages are now logged when a JMon application is loaded. The samples under samples/java_monitor have already been removed in 10.15.0. It is recommend that you now use EPL plug-ins written in Java instead of JMon applications. The JMon migration guide in our Tech Community will help you to port your JMon applications.

C++20 support for projects

The Apama C++ samples and libraries have been updated to be compatible with the latest C++ standard, which is C++20. Therefore, you must now update any projects that are compiling against those libraries so that they use the following flags (or the equivalent for your compiler):

  • For Visual Studio 2019: /std:c++latest
  • For G++ 8.5.0: -std=c++2a
  • For Clang 12: -std=c++20

Chain names can no longer contain newline characters

In previous versions, chain names could contain newline characters, thus causing newlines in the correlator log file. For security reasons, this is no longer allowed. If a newline character is detected in a chain name, an exception is now thrown.

Command Central integration is now deprecated and will be removed in a later release.

Command Central integration is now deprecated and will be removed in a later release. For installation or deployment of Apama, it is recommended to instead either use the Apama Installer https://download.cumulocity.com/Apama/, or to use modern containerization methods at runtime via Docker or Kubernetes with the Apama images. Instead of monitoring Apama from Command Central, it is recommended that you now use Prometheus. See also Deploying Apama applications with Docker, Deploying Apama applications with Kubernetes, and Monitoring with Prometheus.

Cumulocity IoT transport offers new configuration options

The following new configuration properties, which correspond to existing configuration options in the YAML configuration file, are now available:

  • CUMULOCITY_REQUEST_ALL_DEVICES (corresponds to requestAllDevices)
  • CUMULOCITY_SUBSCRIBE_ALL_MEASUREMENTS (corresponds to subscribeToAllMeasurements)
  • CUMULOCITY_SUBSCRIBE_DEVICES (corresponds to subscribeToDevices)
  • CUMULOCITY_SUBSCRIBE_OPERATIONS (corresponds to subscribeToOperations)

It is recommended to use the new configuration properties instead of editing the YAML configuration file. See also [Configuring the Cumulocity transport(/standard-connectivity-plugins/the-cumulocity-iot-transport-connectivity-plug-in/#configuring-the-cumulocity-iot-transport).

Cumulocity IoT transport sends requests concurrently to Cumulocity IoT platform

To improve performance, the Cumulocity IoT transport now sends requests to the Cumulocity IoT platform concurrently using multiple clients. New API and configuration options have been added to control this behavior. For a complete description of the new concurrency behavior and options to control it, see Optimizing requests to Cumulocity with concurrent connections.

Cumulocity IoT transport sends requests concurrently to Cumulocity IoT platform

The Cumulocity IoT transport now supports multi-tenant deployment. For this purpose, the following new configuration options are now available:

  • CUMULOCITY_MULTI_TENANT_APPLICATION
  • CUMULOCITY_MULTI_TENANT_MICROSERVICE_NAME

See also [Configuring the Cumulocity transport(/standard-connectivity-plugins/the-cumulocity-iot-transport-connectivity-plug-in/#configuring-the-cumulocity-iot-transport) and Working with multi-tenant deployments.

engine_deploy no longer allows you to set the deployment directory to be a subdirectory of the project directory

The engine_deploy tool no longer allows you to set the deployment directory (specified with either the -d or --outputDeployDir action) to be a subdirectory of the project directory. See also the announcement in Removed and deprecated features in 10.11.1.

Fix a potential exploit in the HTTPClient logging.

A security issue involving a malformed content-encoding header in the HTTP client transport has been resolved.

GenericRequest now always returns a GenericRequestComplete event

The GenericRequest event in the com.apama.cumulocity package has been changed to improve consistency in error handling. In all cases of a response from the server, a GenericResponseComplete event is now sent to the correct response channel. In addition, the GenericResponseComplete event now has the following additional members:

  • boolean error - Set to true if the GenericRequest received either an error content type or a non-2xx HTTP return code.
  • integer status - The HTTP return code.
  • string details - Details of the response which were not sent as a GenericResponse. It is recommended that you now use the updated GenericResponseComplete event instead of the Error event since the GenericResponseComplete event is sent to the same channel as the GenericResponse event.

In addition, there is a change in behavior where previously some GenericRequest events which received non-2xx response codes were still sending GenericResponse events. As of this version, only requests where error is false and the payload is JSON will send GenericResponse events. For more details, see the com.apama.cumulocity package in the API reference for EPL (ApamaDoc).

In previous versions, no response events were sent at all in some cases. As of this version, the GenericResponseComplete event is always sent for all GenericRequest events in any error case.

HTTP Client transport supports a new queueSize metric

The HTTP client transport now also supports a new queueSize metric which provides the current size of the client request queue in the HTTP transport. See also Monitoring status for the HTTP client.

HTTP Client transport supports executing multiple requests concurrently

The HTTP client transport now supports executing multiple requests concurrently. This can be used to improve performance. To enable concurrency, a new configuration option called numClients is now available. New metadata items called metadata.concurrencyControlKey and metadata.concurrencyControlFlush on each request can now be used to control which requests can be executed concurrently. For detailed information, see Executing HTTP requests concurrently.

ICU time-zone data updated to 2022a

Apama 10.15.0 incorporates the ICU (International Components for Unicode) time-zone data update 2022a, which is the most recent update at the time of release. This will update time-zone data used by the correlator and Time Format EPL plug-in.

Incorrectly configured Cumulocity IoT bundles no longer stop the correlator starting up

There is a change in behavior for the Cumulocity IoT transport. Previously, when a project was run with an incorrectly configured Cumulocity IoT bundle, the correlator failed to start. Now, the correlator keeps running and an error is logged indicating that it failed to create a Cumulocity IoT notification chain instance.

Java 8 support withdrawn for Apama API

This release of Apama withdraws support for using a Java 8 runtime with the Apama APIs. You must now use at least Java 11 to run Java applications using the Apama Java APIs.

Note:

Support for Java 8 was already removed in Apama 10.11.0. It was possible, however, to still use it in some situations. See also Java upgrade in 10.11.0.

Java libraries have been moved to the "APAMA_HOME/lib" directory

Java libraries such as Log4J, SLF4J, Jackson and Spring that used to be distributed under APAMA_HOME/../common/lib/ext have been moved to the APAMA_HOME/lib directory. You may need to update build scripts if you are using these jars from the installation to build custom plug-ins.

Kafka client library upgraded to 2.8.1

To fix a security vulnerability, the Kafka client library which is used by the Kafka transport has been upgraded from version 2.5.0 to version 2.8.1. Connecting to Kafka brokers older than 2.8.1 is no longer supported.

Kafka transport allows future customization of classpath

The classpath for the Kafka transport is now configured in a different way to allow future customization. It now also points to ${APAMA_HOME}/lib/kafka-clients.jar. See also Loading the Kafka transport.

Adding the Kafka bundle to your project gives you your own copy of the configuration. If you have already added the Kafka bundle to your project, you must edit the Kafka.yaml file in your project to use the new configuration. That is, you have to change the following line:

classpath: ${APAMA_HOME}/lib/connectivity-kafka.jar

to this:

classpath: ${APAMA_HOME}/lib/kafka-clients.jar;${APAMA_HOME}/lib/connectivity-kafka.jar

New Apama container images

Apama 10.15.0 introduces several new container images provided via Docker Hub and some of the existing container images have changed content. The new set of images is as follows:

Image name Purpose Change since 10.11
softwareag/apama-correlator Image for the correlator, including support for Java and Python. Does not contain connectivity to other Software AG products. Removed connectivity to Software AG products, including Cumulocity IoT.
softwareag/apama-correlator-minimal Smallest image for the correlator, aimed at pure-Apama use cases. Does not contain Java or Python support, or connectivity to other Software AG products. New image.
softwareag/apama-builder Project build and test tools for deploying and testing projects to be used with the apama-correlator and apama-minimal images in a multi-stage Docker build. Removed connectivity to Software AG products, including Cumulocity IoT.
softwareag/apama-correlator-suite Image for the correlator, containing support for Java, Python, and connectivity to the rest of the Software AG product suite and JMS. Closest equivalent to softwareag/apama-correlator in 10.11.
softwareag/apama-builder-suite Project build and test tools (including Apache Ant) for deploying and testing projects to be used with the apama-correlator-suite image in a multi-stage Docker build. Closest equivalent to softwareag/apama-builder in 10.11.
softwareag/apama-cumulocity-jre Base image for custom Cumulocity IoT microservices, containing the correlator and connectivity to Cumulocity IoT. Added Python support.
softwareag/apama-cumulocity-builder Project build and test tools for deploying and testing projects to be used with the apama-cumulocity-jre image in a multi-stage Docker build. New image.

Users may need to change the image that is included in their Dockerfile with a FROM line or a --build-arg override when building their project files.

The default Dockerfile that is added to projects through Apama Plugin for Eclipse builds with the softwareag/apama-builder and softwareag/apama-correlator images. If you need connectivity to any Software AG components, JMS or Cumulocity IoT, then you must select the appropriate image instead.

For more details, see Deploying Apama applications with Docker or the image pages on Docker Hub, for example, https://hub.docker.com/r/softwareag/apama-correlator.

Platform changes for 10.15

Apama 10.15.0 runs on the platforms listed in the Supported Platforms document for version 10.15. This is available from: Supported platforms.

Check the above mentioned Supported Platforms document for details about the newer platform versions supported by Apama 10.15.0. The platforms listed below are no longer supported.

Operating systems

No removals.

Compilers

No removals.

Application servers

  • Software AG Common Runtime 10.11 (replaced by support for a more recent version).

Databases

  • Microsoft SQL Server 2017 Enterprise (a later version is still supported).

JMS providers

  • IBM Universal Messaging 10.11 (replaced by support for a more recent version).

Integration platforms

  • Apache Kafka Client 2.5 (replaced by support for a more recent version).
  • IBM Universal Messaging Client 10.11 (replaced by support for a more recent version).

Web browsers

  • Microsoft Internet Explorer 11 (no replacement).

Distributed memory store

  • Software AG BigMemory Max 4.3.10 (replaced by support for a more recent version).
  • Software AG Terracotta Store 10.11 (replaced by support for a more recent version).

Miscellaneous

  • PySys 2.0 (replaced by support for a more recent version).
  • Azul Zulu Java 11.48 (replaced by support for a more recent version).
  • Software AG Cumulocity IoT 10.11 (replaced by support for a more recent version).
  • Software AG Designer 10.11 (replaced by support for a more recent version).

Note:

This topic erroneously stated that support for Python 3.9 was removed in 10.15.0 and the Supported Platforms document for version 10.15 said that Python 3.10 is supported. This has been corrected in both this topic and the Supported Platforms document. Python 3.10 is not supported in Apama 10.15.

Sending email requests from Apama to Cumulocity requires the user to have the "ROLE_EMAIL_CREATE" permission

As of Cumulocity IoT release 10.14, sending email requests from Apama to Cumulocity IoT requires the user to have the ROLE_EMAIL_CREATE permission. This means that the user name that is used in Apama Plugin for Eclipse projects and any Apama applications connecting externally to the Cumulocity IoT platform must have this permission.

Support for the old APIs for writing EPL plug-ins in C and C++ has now been removed

Support for the old APIs for writing EPL plug-ins in C and C++ has now been removed. See also the announcement in New API for writing EPL plug-ins in C++ in 10.0.

Support lifecycles for Cumulocity connectivity components aligned with Cumulocity IoT suppot lifecycle

With Apama 10.15.0, the support lifetimes for the components for connectivity to Cumulocity IoT, and the Apama images for Docker with that connectivity, have been aligned with the Cumulocity IoT support schedule. For more details, refer to the Release Availability document for version 10.15. This is available at: Release availability.

The engine_deploy tool can infinitely recurse when creating the output directory.

When a project directory contained another project as a subdirectory, the engine_deploy tool would create recursive directory paths until the path became too long. The tool will now detect a project as a subdirectory of another project and report an error.

The SMSResponse and SMSResourceReference events in the com.apama.cumulocity package are now deprecated

The SMSResponse and SMSResourceReference events in the com.apama.cumulocity package are now deprecated. They are not used by Cumolocity IoT and will therefore be removed in a future release.

Upgrade vulnerable Kafka client third-party from v2.5.0.

The third-party Kafka client library kafka-clients.jar was upgraded from 2.5.0 to 2.8.1 to resolve CVE-2021-38153. Connecting to Kafka brokers older than 2.8.1 is no longer supported.

Vulnerable third-party component jQuery used for Doxygen and Sphinx - upgrade to 3.5+.

The version of jQuery used for Doxygen and Sphinx has been updated from 1.7.2 to 3.6.0.

Vulnerable third-party component jQuery used for static-apama-http-resources needs update.

The version of jQuery used for the correlator web UI has been updated from 1.3.2 to 3.6.0. The version of DataTables used for the correlator web UI has been updated from 1.5.2 to 1.12.1.