About the Universal Messaging transport
Universal Messaging delivers data across different networks. It provides messaging functionality without the use of a web server or modifications to firewall policy. In Apama applications, you can configure and use the connectivity provided by Universal Messaging.
You can use the Apama integration to Universal Messaging as a simpler and more deeply integrated alternative to connecting to a Universal Messaging realm via the Java Message Service (JMS). This can be used both to connect different Apama components together using the internal Apama message format and for integrating with non-Apama systems over Universal Messaging. The Universal Messaging connectivity plug-in supports configurable mapping between Apama events and whatever formats the non-Apama components are using.
Only Universal Messaging channels can be used with Apama.
This transport provides a dynamic chain manager which creates chains automatically when EPL subscribes or sends to a correlator channel with the configured prefix, typically um:
. The selection of which dynamicChains
definition to use is based on the channelPattern
regular expression configured in the Universal Messaging transport of each dynamic chain. See Configuring the Universal Messaging connectivity plug-in for more details on the mapping between correlator channel names and the associated Universal Messaging channel or topic names.
The Universal Messaging connectivity plug-in does not support reliable messaging.
Overview of using Universal Messaging in Apama applications
In an Apama application, correlators can connect to Universal Messaging realms or clusters. A correlator connected to a Universal Messaging realm or cluster uses Universal Messaging as a message bus for sending Apama events between Apama components. Connecting a correlator to Universal Messaging is an alternative to specifying a connection between two correlators by executing the engine_connect
correlator tool.
Using Universal Messaging can simplify an Apama application configuration. Instead of specifying many point-to-point connections, you specify only the address (or addresses) of the Universal Messaging realm or cluster. Apama components connected to the same Universal Messaging realm can use Universal Messaging channels to send and receive events. (Universal Messaging channels are equivalent to JMS topics.) Connections to Universal Messaging are automatically made as needed, giving extra flexibility in how the application is designed.
When an Apama application uses Universal Messaging, a correlator automatically connects to the required Universal Messaging channels. There is no need to explicitly connect Universal Messaging channels to individual correlators. A correlator automatically receives events on Universal Messaging channels that monitors subscribe to and automatically sends events to Universal Messaging channels.
When using the connectivity plug-in for Universal Messaging, you can also talk to non-Apama applications which are connected to Universal Messaging and configure a chain definition to translate those message payloads into ones suitable for Apama.
Comparison of Apama channels and Universal Messaging channels
In an Apama application configured to use Universal Messaging, when an event is sent and a channel name is specified, the default behavior is that Apama determines whether there is a Universal Messaging channel with that name. If there is, then Apama uses the Universal Messaging message bus and the specified Universal Messaging channel to deliver the event to any subscribers. Subscribed contexts can be in either the originating correlator or other correlators connected to the Universal Messaging broker.
If a Universal Messaging channel with the specified name does not exist, then the default is that the channel is an Apama channel. An event sent on an Apama channel is delivered to any contexts that are subscribed to that channel.
Regardless of whether the channel is a Universal Messaging channel or an Apama channel, events are delivered directly to receivers that are connected directly to the correlator.
The following table compares the behavior of Apama channels and Universal Messaging channels.
Apama channels |
Universal Messaging channels |
---|---|
Configuration of multiple point-to-point connections. Each execution of Correlators and IAF adapters require explicitly set connections to communicate with each other. |
Specification of the same Universal Messaging realm address or addresses. Startup options for connected correlators specify the same Universal Messaging realm to connect to. Each IAF adapter configuration specifies the same address for connecting to Universal Messaging. Correlators and IAF adapters automatically connect to Universal Messaging to communicate with each other. |
Configuration changes are required when an Apama component is moved to a different host. |
No configuration change is needed when an Apama component is moved to a different host if both hosts are connected to the same Universal Messaging realm. |
Outside a correlator, channel subscriptions can be from only explicitly connected Apama components. |
Outside a correlator, channel subscriptions can be from any Apama component connected to the same Universal Messaging realm. |
Events sent on an Apama channel go directly to subscribers. |
Events sent on a Universal Messaging channel go to the Universal Messaging broker and then to subscribers. |
Connection configurations must be synchronized with application code. |
Connection to a Universal Messaging realm is independent of application code. |
Less efficient for sending the same event to many Apama components. |
More efficient for sending the same event to many Apama components. |
More efficient when sending an event to a context in the same correlator. The event stays inside the correlator. |
Less efficient when sending an event to a context in the same correlator. The event leaves the correlator, enters the Universal Messaging realm, and then returns to the correlator. |
Default channel, the empty string, is allowed. |
No default channel. |
Choosing when to use Universal Messaging channels and when to use Apama channels
Typically, you want to
- Use Universal Messaging channels to send events from one correlator to another correlator, from adapters to correlators, or from correlators to external receivers. You also might want to use Universal Messaging channels when your application needs the flexibility for a monitor or context to be moved to another correlator. With Universal Messaging, you can re-deploy monitors sending or subscribing to Universal Messaging channels among the correlators connected to the same Universal Messaging realm without having to change any of the configuration for the Universal Messaging connectivity.
- Use Apama channels to send events from one context to one or more contexts in the same correlator.
Consider the case of multiple correlators connected to the same Universal Messaging realm. Specification of a Universal Messaging channel lets events pass between a context sending events on the channel and a context subscribed to that channel, regardless of whether the two contexts are
- in the same correlator, or
- in different correlators on the same host, or
- in different correlators on different hosts.
The first time a channel is used, the default behavior is that Apama determines whether it is a Universal Messaging channel or an Apama channel, and the designation is cached. After the first use, the presence or not of the channel in the Universal Messaging broker is cached, so further use of the channel is not impacted.
Using Universal Messaging channels lets you take advantage of some Universal Messaging features:
- Using a Universal Messaging cluster can guard against failure of an individual Universal Messaging realm server. See the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging for more information on clusters.
- Universal Messaging provides access control lists and other security features such as client identity verification by means of certificates and on the wire encryption. Using these features, you can control the components that each component is allowed to send events to.
Using a Universal Messaging channel rather than an Apama channel can have a lower throughput and higher latency. If there is a Universal Messaging channel that contexts and plug-ins send to and that other contexts and plug-ins in the same correlator (or in different correlators) subscribe to, all events sent on that Universal Messaging channel are delivered by means of the Universal Messaging broker. In some cases, this might mean that events leave a correlator and are then returned to the same correlator. In this case, using an Apama channel is faster because events would be delivered directly to the contexts and plug-ins subscribed to that channel.
General steps for using Universal Messaging in Apama applications
Before you perform the steps required to use Universal Messaging in an Apama application, consider how your application uses channels. You should know which components need to communicate with each other, which events travel outside a correlator, and which events stay in a single correlator. Understand what channels you need and decide which channels should be Universal Messaging channels and which, if any, should be Apama channels.
For an Apama application to use Universal Messaging, the tasks you must accomplish are:
-
The Universal Messaging client libraries are included in the installation packages which contain the Digital Business Platform.
-
Plan and implement the configuration of the Universal Messaging cluster that Apama will use. See the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging and Setting up Universal Messaging for use by Apama.
-
Use Apama Plugin for Eclipse to add one of the Universal Messaging connectivity bundles to your Apama project. For detailed information, see Adding the Universal Messaging connectivity plug-in to a project.
InfoIn addition to using Apama Plugin for Eclipse to add bundles, you can also do this using theapama_project
command-line tool. See Creating and managing an Apama project from the command line for more information. -
Open the
UM.properties
file in your Apama project and specify the location of the Universal Messaging realm server(s) you wish to connect to. You can optionally edit theUM.yaml
file if you need to perform more advanced configuration tasks, such as enabling authentication or customizing the way Universal Messaging messages are mapped to Apama events. See Configuring the Universal Messaging connectivity plug-in for detailed information. -
In your EPL code, subscribe to receive events delivered on Universal Messaging channels. See Subscribing to channels.
As with all connectivity plug-ins, the EPL application is responsible for telling the system when it is ready to start receiving events with
onApplicationInitialized
. See also Sending and receiving events with connectivity plug-ins. -
In your EPL code, specify Universal Messaging channels when sending events. See Generating events with the send statement.
-
Monitor the Apama application’s use of Universal Messaging. See Monitoring Apama application use of Universal Messaging.
Using Universal Messaging channels instead of engine_connect
When you are using Universal Messaging channels in an Apama application, you connect multiple correlators by specifying the same Universal Messaging realm when you start each correlator. By using Universal Messaging channels, you probably do not need to use the engine_connect
tool at all.
While it is possible to configure an Apama application to use both Universal Messaging channels and the engine_connect
tool, this is not recommended.
Using Universal Messaging channels instead of configuring IAF adapter connections
<universal-messaging>
element in the configuration file to use an <apama>
element to talk directly to the correlator. See Apama correlator configuration.In an Apama application, you can use Universal Messaging as the communication mechanism between an IAF adapter and one or more correlators. If you do, then keep in mind the following:
- IAF adapters must send events on named channels. IAF adapters cannot use the default (empty string) channel.
- A service monitor that communicates with an IAF adapter should either be run on only one correlator, or be correctly designed to use multiple correlators. See Considerations for using Universal Messaging channels.
When an IAF adapter needs to communicate with only one correlator, which is often the case for a service monitor, an Apama channel might be a better choice than a Universal Messaging channel. However, even in this situation, it is possible and might be preferable to use a Universal Messaging channel. See Comparison of Apama channels and Universal Messaging channels.
See also Configuring IAF adapters to use Universal Messaging.
Considerations for using Universal Messaging channels
When using Universal Messaging channels in an Apama application, consider the following:
-
Injecting EPL affects only the correlator it is injected into. Be sure to inject into each correlator the event definition for each event that correlator processes. If a correlator sends an event on a channel or receives an event on a channel, the correlator must have a definition for that event.
-
The Universal Messaging message bus can be configured to throttle or otherwise limit events, in which case not all events sent to a channel will be processed.
-
Only events can be sent or received by means of Universal Messaging. You cannot use Universal Messaging for EPL injections, delete requests, engine send, receive, watch or inspection utilities, nor
engine_management -r
requests. -
If you want events to go to only a single correlator, it is up to you to design your deployment to accomplish that. If one or more contexts in a particular correlator are the only subscribers to a particular Universal Messaging channel, then only that correlator receives events sent on that channel. However, there is no automatic enforcement of this. In this situation, using the
engine_send
correlator tool might be a better choice than using a Universal Messaging channel. -
Universal Messaging channels can be configured for fixed capacity, and that is the default configuration used if the correlator creates a Universal Messaging channel. This does mean that if a context is sending to a channel while the same context is subscribed to that channel, then if the output queue, channel capacity and the context’s input queue are all full, the send can deadlock, as the send will hold up processing the next event, but not complete if all queues are full. Similarly, avoid a cycle of contexts and Universal Messaging channels creating a deadlock.
-
When the Universal Messaging channel names are not escaped, it is possible to create or use nested channels. In this case, the slash (/) and backslash (\) characters are treated as path separators on both Windows and Linux.
CAUTION:
Apama treats slash (/) and backslash (\) as different characters while Universal Messaging treats them as identical characters (Universal Messaging generally changes a slash to a backslash). You must choose to use one of these characters in your application and standardize on this. Use of both characters as path separators will result in undefined behavior. In some circumstances, an error message indicating that the user is already subscribed to a channel may be logged when both slashes and backslashes are used.
-
It is possible to use the Universal Messaging client libraries (available for Java, C#, C++ and other languages) to send events to or receive events from Apama correlators and adapters.
-
Universal Messaging is not used by the following:
- Apama client library connections.
- Correlator tools such as
engine_connect
,engine_send
andengine_receive
. - Adapter-to-correlator connections defined in the
<apama>
element of an adapter configuration file. While it is not recommended, it is possible to specify the name of a Universal Messaging channel when you use these Apama interfaces. Even though you specify the name of a Universal Messaging channel, Universal Messaging is not used. Events are delivered only to the Apama components that they are directly sent to. This can useful for diagnostics, but mixing connection types for a single channel is not recommended in production.
-
It is possible for third-party applications to use Universal Messaging channels to send events to and receive events from Apama components.
Third-party applications sending and receiving is supported subject to having a suitable chain definition to handle the third-party message format. This is recommended over the JMS integration.
-
The name of an Apama channel can contain any UTF-8 character. However, the name of a Universal Messaging channel is limited to the following character set:
0-9
a-z
A-Z
/ (slash, used as path separator when escaping is disabled; do not use both slash and backslash characters within the same application as this will result in undefined behavior - see also the above information)
\ (backslash, used as path separator when escaping is disabled)
# _ (underscore)
- (hyphen)
Consequently, some escaping is required if Universal Messaging needs to work with an Apama channel name that contains characters that are not supported in Universal Messaging channel names.
When writing EPL, you do not need to be concerned about escape characters in channel names. Apama takes care of this for you.
When interfacing directly with Universal Messaging, for example in a Universal Messaging client application for Java, you will need to consider escaping.
When creating Universal Messaging channels to be used by an Apama application, you might need to consider escaping. For example, you might already be using Apama channels whose names contain characters that are unsupported in Universal Messaging channel names. To use those same channels with Universal Messaging, you need to create the channels in Universal Messaging, and when you do, you must escape the unsupported characters.
The escape sequence is the hash (#) symbol, followed by the UTF-8 character number in hexadecimal (lowercase) which again is followed by the hash (#) symbol. For example, the following sequence would be used to escape a period in a channel name:
#2e#
Suppose that in Universal Messaging you want to create a channel whose name in Apama isMy.Channel
. In Universal Messaging, you need to create a channel with the following name:My#2e#Channel
-
Universal Messaging supports different protocols. Lower latency can be achieved by using the shm (Shared Memory) protocol if both the correlator and the broker are running on the same host. See the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging for information on how to configure the SHM driver.
Note that the SHM driver keeps a CPU core busy for each end of a connection as it uses spin loops rather than network I/O, which means that two CPU cores are used for each session. As a result, it is recommended to carefully consider and experiment with how many sessions should be used. The default number of sessions is 8, which will typically reduce throughput as it will use too much CPU for Universal Messaging connections.
Setting up Universal Messaging for use by Apama
For Apama to use the Universal Messaging message bus, there are some required Universal Messaging tasks. These steps will be familiar to experienced Universal Messaging users.
Plan and implement the configuration of the Universal Messaging cluster that Apama will use. The recommendation is to have at least three Universal Messaging realms in a cluster because this supports Universal Messaging quorum rules for ensuring that there is never more than one master in a cluster. However, if you can have only two Universal Messaging realms, you can use the isPrime
flag to correctly configure a two-realm cluster. For details about configuring a Universal Messaging cluster, see the topics in the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging that describe the following:
- Quorum
- Clusters with Sites, which describes an exception to the quorum rule.
To set up Universal Messaging for use by Apama, do the following for each Universal Messaging realm to be used by Apama:
-
The Universal Messaging client libraries are included in any installation package which contains the Digital Business Platform.
The Universal Messaging server can be installed on any machine (not necessarily on the same machine as Apama), but you must ensure that the Universal Messaging client libraries are present in the Apama installation, as these libraries are required by the Universal Messaging transport connectivity plug-in.
The Universal Messaging server should be obtained from IBM.
InfoIf you are using JMS to access Universal Messaging, installing the client libraries is not required. -
Start a Universal Messaging server.
-
Use Universal Messaging’s Enterprise Manager or Universal Messaging client APIs to set the access control lists of the Universal Messaging server to allow the user that the correlator is running on. See the Universal Messaging documentation for details.
-
By default, Apama automatically creates channels on the Universal Messaging server (that is, the configuration option
missingChannelMode
is set tocreate
; see Configuring the connection to Universal Messaging (dynamicChainManagers)).In production, it is usually better to change
missingChannelMode
toerror
and then add the channels that Apama will use explicitly using Universal Messaging’s Enterprise Manager (or the client APIs). In this case, configure the following Universal Messaging channel attributes. Together, these attributes provide behavior similar to that provided by using the Apama correlator toolengine_connect
.- Set Channel Capacity to
20000
or some suitable number, at least 2000 events. A number higher than 20,000 would allow larger bursts of events to be processed before applying flow control but would not affect overall throughput. - Select Use JMS engine. See also the information on engine differences in the Universal Messaging documentation.
- Set Honour Capacity when channel is full to
true
. These channel attributes provide automatic flow control. If a receiver is slow, then event publishers block until the receivers have consumed events.
Other channel attributes are allowed. However, it is possible to set Universal Messaging channel attributes in a way that might prevent all events from being delivered to all intended receivers, which includes correlators. For example, Universal Messaging can be configured to conflate or throttle the number of events going through a channel, which might cause some events to not be delivered. Remember that delivery of events is subject to the configuration of the Universal Messaging channel. Consult the Universal Messaging documentation for more details before you set channel attributes that are different from the recommended attributes.
- Set Channel Capacity to
Configuring the Universal Messaging connectivity plug-in
When you have added the Universal Messaging connectivity plug-in to your Apama project in Apama Plugin for Eclipse (see also General steps for using Universal Messaging in Apama applications), you can edit the YAML configuration file that comes with the plug-in.
The YAML file is configured to do the following:
- Load the Universal Messaging transport from the
UMConnectivity
library. - Under
dynamicChainManagers
, configure one Universal Messaging chain manager to connect to different Universal Messaging realms. Keep in mind that you can have only one chain manager here. - Configure one or more
dynamicChains
to handle transforming messages from Universal Messaging into the correlator.
Detailed information for this and much more is given in the topics below.
You can have different dynamicChains
processing messages on different channels in different formats.
Connection-related configuration is specified in the managerConfig
stanza on the dynamicChainManagers
instance, including the rnames
connection string for Universal Messaging.
Per-channel configuration of how to parse received messages is configured via the individual dynamicChains
.
Selection of which chain a manager is to use for a given channel name is done via a channelPattern
stanza on the UMTransport
for each chain.
By default, the UM
chain manager listens for subscribe and send-to requests in your EPL. It subscribes to channels with a specific prefix in Apama (by default, this is um:
) and connect that to the corresponding Universal Messaging channel without the prefix. Therefore, EPL subscribing or sending to um:channelName
will subscribe or send to the Universal Messaging channel channelName
. The prefix and whether it is included on Universal Messaging can be configured via the managerConfig
.
For more information on YAML configuration files, see Using connectivity plug-ins and Configuring the correlator.
Loading the Universal Messaging transport
The Universal Messaging transport is loaded with the following connectivityPlugins
stanza:
connectivityPlugins:
UMTransport:
libraryName: UMConnectivity
class: UMTransport
Configuring the connection to Universal Messaging (dynamicChainManagers)
The Universal Messaging dynamic chain manager monitors channels that are subscribed to or sent-to within the correlator and, as per its configuration, connects these to Universal Messaging channels as needed. This is where you specify how to connect to Universal Messaging. When the dynamic chain manager identifies a channel that should be connected to a Universal Messaging channel, it will create a chain using one of the templates in the dynamicChains
section of the YAML configuration (see also Configuring message transformations (dynamicChains)).
One UM
dynamic chain manager is created under dynamicChainManagers
which specifies the Universal Messaging realm or cluster of realms you are connecting to. The managerConfig
stanza contains all of the configuration for connecting to that Universal Messaging realm.
dynamicChainManagers:
UM:
transport: UMTransport
managerConfig:
rnames: nsp://127.0.0.1:9000
session:
poolSize: 8
certificateAuthorityFile: /mypath/my_UM_certificate
authentication:
username: me
certificateFile: /mypath/my_client_certificate
certificatePassword: mycertificatepassword
channel:
prefix: "um:"
missingChannelMode: create
escapeNamesOnUM: true
includePrefixOnUM: false
createChannelPermissions:
user@host:
- Manage ACL
- Full
"*@*":
- Publish
- Subscribe
- Last EID
Everyone:
- Purge
The following table describes the options that can be used in the managerConfig
section:
Option |
Description |
---|---|
|
Required. The Universal Messaging connection string of realm names to connect to. You can specify one or more Universal Messaging realm names ( Commas indicate that you want the correlator to try to connect to the Universal Messaging realms in the order in which you specify them here. For example, if you have a preferred local server you could specify its associated realm name first and then use commas as separators between specifications of other realm names, which would be connected to if the local server is down. Semicolons indicate that the correlator can try to connect to the specified Universal Messaging realms in any order. For example, use semicolons when you have a cluster of equally powered machines in the same location and you want to load balance a large number of clients. You can specify multiple Universal Messaging realms only when they are connected in a single Universal Messaging cluster. That is, all realm names you specify must belong to the same Universal Messaging cluster. Since channels are shared across a cluster, connecting to more than one Universal Messaging realm lets you take advantage of Universal Messaging’s failover capability. For additional information on communication protocols and realm names, and on clusters, see the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging. Default value: none. |
|
The number of sessions (connections) to create to the Universal Messaging realm. Either a number of sessions or a string of the form |
|
The path to a CA certificate for the Universal Messaging realm. This is used by the Apama to confirm the identity of the Universal Messaging realm server. Default value: none. |
|
The username that is to be used to connect to the Universal Messaging realm. The default is the user which the correlator is running as. Default value: none. |
|
The path to a certificate that is used by the Universal Messaging realm server to authenticate this client. Default value: none. |
|
The password for the certificate. Default value: none. |
|
A prefix for the channel. Only channels with this prefix will be considered as Universal Messaging channels. If the prefix ends with a colon (:), it needs to be enclosed in quotation marks (see also Using YAML configuration files). Default value: |
|
If set to |
|
If set to |
|
Defines the behavior when subscribing to channels which do not exist. You can define one of the following options: -
Default value: |
|
Defines the ACL (access control list) permissions for automatically created channels in the following format: ``` client: - permission1 - permission2 -… The
For additional information on subject, group, wildcards and permission ACLs, see the Universal Messaging documentation. Default:
Note: Apama clients require Setting channel permissions correctly is important to protect the security of your application, and also to protect any personal data included in the messages. For more information, see Protecting personal data in Apama applications. |
Configuring message transformations (dynamicChains)
dynamicChains
contains templates of chains. The Universal Messaging manager will pick a chain template that ends with the UMTransport
plug-in, and use that configuration. The manager also provides configuration from the channel which the chain can use with @{varname}
substitutions (see Using dynamic replacement variables). The manager uses the channelPattern
property of the UMTransport
configuration to decide which chain template should be used for a given channel.
With either the apama.eventMap
or apama.eventString
host plug-ins, we recommend use of the suppressLoopback
configuration property to avoid messages which are sent to that channel being delivered internally as well as being sent to and received from Universal Messaging. We also recommend setting the description
and remoteAddress
properties in order to improve logging and debugging. See Host plug-ins and configuration for more information.
The following chain sends and receives events with the members of the event being set in the nEventProperties
of the Universal Messaging events and an empty payload.
dynamicChains:
UMChain:
- apama.eventMap:
suppressLoopback: true
description: "@{um.rnames}"
remoteAddress: "@{um.rnames}"
- UMTransport:
setTypeFromTag: true
channelPattern: ".*"
The following options can be used with UMTransport
:
Option |
Description |
---|---|
|
If set to For performance-critical applications where the event type is known or can be set in the chain, we recommend setting this option to Default value: |
|
Required. A regular expression that is used to select which chain is used for which channel. Only one chain definition may match any channel, except for the “fallback” definition with the channel pattern Default value: none. |
Using dynamic replacement variables
The UMTransport
provides the following dynamic replacement variables which can be used with the @{varname}
replacement syntax:
Variable | Description |
---|---|
um.channelName |
The name of the Universal Messaging channel. |
um.rnames |
The address of the Universal Messaging realm. |
um.channelCapacity |
The maximum capacity of the Universal Messaging channel. |
For example, for using all channels as mapping directly to nEventProperties
:
UMMessagePropertiesChain:
- apama.eventMap:
suppressLoopback: true
description: "@{um.rnames}"
remoteAddress: "@{um.rnames}"
- UMTransport:
channelPattern: ".*"
Subscribing to automatically created Universal Messaging channels
By default, the configuration option missingChannelMode
is set to create
so that a Universal Messaging channel can be automatically created if it does not already exist when an Apama application needs to use it. See also Configuring the connection to Universal Messaging (dynamicChainManagers).
When a Universal Messaging channel is automatically created, which is helpful for getting started, it has the attributes described in Setting up Universal Messaging for use by Apama. If you want a Universal Messaging channel to have any other attributes, then you must create the channel in Universal Messaging before any Apama component sends to or subscribes to the channel.
In production, it is usually better to change missingChannelMode
to error
and to configure the channels explicitly as described in Setting up Universal Messaging for use by Apama.
You can specify the ACL (access control list) permissions for the channel being created in a YAML configuration file using the createChannelPermissions
option. See Configuring the connection to Universal Messaging (dynamicChainManagers).
Channel lookup caching
After Apama looks up a channel name to determine whether it is a Universal Messaging channel, Apama caches the result and does not look it up again. Consequently, the following situation is possible:
- You use Universal Messaging interfaces to create channels.
- You start a correlator with
missingChannelMode
set toignore
. - Apama looks up, for example,
channelA
and determines that it is not a Universal Messaging channel. - You use Universal Messaging interfaces to create, for example,
channelA
.
For Apama to recognize channelA
as a Universal Messaging channel, you must either restart the correlator or issue a flushChannelCache
request using the engine_management
tool (see also Shutting down and managing components):
engine_management -r flushChannelCache
This operation may take a long time since it verifies the existence of every channel subscribed to in the correlator on Universal Messaging. Therefore, we recommend that you ensure all your channels have been created on Universal Messaging before starting your Apama application.
Supported payloads
The Universal Messaging transport supports different types of Apama message payload:
- Binary payloads (Java
byte[]
or C++buffer_t
). Apama messages with binary payloads are mapped to the data payload of a Universal Messaging message. Optionally, Universal Messaging message properties/headers (nEventProperties
) may be mapped using metadata values in string form with the prefixum.properties
. You can also map all of the Universal Messaging message properties to an EPL dictionary with a Mapper codec rule that movesmetadata.um.properties
in its entirety; see also The Mapper codec connectivity plug-in. If you wish to put a string into the Universal Messaging message data payload, use the String codec to convert it into binary form (as UTF-8); see also The String codec connectivity plug-in. - Map payloads (Java
Map
or C++map_t
). Apama messages with a map payload are mapped to the Universal Messaging message properties/headers (nEventProperties
), and the Universal Messaging message data payload is empty.
If you are sending and receiving using the eventMap
host plug-in (see also Translating EPL events using the apama.eventMap host plug-in), you probably want to make use of the Mapper and Classifier codecs (see Codec Connectivity Plug-ins), unless the Apama event type name is stored in the “tag” of the Universal Messaging messages. Typically, the Apama event format does not match exactly to the format you want in the nEventProperties
, and the Mapper codec allows you to fix that.
When setting nEventProperties
either from the map payload or via um.properties
metadata values, the following EPL types are unsupported and sending events to Universal Messaging will therefore fail:
dictionary<>
types with keys which are not Apama primitives (that is, anything exceptinteger
,boolean
,decimal
,float
,string
). For example,dictionary<Sequence<some type>, String>
is not supported, butdictionary<decimal, <sometype>>
is supported.- Apama
decimal
type anddictionary<>
keys are stringified when sending the events. That also means that sequence<decimal>
is sent as a sequence of strings. - Sequences within sequences. For example,
sequence<sequence<any type>>
. Note thatsequence <Apama Event>
orsequence <dictionary<some primitive, any type>>
are supported.
The binary payload may represent a string. If this is a case, then the binary payload must be converted to a string payload before further processing can happen as a string. To do this, use the String codec. This converts binary payloads to string payloads for hostward messages and string payloads to binary payloads for transportward messages. The String codec should be the last codec in the chain. See The String codec connectivity plug-in for detailed information.
You can also use other codecs such as the JSON codec (see Codec Connectivity Plug-ins for more information). For example:
dynamicChains:
UMJsonChain:
- apama.eventMap:
suppressLoopback: true
description: "@{um.rnames}"
remoteAddress: "@{um.rnames}"
- jsonCodec
- stringCodec
- UMTransport:
channelPattern: ".*"
EPL and Universal Messaging channels
In an Apama application that is configured to use Universal Messaging, you write EPL code to subscribe to channels and to send events to channels as you usually do. The only difference is that you cannot specify the default channel (the empty string) when you want to use a Universal Messaging channel. You must specify a Universal Messaging channel name to use Universal Messaging.
As with all connectivity plug-ins, the EPL application is responsible for telling the system when it is ready to start receiving events with onApplicationInitialized
. See also Sending and receiving events with connectivity plug-ins.
A monitor that subscribes to a Universal Messaging channel causes its containing context to receive events delivered to that channel. There is nothing special you need to add to your EPL code.
Using Universal Messaging channels makes it easier to scale an application across multiple correlators because Universal Messaging channels can automatically connect parts of the application as required. If you use the EPL integer.incrementCounter("UM")
method, remember that the return value is unique for only a single correlator. If a globally unique number is required, you can concatenate the result of integer.incrementCounter("UM")
with the correlator’s physical ID. Obtain the physical ID from Apama’s Management interface with a call to the getComponentPhysicalId()
method. For further information, see Using the Management interface.
Using Universal Messaging connectivity from EPL
In EPL, in order to receive events from a Universal Messaging channel, you just need to subscribe to a channel with the appropriate prefix:
on all EventTypeOnUM() {... }
monitor.subscribe("um:UMChannelName");
This creates a chain with a channel pattern matching um:UMChannelName
and subscribe to UMChannelName
on the connected realm. Events from that channel are delivered to the context after being parsed by the chain.
To send to a Universal Messaging channel, you just need to use the send...to
statement to deliver an event to that channel name:
send EventTypeOnUM() to "um:UMChannelName";
This will use the same chain definition as above to deliver the mapped event to the Universal Messaging channel UMChannelName
.
The samples/connectivity_plugin/application/genericsendreceive
directory of your Apama installation includes a simple sample which provides an easy way to get started with sending and receiving messages to or from any connectivity plug-in. For more information, see the README.txt
file in the above directory and Sending and receiving events with connectivity plug-ins.
Monitoring Apama application use of Universal Messaging
You can use the Universal Messaging Enterprise Manager or Universal Messaging APIs to find out about the following:
- Which correlators are subscribed to which Universal Messaging channels.
- The number of events flowing through a Universal Messaging channel.
- The contents of the events going through a Universal Messaging channel.
See the Universal Messaging documentation at https://docs.webmethods.io/on-premises/webmethods-universal-messaging for more information on the Enterprise Manager.
To monitor and manage Apama components, you must use Apama tools and APIs.