Defining monitors

About monitor contents

A file that defines a monitor has the following form:

1. An optional package declaration
2. Followed by
   1. Zero or more using declarations
   2. Zero or more custom aggregate function definitions
   3. Zero or more event type definitions
3. One or more monitor definitions

When you define monitors that are closely related, it is your choice whether to define them in the same file or different files.

A monitor must have information about any event types it processes. Hence, the correlator must receive and parse all of the event types used by the monitor before it is able to correctly parse the monitor itself.

A monitor can contain one or more global variables. A global variable declaration appears inside a monitor but outside any actions. The variable is global within the scope of the monitor.

A monitor can also contain a number of actions. Actions are similar to procedures. Finding an event, or pattern of events, of interest can trigger an action. You can also trigger an action by invoking it from inside another action.

Any construct that you declare inside a monitor is available only from within that monitor. In other words, its use is restricted to the scope of the monitor.

Below is a minimal monitor:

monitor EmptyMonitor {
   action onload() {
   }
}

The monitor above does not do anything; it does not register interest in any event or event pattern, it does not have variables, and it does not do anything in its single action statement. However, it does show the minimum structure of a monitor:

• It specifies the monitor keyword followed by the name of the monitor. In the example, the name of the monitor is EmptyMonitor. The name of the monitor and the name of the file that contains the monitor do not need to be the same. A single file can contain multiple monitors.

• It declares the onload() action. When you inject a monitor into the correlator, the correlator executes the monitor’s onload() action. Every monitor must contain an onload() action. The onload() action is similar to the main() function in C/C++.

  If you define two or more monitors in the same file, the correlator executes the onload() actions of the monitors in the order in which you define the monitors. If there is an onload() action whose execution is dependent on the results of the execution of the onload() action of another monitor, but sure you define that other monitor earlier in the same file. If you define that other monitor in a separate file, be sure you inject that file first. Tip: it is better to avoid these dependencies as much as possible by using initialization events. See Using events to control processing.

EPL provides a number of actions, such as onload(), onunload(), and ondie(). You can define additional actions, and assign a name of your choice that is not an EPL keyword. See also Keywords.

Do not create EPL structures in the com.apama namespace. This namespace is reserved for future Apama features. If you do inadvertently create an EPL structure in the com.apama namespace, the correlator might not flag it as an error in this release, but it might flag it as an error in a future release.

Loading monitors into the correlator

During development, you use Apama Plugin for Eclipse to load your project, including monitors, into the correlator. Apama Plugin for Eclipse ensures that files are loaded in the required order.

At any time, you can use the engine_inject correlator tool to load EPL files into the correlator. See Injecting code into a correlator.

In a deployment environment, you can load monitors into the correlator in any of the following ways:

• Use the engine_inject tool.
• Write a program in C++, Java, or.NET and use the corresponding Apama client API.

If you try to inject a monitor whose name is the same as a monitor that was already injected, the correlator rejects the monitor. You can inject two monitors with the same name into the correlator only if they exist in different packages. To specify the package for a monitor or event type, add a package statement as the first line in the EPL file that contains the monitor/event definition. For example:

package com.mycompany.mypackage;
monitor Foo {
    ...
}

Terminating monitors

A monitor instance terminates when one of the following events occurs:

• The monitor instance executes a die statement in one of its actions.
• A runtime error condition is raised from an onload() or spawned action or within a stream query.
• A runtime error condition is raised from a listener or a stream listener in a monitor with an ondie() action defined.
• The monitor is terminated externally (for example, with the engine_delete utility). When the correlator deletes a monitor it terminates all instances of that monitor.
• The monitor instance has executed all its code and there are no active event or stream listeners. This will occur rapidly if the monitor’s onload() action does not create any listeners. See also Beware of accidental stream leaks.

When a monitor instance terminates, the correlator invokes the monitor’s ondie() action, if it is defined. You cannot spawn in an ondie() action.

Unloading monitors from the correlator

The correlator unloads a monitor in the following situations:

• All of the monitor’s instances have terminated.
• An external request kills the monitor. This kills any instances of the monitor.

If the monitor defines an onunload() action, the correlator executes it just before it unloads the monitor. You cannot spawn in an onunload() action.

If an owning monitor has an internal event type, it is possible for another dependent monitor to hold an instance of that internal event type in a variable of the any type (see the description of the any type in the API reference for EPL (ApamaDoc)) if the owning monitor has sent or routed an instance of the monitor-internal event. In this case, a monitor is not completely unloaded, even if all of its monitors have terminated, because another monitor still depends on one of the monitor-internal types. The monitor name will stay in the correlator, but there will be no monitor instances running. The onunload() action, if defined, still executes when the last monitor instance is terminated. The monitor is not automatically deleted in this case. The monitor name needs to be explicitly deleted with the engine_delete tool or by using the client API, which can only be done if the monitors that are dependent on the internal type are either no longer dependent or have been deleted themselves. See also Deleting code from a correlator.

Example of a simple monitor

The empty monitor discussed in About monitor contents does not do anything. To write a useful monitor, add the following:

• An event type definition
• A global variable declaration
• An event expression that indicates the pattern to monitor for
• An action that operates on an event that matches the specified pattern

For example, the EPL below

• Defines the StockTick event type, which is the event type that the monitor is interested in.
• Defines the newTick global variable, which is accessible by all actions within this monitor. The newTick variable can hold a StockTick event.
• Registers an interest in all StockTick events.
• Invokes the processTick() action when it finds a StockTick event. The processTick() action uses the log statement to output the name and price of all StockTick events received by the correlator.

Lines starting with // are comments. EPL also supports the standard C++/Java /*... */ multi-line comment syntax.

// Definition of the event type that the correlator will receive.
// These events represent stock ticks from a market data feed.
event StockTick {
   string name;
   float price;
}

// A simple monitor follows.
monitor SimpleShareSearch {
   // The following is a global variable for storing the latest
   // StockTick event.
   StockTick newTick;
      // The correlator executes the onload() action when you inject the
      // monitor.
      action onload() {
         on all StockTick(*,*):newTick processTick();
      }
      // The processTick() action logs the received StockTick event.
      action processTick() {
         log "StockTick event received" +
            " name = " + newTick.name +
            " Price = " + newTick.price.toString() at INFO;
      }
}

About the variable in the example

The single global variable is of the event type StockTick. A variable can be of any primitive type (boolean, decimal, float, integer, string), or any reference type (action, context, dictionary, event, listener, location, sequence or stream).

About the onload() action

In this example, the onload() action contains only one line of code:

on all StockTick(*,*):newTick processTick();

This line specifies the following:

• on all StockTick(*,*) indicates the event to look for.

  The on statement begins the definition of an event listener. It means, “when the following event (or a pattern of events) is received …”. This event listener is looking for all StockTick events. The asterisks indicate that the values of the StockTick event fields do not matter.

• :newTick processTick(); indicates what to do when a StockTick event is found.

  If the event listener finds a StockTick event, the coassignment (:) operator indicates that you want to copy the found event into the newTick global variable. The onload() action then invokes the processTick() action.

About event listeners

The on statement must be followed by an event expression. An event expression specifies the pattern you want to match. It can specify multiple events, but this simple example specifies a single event in its event expression. For details, see About event expressions and event templates.

The all keyword extends the on statement to listen for all events that match the specified pattern. Without the all keyword, the event listener would listen for only the first matching event. In this example, without the all keyword, the event listener would terminate after it finds one StockTick event.

In the sample code, the event expression is StockTick(*,*). Each event expression specifies one or more event templates. Each event template specifies one event that you want to listen for. The StockTick(*,*) event expression contains one event template.

The first part of an event template defines the type of event the event listener is looking for (in this case StockTick). The section in parentheses specifies filtering criteria for contents of events of the desired type. In this example, the event template sets both fields to wildcards (*). This declares an event listener that is interested in all StockTick events, regardless of content.

When an event listener finds a matching event, the listener can use the as operator to place the event into an implicitly declared variable only available in the scope of the listener processing block or the : assignment operator to place that event in a global or local variable. For example:

on StockTick(*,*) as newTick {
     processTick(newTick);
}

This copies a StockTick event into the newTick variable which is only in scope of the processing block. This is known as implicit coassignment.

Or:

on all StockTick(*,*):newTick processTick();

This copies a StockTick event into the newTick global variable. This is known as a variable coassignment.

Finally, the on statement invokes the processTick() action. For all received StockTick events, regardless of content, the sample monitor copies the matching event into the newTick global variable, and then invokes the processTick() action. For details, see Using global variables.

About the processTick() action

The processTick() action executes the log statement to output some data on the registered logging device, which by default is standard output. This log statement is used to report some of the fields from the received event. For details, see Logging and printing.

Accessing fields in events

EPL uses the . operator to access the fields of an event. You can see that the processTick() action uses the . operator to retrieve both the name (newTick.name) and price (newTick.price) fields of each event.

The log statement requires strings as fields, so the processTick() action specifies the built-in .toString() operation on the non-string value:

newTick.price.toString()

Spawning monitor instances

It is frequently necessary to enable a single monitor to concurrently listen for multiple kinds of the same event type. For example, you might want one monitor to listen for and process stock ticks that each have a different stock name. You accomplish this is by spawning monitor instances as described in the topics below.

How spawning works

In a monitor, you spawn a monitor instance by specifying the spawn keyword followed by an action. When the correlator spawns a monitor instance, it does the following:

1. Creates a new instance of the monitor that is spawning.
2. Copies the following, if there are any, to the new monitor instance:
   • Current values of the spawning monitor instance’s global variables
   • Any arguments declared in the action that is specified in the spawn statement
   • Anything referred to indirectly by means of the copied variables and arguments
3. Executes the named action with the specified arguments in the new monitor instance.

The new monitor instance does not contain any active event listeners, stream listeners, streams or stream queries that were in the spawning monitor instance. For example, data held in local variables that are bound to a listener are not copied from the spawning monitor instance to the new monitor instance. The figure below illustrates this process:

The figure shows a monitor that spawns when it receives a NewStock event. Initially, the monitor has one active event listener. When the event listener finds the first NewStock event, the monitor

1. Copies the name IBM to the chosenStock variable.
2. Spawns a monitor instance.

The spawned monitor instance duplicates the initial monitor instance’s state. In this example, this means that the value of the chosenStock variable in the spawned monitor instance is IBM. When the initial monitor instance receives another NewStock event (the value of the name field is ATT), it again copies the stock’s name to the chosenStock variable and spawns. The same occurs for the XRX event, resulting in three spawned monitor instances.

Each new monitor instance starts with no active event listeners. It then creates a new event listener for StockTick events of the chosen stock (see the sample code in the next topic). The initial monitor instance’s event listener for NewTick events remains active after spawning. However, because the action to create a new StockTick event listener is executed only in the spawned monitor instances, the initial monitor instance continues to listen for only NewTick events.

Sample code for spawning

EPL that implements the example described in How spawning works is as follows:

// The following event type defines a stock that a user is interested
// in. The event type includes the name of the stock (name) and the
// user's personal name (owner).
//
event NewStock {
   string name;
   string owner;
}

event StockTick {
   string name;
   float price;
}

monitor SimpleShareSearch {
   NewStock chosenStock;
   integer numberTicks;
   StockTick newTick;

   // Listen for all NewStock events. When a NewStock event is found
   // assign it to the chosenStock variable and spawn with a call to
   // the matchTicks() action. This clones the state of the monitor
   // and launches a monitor instance that executes matchTicks().
   action onload() {
      numberTicks := 0;
      on all NewStock (*, *):chosenStock spawn matchTicks();
   }

   // In the spawned monitor instance, listen for only those StockTick
   // events whose name matches the name in the chosenStock variable.
   action matchTicks() {
      on all StockTick(chosenStock.name,*):newTick processTick();
   }

   action processTick() {
      numberTicks := numberTicks + 1;
      log "A StockTick regarding the stock "
         + newTick.name + "has been received "
         + numberTicks.toString() + " times. This is relevant for "
         + " Trader name: " + chosenStock.owner
         + " and the price is " + newTick.price.toString()
         + "." at INFO;
   }
}

This example defines a new event type named NewStock. Traders dispatch this event when they want to look for a specific kind of stock event. The code example spawns a monitor instance when the monitor finds a NewStock event. For example, if three newStock events are received by the initial monitor instance, there will be three spawned monitor instances. Other than spawning, the difference between this code sample and the sample in Example of a simple monitor is that this one specifies an owner in each NewStock event and the monitor’s state now includes a counter.

In this example, after spawning, all processing is within a spawned monitor instance. Processing begins with execution of the matchTicks action. This action starts by defining an event listener for StockTick events whose name field matches the name field in the spawned monitor instance’s chosenStock variable. When there are multiple, spawned monitor instances, each spawned monitor instance listens for only the StockTick events that match their chosenStock name.

The numberTicks counter variable and the chosenStock event variable, which contains the stock name and the owner’s name, are available in the cloned state of the spawned monitor instance. This lets the processTick() action in each spawned monitor instance

• Customize output to include the originating trader’s name
• Maintain a counter of how many StockTicks for a particular stock have been detected for a trader

The really important aspect that distinguishes spawning is that the entire variable space is cloned at the moment of spawning. In the example, every spawned monitor instance has a copy of the chosenStock variable that contains the NewStock event that triggered spawning. Also, every spawned monitor instance has a copy of the numberTicks variable, which is always set to 0 when the initial monitor instance spawns. This ensures that each spawned monitor instance can maintain an accurate count of how many matching StockTick events have been found.

The initial monitor instance listens for NewStock events. Remember that spawning does not clone active listeners, so the spawned monitor instances do not have listeners that watch for NewStock events. Each spawned monitor instance listens for only those StockTick events that contain name fields that match that spawned monitor instance’s value for the chosenStock variable.

Typically, spawning is not an expensive operation. However, its overhead does increase as the size of the monitor being spawned increases. When writing an EPL application avoid repeated spawning of monitors that contain a large number of variables.

Spawned monitor instances contain copies of all global state from the spawning monitor instance. It does not matter whether the spawned monitor instance is going to use that state or not. To avoid wasting memory, a typical practice is to hold state in events that are referred to by local variables, which are not copied during spawning. This ensures that you do not have a lot of state information in global variables when the monitor instance spawns. Alternatively, you can insert code so that the new monitor instance clears unneeded state immediately after it starts running.

For information about spawning to actions that are members of events, see Spawning.

Terminating monitor instances

The example discussed in Sample code for spawning spawns a monitor instance for each newStock event that the initial monitor instance receives. This is not always desirable. For example, if two identical newStock events are received, two identical monitor instances are spawned. To prevent this, you can use the die statement to delete a monitor instance if a more recent one (with the same spawning properties) has been created. For example:

action onload() {
   on all NewStock(*, *):chosenStock spawn matchTicks();
}
action matchTicks() {
   on NewStock (chosenStock.name, chosenStock.owner) die;
   //...
}

In this fragment, the monitor spawns when it receives a NewStock event. In the spawned monitor instance, the initial on statement activates an event listener for a NewStock event that is identical to the one that caused the spawning. In other words, the spawned monitor instance is listening for a NewStock event where the fields are the same as that held by the chosenStock variable. If such an event arrives, the monitor instance terminates. This structure ensures that only one monitor instance for each stock name and owner exists at any one time. The same NewStock event kills the existing monitor instance and causes spawning of a new monitor instance. That is, the same event triggers the concurrent event listeners of the initial monitor and the spawned monitor instance.

In this solution, when a NewStock event kills an existing monitor instance and spawns a new monitor instance, the value of the numberTicks variable in the new instance is zero. Often, this kind of behavior is required. You want to ignore the state of the old monitor instance and start afresh.

Note that the event that triggers the initial monitor instance’s event listener and causes the spawning of a monitor instance does not get processed by the spawned monitor instance’s new event listener. An event is available to only those event listeners that are active when the correlator receives the event.

You can also use the die statement to kill a monitor instance at will. For example, consider the following fragments:

event StopStock {
   string name;
   string owner;
}

action onload() {
   on all newStock(*, *):chosenStock spawn matchTicks();
}

action matchTicks() {
   on StopStock (chosenStock.name, chosenStock.owner) die;
   //...
}

Traders would send StopStock events when they are no longer interested in a particular stock. Receiving a matching StopStock event kills the monitor instance that is listening for that stock. You can use this technique to explicitly kill any monitor instance.

About executing ondie() actions

A monitor instance can terminate for any of the following reasons:

• It executes all its code and has no active listeners or streaming elements.
• It executes a die statement in one of its actions.
• The engine_delete utility or an Apama client API removes the monitor from the correlator.
• A runtime error is detected in the monitor’s code, which causes that instance of the monitor to die.

Runtime errors occurring from within listeners or stream listeners - that is, not called from onload() or a spawn action or from an action within a stream query - will not terminate a monitor instance, unless an ondie() action is defined.

In all of these situations, if the monitor defines an ondie() action, the correlator invokes it. Like the onload() and onunload() actions, ondie() is a special action because the correlator invokes it automatically in certain situations.

Suppose that a monitor that defines the ondie() action spawns ten times, and each monitor instance dies. The correlator invokes ondie() eleven times: once for each spawned monitor instance, and once for the initial monitor instance. Then, just before the monitor’s EPL is unloaded from the correlator, the correlator invokes the onunload()action only once, and it does so in the context of the last remaining monitor instance.

The correlator executes each ondie() operation in the context of its monitor instance. Therefore, the ondie() operation can access the variables in the monitor instance being terminated.

You cannot spawn in an ondie() or an onunload() action.

There are two forms of the ondie action: one form can have no argument and the other form can have optional<com.apama.exceptions.Exception> and optional<string> arguments. If the monitor instance terminates due to an uncaught exception, this exception is passed as a first argument to the ondie action. The second argument of the ondie action is populated with the reason if monitor instance terminated without an exception. Constants for the reason string are defined on the monitor type. See the API reference for EPL (ApamaDoc) for more information.

Example:

action ondie(optional<com.apama.exceptions.Exception> exc, optional<string>
  reasonCode) {
    ifpresent exc {
      // do something
    }
    ifpresent reasonCode {
        if reasonCode = monitor.DIE or reasonCode = monitor.NO_LISTENERS
           or reasonCode = monitor.ENGINE_DELETE {
              //do something
        }
    }
}

Specifying parameters when spawning

When spawning a monitor instance, you can pass parameters to an action. For example:

monitor m {
   action onload() {
      spawn forward("a", "channelA");
      spawn forward("b", "channelB");
   }

   action forward(string arg, string channel) {
      on all Event(arg) as e {
         send e to channel;
      }
      on StopForwarding(arg) {
         die;
      }
   }
}

Communication among monitor instances

In EPL applications, everything in a monitor instance is private. There is no direct way for a monitor instance to invoke an action or access the state of another monitor instance. Instead, messages, in the form of events, are the mechanism for communication among monitor instances. All events are visible to all interested monitor instances.

Consequently, how you divide your application operations into monitors and what events the monitor instances use to communicate are crucial design decisions. An understanding of the order in which the correlator processes events for monitors helps you determine where and when to allocate events.

The topics below provide information for making these decisions.

You can use the MemoryStore to share state between monitors, see Using the MemoryStore.

Organizing behavior into monitors

Typically, an Apama application consists of several monitors each doing a specific task. For example, a simple algorithmic trading system might consist of the following monitors:

• A monitor that manages order processing by spawning a monitor instance for each order.
• One or more market data monitors. Each monitor listens for a different type of market data (such as tick data, market depth) required to process orders. Each of these monitors typically spawns a monitor instance for each stock you want to observe.

A more complex application might organize its orders into portfolios or split sets of orders into smaller orders for wave trading or some other purpose.

In an Apama application, each monitor can usually be categorized as a core processing monitor or a service monitor. A core processing monitor performs the tasks you want to accomplish. A service monitor provides data needed by the core processing monitors. Typically, the core processing monitors spawn multiple monitor instances. These monitor instances will consume data from the same service monitors. For example, all monitor instances that manage the individual orders for a given stock would obtain tick data from the same instance of a service monitor. The ordinality of the solution elements (for example, N order processors that require data from 1 tick data provider) often dictates how the solution code should be organized into separate monitors. See also About service monitors.

The ordinality of the solution elements often dictates how the solution code should be organized into separate monitors. For example, there is an N:1 relationship between the N order processor monitor instances that require market data for a given stock and the 1 market data service monitor instance that supplies it.

Event processing order for monitors

As mentioned earlier, contexts allow EPL applications to organize work into threads that the correlator can execute concurrently. When you start a correlator it has a main context. In a monitor, you can create additional contexts to enable the correlator to concurrently process events.

Each context, including the main context, has its own input queue, which receives

• Events sent specifically to that context from other contexts.
• Events sent to a channel that a monitor in the context is subscribed to. See Channels and input events.

Concurrently, in each context, the correlator

• Processes events in the order in which they arrive on the context’s input queue
• Completely processes one event before it moves on to process the next event

When the correlator processes an event within a given context, it is possible for that processing to route an event. A routed event goes to the front of that context’s input queue. The correlator processes the routed event before it processes the other events in that input queue.

If the processing of a routed event routes one or more additional events, those additional routed events go to the front of that context’s input queue. The correlator processes them before it processes any events that are already on that context’s input queue.

For example, suppose the correlator is processing the E1 event and events E2, E3, and E4 are on the input queue in that order.

While processing E1, suppose that events En1 and En2 are created in that order or sent to the context. Assuming that there is room on the input queue of that context, those events go to the end of the input queue of that context:

While still processing E1, suppose that events R1 and R2 are created in that order and routed. These events go to the front of the queue:

When the correlator finishes processing E1, it processes R1. While processing R1, suppose that two event listeners trigger and each event listener action routes an event. This puts event R3 and event R4 at the front of the context’s input queue. The input queue now looks like this:

It is important to note that R3 and R4 are on the input queue in front of R2. The correlator processes all routed events, and any events routed from those events, and so on, before it processes the next routed or non-routed event already on that queue.

Now suppose that the correlator is done processing R1 and it begins processing R3. This processing causes R5 to be routed to the front of that context’s input queue. The context’s queue now looks like the following:

See also Understanding time in the correlator.

Allocating events in monitors

Events are reference types. This means that, for example, a variable of event type Foo is not an instance of Foo. The variable is a reference to an instance of Foo.

You cannot initialize the fields of a monitor-level variable. You can, however, initialize a monitor-level instance of the event that the variable refers to. For example:

Foo a := Foo(1, 2.3);

This instantiates a Foo event and specifies that a refers to that event. Now suppose you declare the following:

Foo b := a;

This does not instantiate a new Foo event. It only initializes b as an alias for a.

When you declare an event at the monitor level, the correlator can automatically use default values for the event’s fields. You can, but you do not have to, initialize field values. This is because the correlator implicitly transforms a statement such as this:

Foo a;

into this:

Foo a := new Foo;

Before you use a locally declared event variable in an action, you must either assign it to an existing event of the same type, or you must specify the new operator to create a new event to assign to the variable. Note that each event field of an event created using new initially has the default value for that event field type.

The following code illustrates these points:

event Foo
{
   integer i,
   float x;
}

monitor Bar
   Foo a; //  Global (monitor-level) declaration.
          //  The correlator creates a Foo event with default
          //  values for fields.

   action onload() {
      a.i := 10;      // Assign non-default value.
      a.x := 20.0;    // Assign non-default value.
      Foo b;          // Local (in an action) declaration.
                      // The correlator does not create an event yet.
      b := new Foo;   // Create a default Foo event and assign
                      // it to local event.
      b.i := 10;      // Assign a non-default value.
      b.x := 20.0;    // Assign a non-default value.
      Foo c := a;     // You can assign a locally declared event to
                      // reference an existing event.
                      // Variables a and c alias the same event.
      c.i := 123      // The value of a.i is now also 123.
      Foo d := Foo(15,30.0);
                      // Create an event and also initialize it.
   }

Sending events to other monitors

After you inject a monitor into the correlator, it can communicate with other injected monitors under the following conditions:

• If the source monitor instance and the target monitor instance are in the same context, the source monitor instance can route an event that the target monitor instance is listening for. A routed event goes to the front of the context’s input queue. The correlator processes all routed events before it processes the next non-routed event on the context’s input queue. If the processing of a routed event routes another event, that event goes to the front of the input queue and the correlator processes it before it processes any other routed events on the queue. See Event processing order for monitors.
• If the source monitor instance and the target monitor instance are in different contexts, the source monitor instance must have a reference to the context that contains the target monitor instance. The source monitor instance can then send an event to the context that contains the target monitor instance. The target monitor instance must be listening for the sent event or the context that contains the target monitor instance must be subscribed to the channel that the event is sent on. See Sending an event to a particular context and Subscribing to channels.

Within a context, an application can use routed events and completion event listeners to initiate and complete a service request inline, that is, prior to processing any subsequent events on the input queue. See Specifying completion event listeners.

In the following example, the event listeners trigger in the order in which they are numbered.

monitor Client {
...
   listener_1:= on EventA() { route RequestB(...) }
   listener_5:= on ResponseForB () { doWork(); }
   listener_6:= on completed EventA() { doMoreWork(); }
...
}

monitor Service1{
...
   listener_2:= on RequestB(...)
   route RequestC();
   listener_4:= on ResponseForC{
   route ResponseForB ();
   }
...
}

monitor Service1a{
...
   listener_3:= on RequestC (...)
   route ResponseForC();
}

Best practices for working with routed events include:

• Keep them small; preferably zero, one, or two fields.
• Specify wildcards wherever appropriate in definitions of events that will be routed.

See also Generating events with the route statement.

Defining your application’s message exchange protocol

Monitors use events to communicate with each other. Consequently, an EPL application will have a well-defined message exchange protocol. A message exchange protocol defines the following:

• Types and structure of events that function as messages between monitor instances.
• Relationships among these events.
• Sequence and flow of events — which events are sent in response to receiving particular events.
• Which monitors need to be able to handle which events, and conversely, which monitors should not receive which events.
• Which channels these events are sent to, or whether they are sent directly between contexts.

When you define your application’s message exchange protocol, keep in mind that any event that the correlator processes is potentially available to all loaded monitors. Consequently, you want to follow conventions that prevent the inadvertent matching of events with event listeners. These conventions are:

• Use packages to restrict the scope of event names (for example, MyPackage, YourPackage).
• Use duplicate event definitions with different event names (for example, MyStartEvent, YourStartEvent).
• Use discriminating/addressing information in the event (for example, Request{integer senderId;...}, Response { integer toSender;...}).

While event definitions provide partial support for a robust message exchange protocol, they lack the ability to specify event patterns, request-response associations, and so on. You should insert structured comments in your event definition files to define this part of the message exchange protocol. The comments that describe the relationships among the events define the contract that the participating monitors must adhere to. It is up to you to document the expected flows and patterns and to ensure that your monitors comply with the contract.

Some common message exchange patterns are:

• Request/response
• Publish/subscribe/unsubscribe
• Start/stop

To identify the event types that a core monitor needs to support, consider the following:

• What actions do you want to perform on the object that the monitor represents? You might want to define an event that is dedicated to each action. For example, for an order processing monitor, you might define an event type for each of the following actions:
  • Place an order
  • Change an order
  • Cancel an order
  • Suspend trading
  • Resume trading
• What initialization and termination events are needed? Keep in mind that a core monitor is typically a factory that creates monitor instances that each represent a single entity. You probably want to define at least one event type for initialization and one event type for termination.
• Do you need other control events? For example, in the order processing example, do you need a control event that suspends all trading and applies to all orders? See Using events to control processing.
• Do you need to add any events to observe what is happening in the monitor? For example, each order processing monitor could support a request/response protocol to inquire of its state or it could simply send an OrderProcessingState event each time there is a significant state change.

Using events to control processing

In addition to using events to share data, you can use events to control processing. Control events are like switches. You use them to move a monitor from one state to another. Control events typically contain little or no data; that is, they have one or no fields.

A common use for control events is to initialize or terminate a process. For example, rather than use an onload() statement to set things up, it is good practice to use a monitor’s onload() statement to create an event listener for a start event. This practice defers initialization until the start event is received. Similarly, you can use a stop event to signal to a monitor that it should perform shutdown actions such as deallocating resources before you terminate the correlator.

For example, consider the following action:

action initialize() {
   on EndAuction() and not BeginAuction() startNormalProcessing();
   on BeginAuction() and not EndAuction() startAuctionProcessing();
   route RequestAuctionState(); //A service monitor will respond with
                                //an EndAuction or BeginAuction event
}

In this code, EndAuction and BeginAuction can be viewed as control events. Receipt of one of these events determines whether the monitor executes the logic associated with being in an auction or out of an auction.

About service monitors

Of course, all monitors can be considered to be providing some kind of service. However, as mentioned earlier, it can be helpful to view the monitors that make up your application as either core processing monitors or service monitors. It is common for a single instance of a service monitor to provide data to a set of monitor instances spawned from a core processing monitor instance.

Apama provides a number of service monitors that fit this pattern. These service monitors provide support for the following:

In addition, there are a number of service monitors for use by adapters:

Adding predefined annotations

Some EPL language elements can take predefined annotations. They provide the runtime and Apama Plugin for Eclipse with extra information about these language elements. These predefined annotations can appear immediately before the following:

• Event declarations
• Actions in monitors or event definitions

Annotations have packaged names like events. Thus, either their full name, or (preferably) a using declaration should be added to the file to allow the name to be used without having to specify its full name. Annotations are written as an at symbol (@) followed by the name of the annotation, followed by parameters in parentheses. The values used in annotation parameters must be literals. If both annotations and ApamaDoc are specified, the order should be: ApamaDoc, followed by annotations, followed by the language element that they apply to.

The following annotations are available:

MessageId("nestedEvent.fieldOfEvent")

using com.apama.epl.SideEffectFree;

monitor SomeMonitor {
    action onload() {
        on all Event() as e {
            log prettyPrint(e) at DEBUG;
        }
    }

    @SideEffectFree()
    action prettyPrint(Event e) returns string {
        return e.field1 +" : "+e.field2.toString();
    }
}

Subscribing to channels

Adapters and clients can specify the channel to deliver events to. In EPL, you can send an event to a specified channel. To obtain the events delivered to particular channels, monitor instances and external receivers can subscribe to those channels.

In a monitor instance, to receive events sent to a particular channel, call the subscribe() method on the monitor pseudo-type by using the following format:

monitor.subscribe(channel_name);

Replace channel_name with a string expression that indicates the name of the channel you want to subscribe to. You cannot specify a com.apama.Channel object that contains a string.

Call the subscribe() method from inside an action. Any monitor instance in any context can call monitor.subscribe().

The subscribe() method subscribes the calling context to the specified channel. When a context is subscribed to a channel events delivered to that channel are processed by the context, and can match against any listeners in that context. This includes listeners from monitor instances other than the instance that called subscribe(). However, the subscription is owned by the monitor instance that called monitor.subscribe(). If that monitor instance terminates, then any subscriptions it owned also terminate.

A subscription ends when the monitor instance that subscribed to the channel terminates or executes monitor.unsubscribe.

Whether an event is coming into the correlator or is generated inside the correlator, it is delivered to everything that is subscribed to the channel. If the target channel has no subscriptions from monitor instances nor external receivers then the event is discarded.

For example:

monitor pairtrade
{
   action onload()
   {
      on all PairTrade() as pt {
         spawn start_trade(pt.left, pt.right) to context(pt.toString());
      }
   }

   action start_trade(string sym1, string sym2)
   {
      monitor.subscribe("ticks-"+sym1);
      monitor.subscribe("ticks-"+sym2);
      // Next, set up listeners for sym1 and sym2.
...
   }
}

This code spawns a monitor for each trade pair. The spawned monitor subscribes to just the ticks for the symbols passed to it. If a symbol in one pair is slow to process, any unrelated pairs of symbols are unaffected. See Event association with a channel.

In a context, any number of monitor instances can subscribe to the same channel. When multiple monitors in a context require data from a channel the recommendation is for each monitor to subscribe to that channel. This ensures that the termination of one monitor does not affect the events received by other monitors. Subscriptions are reference counted. The result of multiple subscriptions to the same channel from the same context is that each event is delivered once as long as any of the subscriptions are active. An event is not delivered once for each subscription.

Suppose that in one monitor instance you unsubscribe from a channel but another monitor instance in the same context is subscribed to that channel. In the monitor instance that unsubscribed, be sure to terminate any listeners for the events from the unsubscribed channel. Events from the unsubscribed channel continue to come in because of the subscription from the other monitor instance.

To explicitly terminate a subscription, call monitor.unsubscribe(channel_name). In a given context, if you terminate the last subscription to a particular channel then the context no longer receives events from that channel. If events from the previously subscribed channel were delivered but not yet processed (they are waiting on the input queue) those events will be processed. This could include the processing of any listener matches. It is an error to unsubscribe from a channel that the calling monitor instance does not have a subscription to, and this will throw an exception.

If a monitor is going to terminate anyway there is neither requirement nor advantage to calling unsubscribe(). Calling unsubscribe() can be useful when a monitor listens to configuration data during startup but does not need to listen to it during normal processing.

If a correlator is configured to connect to Universal Messaging, then a channel might have a corresponding Universal Messaging channel. If there is a corresponding Universal Messaging channel, the monitor is subscribed to the Universal Messaging channel. See Choosing when to use Universal Messaging channels and when to use Apama channels.

About the default channel

The name of the default channel is the empty string.

Public contexts, including the main context, are always subscribed to the default channel.

When an adapter or client that is sending events to the correlator does not specify a target channel the event goes to the default channel. There is no need for a public context to subscribe to the default channel.

Events generated by the route statement are not delivered to the default channel.

An adapter that is using Universal Messaging to send events cannot use the default channel. See Configuring IAF adapters to use Universal Messaging.

About wildcard channels

An external receiver can be configured to listen on the com.apama.input channel, which is a wildcard channel for all events that come into the correlator. This can be useful for diagnostics, testing, or auditing, but it is not recommended for production. In a production environment, the recommendation is to explicitly specify the channels that the receiver should listen on.

A monitor instance cannot subscribe to com.apama.input.

To configure an external receiver to process all events generated in the correlator, specify that the receiver listens on the default channel (""). With this specification, a receiver would get all events generated by the send…to channel and emit statements regardless of the channel the event was directed to. Events generated by the route statement are not delivered to the default channel.

Adding service monitor bundles to your project

Depending on what your Apama application does, it might require one or more provided service monitors. Apama organizes service monitors into bundles. To use the service, you add the bundle to your Apama project in Apama Plugin for Eclipse. See also Specifying the project bundles properties.

When the bundle has been added in Apama Plugin for Eclipse, expand the bundle directory to see the contents. To understand exactly what each service monitor provides, open the service’s EPL file. The comments in the EPL file explain the purpose of each service monitor and how to use it.

You can also write your own service monitors. Best practices for doing this include:

• Follow good engineering practices for defining message exchange protocols.
• Copy the conventions used in the Apama-provided service monitors as these monitors implement common patterns.

Utilities for operating on monitors

Apama provides the following command-line utilities for operating on monitors. For details about using these utilities, see Correlator Utilities Reference.

• engine_inject — injects files into the correlator.
• engine_delete — removes items from the correlator.
• engine_send — sends Apama-format events to the correlator.
• engine_receive — lets you connect to a running correlator and receive events from that correlator.
• engine_watch — lets you monitor the runtime operational status of a running correlator.
• engine_inspect — lets you inspect the state of a running correlator.
• engine_management — lets you shut down a running correlator or obtain information about a running correlator. You can also use this utility to manage other types of components, such as adapters and continuous availability processes.