Concepts

This section discusses the concepts that are central to all Apama applications. A thorough understanding of these concepts can help you design and develop more robust Apama applications.

Event-driven programming

Events are data elements. Each event is a collection of attribute-value pairs that capture the state (or changes to state) of real-world or computer-based objects. Events consist of data and temporal attributes that represent the what, when, and where of an object. This can be the state of an object or the interaction of objects at a particular time. Real world examples of events include:

  • Sensor measurements from devices in a smart factory
  • Stock market trades and quotes
  • RFID signals
  • Satellite telemetry data
  • Card swipes at a turnstile
  • ATM transactions
  • Network activities/faults
  • Troop movement on a battlefield
  • Activity on a website
  • Electronic funds transfers
  • SCADA alerts (Supervisory Control and Data Acquisition)

Processing events requires event-driven programming. The hallmarks of event-driven programming include the following:

  • Program execution does not flow sequentially from beginning to end. There is no standard starting point.
  • Program execution happens in response to the arrival of events. Some external source pushes the events into your program.
  • Events arrive in asynchronous messages.
  • There are two main bodies of code: code that analyzes incoming events to determine if the events are of interest and code that performs actions when events of interest are found.

There are a lot of similarities between GUI programming and event driven programming. For example, in a GUI program you typically write code that responds to mouse clicks.

See also How EPL applications compare to applications in other languages.

Complex event processing

Complex Event Processing (CEP) is software technology that enables the detection and processing of

  • Events derived from other events. A derived event is an event that your application generates as a result of applying a method or action to one or more other events.
  • Event sequences, often with temporal constraints.

CEP programs find patterns in event data that enable detection of opportunities and threats. Timely responses are then pushed to the appropriate recipients. The responses can be in the form of automated events, such as placing orders in algorithmic trading systems, or alerts to someone using Business Activity Monitoring (BAM) dashboards. The result is faster and better operational decisions

EPL provides the features needed to write applications that can perform CEP. The following example shows how EPL can concisely define event patterns and rules.

The NewsCorrelation monitor’s onload() action defines a listener that specifies a complex event expression. The literal translation of the expression is “look for all news articles about any stock, followed by a 5% rise in the value of that stock within 5 minutes”. This is the kind of implied news impact that might be of interest to a trader or a market risk analyst.

monitor NewsCorrelation {
   action onload() {
      on all NewsItem() as news {
         on StockTick(symbol=news.subject) as tick {
            on StockTick(symbol=news.subject,
                         price >= (tick.price*1.05))
               within(300.0) alertUser;
         }
      }
   }
   action alertUser() {
      log "News to price movement Correlation for stock "
                  +news.subject+" has occurred";
   }
}

The on keyword specifies a listener. The initial listener nests two additional listeners that define the event sequence of interest. The listeners do the following:

  1. The initial listener watches for all NewsItem events.

  2. Each time the correlator detects a NewsItem event, this listener captures it in a news variable.

  3. The first nested listener then watches for a StockTick event for the stock that the news item was about. This listener uses the news variable to access the information from the previously detected event.

  4. When the correlator detects a matching StockTick event, the first nested listener captures it in the tick variable.

  5. The innermost listener then watches for another StockTick event for the same stock but with a price that is at least 5% higher than the price in the event captured by the tick variable. The within keyword indicates that the correlator must detect the second StockTick event within 300 seconds (5 minutes) of finding the initial NewsItem event.

  6. If the correlator finds a second StockTick event that matches within 5 minutes, the monitor sends a message to the log file. The nested listeners terminate.

    If the correlator does not find a second StockTick event that matches within the 5 minutes, the nested listeners terminate without sending a message to the log.

Understanding monitors and listeners

An introduction to monitors and listeners is in Description of Apama EPL. As mentioned there, monitors are the basic program component that you inject into the correlator. You write monitors in EPL.

A monitor defines:

  • One or more listeners. A listener is the EPL mechanism that specifies the event or sequence of events that you are interested in. Conceptually, listeners sift through the streams of events that come in to the correlator and detect matching events.
  • One or more actions. An action is one or more operations that the correlator performs. An action might be the registration of a listener or it might be the execution of an operation when the correlator finds a match between an incoming event or sequence and a listener.

When the correlator executes an on statement, it creates a listener. A listener watches for an event, or a sequence of events, that matches the event expression specified in the on statement. An event expression defines one or more event templates. Each event template defines an event type to look for, and specifies whether the event’s fields should have any specific values. In addition, listeners can specify

  • Temporal constraints: For example, a listener can specify that two events of interest must be received within 10 minutes.
  • Logic: For example, a listener can specify that it is interested in event A or event B or event C.

It is often desirable to listen, separately but concurrently, for different instances of the same event type. For example, you might want to listen for and process, separately but concurrently, stock tick events for different stocks. EPL accomplishes this by letting a monitor instance spawn other monitor instances.

In the monitor code, you spawn a monitor instance by specifying the spawn keyword followed by an action. Each act of spawning creates a new instance of the monitor.

When the correlator spawns a monitor instance, it does the following:

  1. The correlator creates a new monitor instance from the original monitor instance. The new monitor instance is almost identical to the original. The new monitor instance has a copy of the variables from the original but the active listeners from the original monitor instance are not copied.
  2. The correlator invokes the named action on the new monitor instance.

Monitors that contain spawn statements typically act as factories, creating new monitor instances that all listen for the same event type but where each listens for events that have different values in one or more fields. Also, monitors can spawn to particular threads, referred to as contexts in EPL. This enables the correlator to concurrently process multiple monitor instances. (You must create contexts in EPL to implement parallel processing. You can refer to contexts from EPL.)

The lifecycle of a monitor is as follows:

  1. You use Apama Plugin for Eclipse or a correlator utility to inject the EPL that defines the monitor into the correlator.

  2. The correlator creates the original monitor instance, including space for variables as needed.

  3. The correlator executes the monitor instance’s onload() action.

  4. The original monitor instance might spawn several times creating new monitor instances. For each spawned monitor instance, the correlator creates a copy of the original monitor instance’s variable space and then executes the specified action.

  5. A monitor instance terminates when it has no active listeners. Upon termination, the correlator invokes the monitor instance’s ondie() method, if one is defined. Note that it is possible for a monitor instance to remain active after the monitor instance that spawned it has terminated.

  6. When the last instance of a particular monitor terminates, the correlator calls the monitor’s onunload() method, if it defines one. The last monitor instance to terminate might be the original monitor instance or a spawned monitor instance. Regardless, when the last instance terminates the correlator invokes the monitor’s ondie() method and then the monitor’s onunload() method, if these methods are defined.

    For example, suppose that a monitor definition specifies an ondie() method and an onunload() method. You inject this monitor and the correlator creates the original monitor instance. The original monitor instance spawns 9 times. Consequently, there are 10 instances of that monitor in the correlator. After all of these monitor instances have terminated, the correlator will have called ondie() 10 times and it will have called onunload() once.

See Getting started with Apama EPL.