Event file format

You can use the engine_send tool to stream a sequence of events through the correlator. The engine_send tool accepts input from one or more data files to support tests or simulations, or from stdin to allow dynamic generation of events. In the latter case, you can generate events from user input or by piping output from an event generation program to engine_send. In all cases, engine_send requires event data formatted as described in this section.

The engine_receive tool outputs events in this same file format. This means you can use events generated by the engine_receive tool as input to a second correlator that is executing the engine_send tool. For detailed information on the engine_receive tool, see Receiving events from correlators.

Event representation

A single event is identified by the event type name and the values of all fields as defined by that type. Event type names must be fully-qualified by prefixing the package name into which the corresponding event type was injected, unless the event was injected into the default package.

The specific EPL types and how they map from the event representation are shown in an example in Event types, but there are certain basic types that can be included as shown in the following example:

// integer
MyEvent(-1,1)

// decimal and float
MyEvent(-2.0,2.0)

// decimal and float in exponential form (0.02,200)
MyEvent(-2.0e-2,2.0e2)

// string
MyEvent("three")

// boolean
MyEvent(true,false)

Both decimal and float types can be represented in scientific form if required, including when nested in optional or any types and inner events.

A string is a sequence of characters enclosed in double quotes ("). The backslash character (\) is used as an escape character to allow inclusion of special characters such as newlines and horizontal tabs.

Examples:

MyEvent("Hello, World!")
MyEvent("\ta\tstring\twith\ttabs\tbetween\twords")
MyEvent("a string on\n two lines")
MyEvent("a string with \\ a backslash and a \" quote")

Localization, such as different formats for decimals or quotation marks, is not supported.

Each event is given on a separate line. Only single-line comments are allowed. Start each comment line with // or #. Any blank lines are ignored.

For example, following are three valid events:

// This is an event file that contains some sample events.
// Here are three stock price events:
my.test.StockPrice("XRX",11.1)
my.test.StockPrice("IBM",130.6)
my.test.StockPrice("MSFT",70.5)

For those events, the following event type definition must be injected into the package test:

package my.test;

event StockPrice {
 string stockSymbol;
 float stockValue;
}

If the above events were saved in an .evt file, engine_send would send each event in turn, as soon as the previous event finished transmission. This behavior can optionally be modified in several ways:

• Specifying that batches of events should be sent at specified time intervals.
• Specifying that all events on all queues should be processed before sending the next event.

Event timing

In .evt files, it is possible to specify the following:

• Time intervals for sending batches of events to the correlator.
• Waiting for all events on all queues at that point in time to be processed before sending the next event.

Adding BATCH tags to send events at intervals

You can specify time intervals for sending batches of events to the correlator. This is achieved by specifying the BATCH tag followed by a time offset in milliseconds. For example, the following specifies two batches of events to be sent 50 milliseconds apart.

BATCH 50
StockPrice("XRX", 11.1)
StockPrice("IBM", 130.6)
StockPrice("MSFT", 70.5)
BATCH 100
StockPrice("XRX", 11.0)
StockPrice("IBM", 130.8)
StockPrice("MSFT", 70.1)

The addition of a “time” allows simulations of “bursts” of events, or more random distributions of event traffic. Times are measured as an offset from when the current file was opened. If only one file of events is being read and transferred, then this would be the same as since the start of a run (that is, from the time that the engine_send tool starts processing the event data). If multiple files are being read in, the timing starts all over again upon the (re)opening of each file.

If the time given for a batch is less than the current time, or if no time is given following a BATCH tag (or if no BATCH tag is provided), then the events are sent as soon as they are read in, immediately following the preceding batch.

Using &FLUSHING mode for more predictable event processing order

Sending events in flushing mode can help provide a more predictable event processing order. However, flushing mode is slower than the default behavior.

By default, events are delivered in an optimal way, not waiting for previously sent events to be processed before the next event is delivered to contexts (or other consumers of channels). When flushing mode is enabled the behavior is as follows:

1. The correlator sends an event.
2. The correlator processes all events on all queues at that point in time, repeating this as many times as specified in the flushing specification.
3. The correlator sends the next event.

To enable flushing mode, insert the following line in a .evt file:

&FLUSHING(n)

Replace n with an integer that specifies how many times to flush queues in between each event. Set this to the maximum length of a chain of send-to operations between contexts that could occur in your application. If you specify a number that is bigger than required the correlator simply repeats the flush operation, which incurs a small overhead. To disable flushing mode, insert the following line in the .evt file:

&FLUSHING(0)

Enabling or disabling flushing mode affects only the events sent on that connection or from that event file.

When sending &TIME events in to a multi-context application, the time ticks are delivered directly to all contexts. This can be different than the way in which events in the .evt file are sent in to the correlator and then sent between contexts in an application. This difference can result in processing events at an incorrect simulated time. In these cases, sending &FLUSHING(1), for example, before sending time ticks and events can result in more predictable and reliable behavior.

Event types

The following example illustrates how each type is specified in an event representation. Given the event type definitions:

event Nested {
  integer i;
}
event EveryType {
  boolean b;
  integer i;
  float f;
  string s;
  location l;
  sequence<integer> si;
  dictionary<integer, string> dis;
  Nested n;
  optional<Nested> opt;
  any anyValue;
}

the following is an expanded string representation for an EveryType event:

EveryType (
  true,                         // boolean is true/false (lower-case)
  -10,                          // positive or negative integer
  1.73,                         // float
  "foo",                        // strings are (double) quoted
  location(1.0,1.0,5.0,5.0),    // locations are 4-tuples of float values
  [1,2,3],                      // sequences are enclosed in brackets []
  {1:"a",2:"b"},                // dictionaries are enclosed in braces {}
  Nested(1),                    // nested events include event type name
  optional<Nested>(Nested(10)), // optional payload inside parentheses, may be
                                // empty, in which case it is represented by the
                                // string "optional()"
  any(integer,42)               // any names a type and the string form of that
                                // type; other examples include the following:
                                // any(sequence<Nested>,[Nested(1)])
                                // may also be empty, in which case the string
                                // form is: "any()"
);

Note: This example is split over several lines for clarity. In practice, this definition must all be written on the same line, and without the comments. Otherwise the correlator will fail to parse the event.

Types can of course be nested to create more complex structures. For example, the following is a valid event field definition:

sequence<dictionary<integer, Nested>>

and the following is a valid representation of a value for this field:

[{1:Nested(1)}, {2:Nested(2)}, {3:Nested(3)}]

You can also get an event’s string representation in EPL by using the toString() method.

Event association with a channel

The engine_send tool can send an event file that associates channels with events. Likewise, the engine_receive tool can output an event file that includes the channel on which an event was received. The event format is the same for both tools:

"channel_name",event_type_name(field_value1[, field_valuen]...)

For example, suppose you want to send Tick events, which contain a string followed by an integer, to the PreProcessing channel. The contents of the .evt file would look like this:

"PreProcessing",Tick("SOW", 35)
"PreProcessing",Tick("IBM", 135)

A channel name is optional. In a file being sent with the engine_send tool, you can mix event representations that specify channels with event representations that do not specify channels. Events for which a channel is specified go to only those contexts subscribed to that channel.

The default behavior is that events are sent on the default channel (the empty string) when a channel is not explicitly specified. Events sent on the default channel go to all public contexts. To change the default behavior for events sent by the engine_send tool, you can specify engine_send -c channel. If a channel is not explicitly specified for an event, then it is sent to the channel identified with the -c option.