In EPL programs (monitors), you can use standard EPL plug-ins provided with Apama and you can also use EPL plug-ins that you define yourself. An EPL plug-in consists of an appropriately formatted library of C++ functions, which can be called from within EPL code. In the case of a plug-in written in Java, the Java classes that are called from an application’s EPL code are contained in a jar file. The correlator does not need to be modified to enable or to integrate with a plug-in, as the plug-in loading process is transparent and occurs dynamically when required.
When using a plug-in, you can call the functions it contains directly from EPL, passing EPL variables and literals as parameters, and getting return values that can be manipulated.
Overhead of using plug-ins
The overhead when using EPL plug-ins is very small.
However, you do need to ensure that you do not block the correlator for a long period of time. For example, you do not want to use a plug-in for doing extensive, synchronous, time-consuming calculations.
If you need to perform a time-consuming operation, use asynchronous processing in the plug-in to perform the calculation on a background thread, and then deliver the result via an event to the monitor instance which requested it. For example, your plug-in method might return an ID for the monitor to listen to an event with that ID. When the calculation is complete, the plug-in sends an event with that ID to the context from which it was originally called:
integer id := plugin.compute(inputs);
on ComputeResult(id=id) as cr {
// process cr.result
}
When to use plug-ins
Custom plug-ins can be written in C++, Java or Python. A custom plug-in is a suitable solution in the following situations:
You have an in-house or third-party library of (possibly complex) functions or classes that you want to re-use.
The operations you need to perform are more easily/efficiently performed in another language than using EPL. For example, you need to use data structures that are not easily represented in EPL.
Info
If your concern is purely performance, then the compiled EPL runtime (available on Linux, see also the --runtime option in Starting the correlator) may be sufficient and in some cases can produce results better than other languages, including C++ and Java.
When not to use plug-ins
In general, when you can efficiently write the desired operation in EPL, an all-EPL solution is preferable to one that involves custom-developed plug-ins. Apama customers who experience problems with correlator stability when using custom-developed plug-ins will be asked by Cumulocity product support to remove the plug-in and reproduce the problem prior to being offered further technical help. Cumulocity product support lifts this restriction only if the plug-ins have certification from Apama product management.
Using the TimeFormat event library
The TimeFormat event library uses the Time Format plug-in.
The TimeFormat event provides routines to obtain the current time and convert to or from string representations of time.
Internally, the correlator expresses time as seconds since the Unix epoch (1 Jan 1970, midnight UTC). UTC has no daylight savings applied and each day consists of 86,400 seconds. Leap seconds are effectively treated as double-length seconds, so that the number of seconds in the day is constant. This is the form of currentTime and is convenient for performing arithmetic, such as differences between times. For more information on this variable, see currentTime.
To convert from string form to float form, use a parseTime method. To convert from float form to string form, use a format method. Both take a format String, which is a string which describes the string form of the time. For more information, see Format specification for the TimeFormat functions.
The parseTime method is available on the TimeFormat event directly. Or you can pre-compile a pattern and then perform parsing using the compiled pattern. A CompiledPattern object is obtained from the TimeFormat event using one of the compilePattern methods (depending on which time zone the pattern should use by default). The CompiledPattern object can be stored in a monitor variable, as an instance of an event or in a local variable and used by listeners. Re-using a CompiledPattern is more efficient than calling one of the TimeFormat.parseTime methods as the format String only needs to be read and compiled once. Calling parse on the TimeFormat event is equivalent to passing the same format String to generate a CompiledPattern and calling parse on that event. It is also possible to create multiple CompiledPattern events if your application needs to use several different formats for time.
There are also functions to obtain the current system time. It depends on whether your purpose is to measure the time intervals between two events on the same machine (getMicroTime()), or to identify when an event occurred in human terms or across machines (getSystemTime() or currentTime).
getSystemTime() provides an absolute time. Its purpose is to give an indication of the wall clock time rather than measuring time intervals. You would see a jump in the values if someone changed the time (for example, manually in the operating system or to synchronize with an NTP server). These jumps would be annoying if you were trying to measure latency of some operation. Whereas, if you use getMicroTime(), you would not see any effect from such a change to the operating system time.
getMicroTime() provides a high precision time, which is suitable for high precision relative times. The absolute value of getMicroTime() depends on the host operating system. Its main purpose is for measuring small durations of time. Unless you need such high precision, it is probably better to use currentTime or getSystemTime() instead.
The following table gives a brief overview of the different time functions:
getMicroTime()
getSystemTime()
currentTime
Epoch
Undefined / determined by the operating system
Unix epoch (midnight 1 January 1970 UTC)
Unix epoch (midnight 1 January 1970 UTC)
Affected by the host operating system’s automatic adjustment for daylight savings
No
No
No
Affected by a change to the host operating system’s time zone
No
No
No
Affected by a manual change to the host operating system’s time
No
Yes
Yes
Affected by the host operating system’s automatic time synchronization
No
Yes
Yes
Represents a unique point in time
No
Yes, except for leap seconds
Yes, except for leap seconds
Precision
Microseconds
Milliseconds
Dependent on the correlator clock rate *
Remains constant while processing a single event
No
No
Yes
Use for date/time formatting
No
Yes
Yes
Info
All of the above are floating point values representing seconds since their epoch.
Patterns with textual elements operate by default in English, but will instead both produce output and expect input in another language if that has been set in the environment. For example, under Linux, if the correlator is running with the LC_ALL environment variable set to "fr_FR", the format "EEEE dd MMMM yyyy G" produces and expects "jeudi 01 janvier 1970 ap. J.-C." for time 0.0.
When you use the TimeFormat event library you can use the TZ environment variable to select a particular locale to be used by the event library. Specify the value in either of the following formats:
Continent/City
Ocean/Archipelago
For example: TZ=Europe/London. The alternative shortened format will not work correctly. For example, TZ=GB will not be recognized. If you specify something like this, Coordinated Universal Time (UTC) is used instead.
The parse functions parse the value contained by the timeDate parameter according to the format passed in the format parameter or wrapped by the CompiledPattern.
All functions return the result as a float of seconds since the epoch.
If the timeDate parameter specifies only a time, the date is assumed to be 1 January 1970 in the appropriate time zone. If the timeDate parameter specifies only a date, the time is assumed to be the midnight that starts that day in the appropriate time zone. Adding them together as seconds gives the right result.
If the timeDate string specifies a time zone, and there is a matching z, Z, v, or V in the format string, the time zone specified in the timeDate string takes precedence over any other ways of specifying the time zone. For example, when you call the parseUTC() or parseTimeWithTimeZone() function, and you specify a time zone or offset in the timeDate string, the time zone or offset specification in the timeDate string overrides the time zone you specify as a parameter to the parseTimeWithTimeZone() function and the normal interpretation of times and dates as UTC by the parseUTC() function.
Parsing behavior is undefined if the format string includes duplicate elements such as "MM yyyy MMMM", has missing elements such as "MM", or it includes potentially contradictory elements and is given contradictory input, for example, "Tuesday 3 January 1970" (it was actually a Saturday).
Dates before 1970 are represented by negative numbers.
Example
The following example returns 837007736:
TimeFormat.parseTime("yyyy.MM.dd G 'at' HH:mm:ss", "1996.07.10 AD at 15:08:56")
In the first example, the +0900 specification in the timeDate string overrides the UTC specification for the time zone name parameter. In the second example, the +0900 specification in the timeDate string overrides the UTC specified by calling the parseUTC() function.
TimeFormat utility functions
The utility functions provide methods for manipulating datetimes. These include, for a given datetime, finding the time of the previous midnight, how many seconds have passed since that midnight, the number of days that have passed since the beginning of the epoch, and the offset in seconds for time zones from UTC.
The format and parse functions make use of the SimpleDateFormat class provided in the International Components for Unicode libraries. SimpleDateFormat is a class for formatting and parsing dates in a language-independent manner.
Pattern letters in format strings
The TimeFormat functions use the SimpleDateFormat class to transform between a string that contains a time and/or date and a normalized representation of that time and/or date. In this case, the normalized representation is the number of seconds since the epoch.
For the operation to succeed, it is important to define the format string so that it exactly represents the format of the time and/or date you provide as a string in the timeDate parameter to a parse function, or expect to be returned from a format function. You specify the format as a time pattern. In this pattern, all ASCII letters are reserved as pattern letters.
The number of pattern letters determines the format as follows:
For pattern letters that represent text
If you specify four or more letters, the SimpleDateFormat class transforms the full form. For example, EEEE formats/parses Monday.
If you specify fewer than four letters, the SimpleDateFormat class transforms the short or abbreviated form if it exists. For example, E, EE, and EEE each formats/parses Mon.
For pattern letters that represent numbers
Specify the minimum number of digits.
If necessary, SimpleDateFormat prepends zeros to shorter numbers to equal the number of digits you specify. For example, m formats/parses 6, mm formats/parses 06.
Year is handled specially. If the count of y is 2, the year is truncated to 2 digits. For example, yyyy formats/parses 1997, while yy formats/parses 97.
Unlike other fields, fractional seconds are padded on the right with zeros.
For pattern letters that can represent text or numbers
If you specify three or more letters, the SimpleDateFormat class transforms text. For example, MMM formats/parses Jan, while MMMM formats/parses January.
If you specify one or two letters, the SimpleDateFormat class transforms a number. For example, M formats/parses 1, and MM formats/parses for 01.
The following table provides the meaning of each letter you can specify in a pattern. After the table, there are a number of combined examples.
Descriptions of pattern letters in format strings:
Symbol
Meaning
Presentation
Example
Sample Result
G
Era designator
Text
GG
ADBC
y (lowercase)
Year
Number
yyyyyy
961996
Y (uppercase)
Year for indicating which week of the year. Use with the w symbol. See “Week in year” later in this table.
Day of week (1-7) This is locale dependent. Typically, Monday is 1.
Number
e
4
D
Day in year
Number
DDD
DDD
DDD
707
007
123
F
Day of particular week in month (1-7). Use with W (uppercase) for week in month. See “Week in month” later in this table.
Number
See example for “Week in month”.
w (lowercase)
Week in year. Use with uppercase Y. The first week of a month/year is defined as the earliest seven day period beginning on the specified “first day of the week” and containing at least the specified “minimal number of days in the first week” of that month/year.
Note that the first day of the week and the minimum days in the first week (1 to 7) are dependent upon the calendar in use (which depends upon the locale resource data).
For example, January 1, 1998 was a Thursday.
If the first day of the week is a Monday and the minimum days in a week is 4 (these are the values reflecting ISO 8601 and many national standards), then week 1 of 1998 starts on December 29, 1997, and ends on January 4, 1998. However, if the first day of the week is a Sunday and the minimum number of days in a week is 4, then week 1 of 1998 starts on January 4, 1998, and ends on January 10, 1998. The first three days of 1998 are then part of week 53 of 1997.
Number
The first example below uses uppercase Y. The second example shows the difference when you use lowercase y. "'Week' w YYYY"
"'Week' w yyyy"
Suppose you are transforming December 31st, 2008, which is a Wednesday (results for the US locale). "Week 1 2009"
"Week 1 2008"
W (uppercase)
Week in month. The values are calculated similar to “Week in year” (w).
Weeks numbered with a minus sign (such as -2 or -1) and 0 precede the first week. Weeks numbered 2, 3 and so on follow the first week.
Number
"'Day' F 'of Week' W"
"Day 2 of Week 3"
a
AM/PM marker
Text
aa
AMPM
k
Hour in day (1-24)
Number
kkk
kk
101
24
K
Hour in AM/PM (0-11)
Number
KKK
KK
007
11
z (lowercase)
Time zone
Text
z, zz, zzzzzzz
GMT+05:30
Pacific Standard Time
Z (uppercase)
Time zone (RFC 822)
Number
Z
-0800
v (lowercase)
Generic time zone
Text
vvvvv
PT (assuming US locale)
Pacific Time
V (uppercase)
Short time zone ID
Text
V
gblon
VV
Long time zone ID
Text
VV
Europe/London
VVV
Time zone exemplar city
Text
VVV
London
VVVV
Time zone location text
Text
VVVV
United Kingdom Time
O (uppercase)
Localized GMT time zone
Text
OOOOO
GMT-8GMT-8:00
X (uppercase)
ISO8601 time zone with Z
Text
XXX
XXX
XXXX
XXXXX
-08, +0530, Z-0800, Z
-08:00, Z
-0800, -075258, Z
-08:00, -07:52:58, Z
x (lowercase)
ISO8601 time zone without Z
Text
xxx
xxx
xxxx
xxxxx
-08, +0530-0800
-08:00
-0800, -075258
-08:00, -07:52:58
g
Julian day
Number
g
2451334
A
Milliseconds in day
Number
A
69540000
'
Escape for text
Delimiter
"'Week' w YYYY"
"Week 1 2009"
''
Single quote
Literal
"KK 'o''clock'"
"11 o'clock"
Any character in the format pattern that is not in the range of [’a’..’z’] or [’A’..’Z’] is treated as quoted text. For example, the following characters can be in a timeDate string without being enclosed in quotation marks:
:
.
,
#
@
A pattern that contains an invalid pattern letter results in a -1 return value.
The following table gives examples that assume the US locale:
Format pattern
Suitable timeDate string
yyyy.MM.dd G 'at' HH:mm:ss z
1996.07.10 AD at 15:08:56 GMT+08:30
EEE, MMM d, ''yy
Wed, July 10, '96
h:mm a
12:08 PM
hh 'o''clock' a, zzzz
12 o'clock PM, Pacific Daylight Time
K:mm a, z
0:00 PM, GMT+07:30
yyyyy.MMMMM.dd GGG hh:mm aaa
1996.July.10 AD 12:08 PM
When parsing a date string using the abbreviated year pattern (y or yy), SimpleDateFormat (and hence all parse functions) must interpret the abbreviated year relative to some century. It does this by adjusting dates to be within 79 years before and 19 years after the time the SimpleDateFormat instance is created. For example, using a pattern of MM/dd/yy and a SimpleDateFormat instance created on Jan 1, 1997, the string 01/11/12 would be interpreted as Jan 11, 2012 while the string 05/04/64 would be interpreted as May 4, 1964. During parsing, only strings consisting of exactly two digits, as defined by Unicode::isDigit(), will be parsed into the default century. Any other numeric string, such as a one digit string, a three or more digit string, or a two digit string that is not all digits (for example, -1), is interpreted literally. So 01/02/3 or 01/02/003 are parsed, using the same pattern, as Jan 2, 3 A.D. Likewise, 01/02/-3 is parsed as Jan 2, 4 B.C. Behavior is undefined if you specify a two-digit date that might be either twenty years in the future or eighty years in the past.
If the year pattern has more than two y characters, the year is interpreted literally, regardless of the number of digits. So using the pattern MM/dd/yyyy, 01/11/12 parses to Jan 11, 12 A.D.
When numeric fields abut one another directly, with no intervening delimiter characters, they constitute a run of abutting numeric fields. Such runs are parsed specially. For example, the format HHmmss parses the input text 123456 to 12:34:56, parses the input text 12345 to 1:23:45, and fails to parse 1234. In other words, the leftmost field of the run is flexible, while the others keep a fixed width. If the parse fails anywhere in the run, then the leftmost field is shortened by one character, and the entire run is parsed again. This is repeated until either the parse succeeds or the leftmost field is one character in length. If the parse still fails at that point, the parse of the run fails.
For time zones that have no names, SimpleDateFormat uses strings GMT+hours:minutes or GMT-hours:minutes.
The calendar defines what is the first day of the week, the first week of the year, whether hours are zero based or not (0 vs. 12 or 24), and the time zone. There is one common number format to handle all the numbers; the digit count is handled programmatically according to the pattern.
Midnight and noon
The format "HH:mm" parses "24:00" as midnight that ends the day. Given the formal "hh:mm a", both "00:00 am" and "12:00 am" parse as the midnight that begins the day. Note that "00:00 pm" and "12:00 pm" are both midday.
Supported time zones
The TimeFormat event library supports the following time zones. We recommend use of the “Area/City” time zones. Constants to provide ease of access can be found in the TimeZone namespace in the API reference for EPL (ApamaDoc).
ACT
AET
AGT
ART
AST
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
Africa/Asmera
Africa/Bamako
Africa/Bangui
Africa/Banjul
Africa/Bissau
Africa/Blantyre
Africa/Brazzaville
Africa/Bujumbura
Africa/Cairo
Africa/Casablanca
Africa/Ceuta
Africa/Conakry
Africa/Dakar
Africa/Dar_es_Salaam
Africa/Djibouti
Africa/Douala
Africa/El_Aaiun
Africa/Freetown
Africa/Gaborone
Africa/Harare
Africa/Johannesburg
Africa/Juba
Africa/Kampala
Africa/Khartoum
Africa/Kigali
Africa/Kinshasa
Africa/Lagos
Africa/Libreville
Africa/Lome
Africa/Luanda
Africa/Lubumbashi
Africa/Lusaka
Africa/Malabo
Africa/Maputo
Africa/Maseru
Africa/Mbabane
Africa/Mogadishu
Africa/Monrovia
Africa/Nairobi
Africa/Ndjamena
Africa/Niamey
Africa/Nouakchott
Africa/Ouagadougou
Africa/Porto-Novo
Africa/Sao_Tome
Africa/Timbuktu
Africa/Tripoli
Africa/Tunis
Africa/Windhoek
America/Adak
America/Anchorage
America/Anguilla
America/Antigua
America/Araguaina
America/Argentina/Buenos_Aires
America/Argentina/Catamarca
America/Argentina/ComodRivadavia
America/Argentina/Cordoba
America/Argentina/Jujuy
America/Argentina/La_Rioja
America/Argentina/Mendoza
America/Argentina/Rio_Gallegos
America/Argentina/Salta
America/Argentina/San_Juan
America/Argentina/San_Luis
America/Argentina/Tucuman
America/Argentina/Ushuaia
America/Aruba
America/Asuncion
America/Atikokan
America/Atka
America/Bahia
America/Bahia_Banderas
America/Barbados
America/Belem
America/Belize
America/Blanc-Sablon
America/Boa_Vista
America/Bogota
America/Boise
America/Buenos_Aires
America/Cambridge_Bay
America/Campo_Grande
America/Cancun
America/Caracas
America/Catamarca
America/Cayenne
America/Cayman
America/Chicago
America/Chihuahua
America/Ciudad_Juarez
America/Coral_Harbour
America/Cordoba
America/Costa_Rica
America/Creston
America/Cuiaba
America/Curacao
America/Danmarkshavn
America/Dawson
America/Dawson_Creek
America/Denver
America/Detroit
America/Dominica
America/Edmonton
America/Eirunepe
America/El_Salvador
America/Ensenada
America/Fort_Nelson
America/Fort_Wayne
America/Fortaleza
America/Glace_Bay
America/Godthab
America/Goose_Bay
America/Grand_Turk
America/Grenada
America/Guadeloupe
America/Guatemala
America/Guayaquil
America/Guyana
America/Halifax
America/Havana
America/Hermosillo
America/Indiana/Indianapolis
America/Indiana/Knox
America/Indiana/Marengo
America/Indiana/Petersburg
America/Indiana/Tell_City
America/Indiana/Vevay
America/Indiana/Vincennes
America/Indiana/Winamac
America/Indianapolis
America/Inuvik
America/Iqaluit
America/Jamaica
America/Jujuy
America/Juneau
America/Kentucky/Louisville
America/Kentucky/Monticello
America/Knox_IN
America/Kralendijk
America/La_Paz
America/Lima
America/Los_Angeles
America/Louisville
America/Lower_Princes
America/Maceio
America/Managua
America/Manaus
America/Marigot
America/Martinique
America/Matamoros
America/Mazatlan
America/Mendoza
America/Menominee
America/Merida
America/Metlakatla
America/Mexico_City
America/Miquelon
America/Moncton
America/Monterrey
America/Montevideo
America/Montreal
America/Montserrat
America/Nassau
America/New_York
America/Nipigon
America/Nome
America/Noronha
America/North_Dakota/Beulah
America/North_Dakota/Center
America/North_Dakota/New_Salem
America/Nuuk
America/Ojinaga
America/Panama
America/Pangnirtung
America/Paramaribo
America/Phoenix
America/Port-au-Prince
America/Port_of_Spain
America/Porto_Acre
America/Porto_Velho
America/Puerto_Rico
America/Punta_Arenas
America/Rainy_River
America/Rankin_Inlet
America/Recife
America/Regina
America/Resolute
America/Rio_Branco
America/Rosario
America/Santa_Isabel
America/Santarem
America/Santiago
America/Santo_Domingo
America/Sao_Paulo
America/Scoresbysund
America/Shiprock
America/Sitka
America/St_Barthelemy
America/St_Johns
America/St_Kitts
America/St_Lucia
America/St_Thomas
America/St_Vincent
America/Swift_Current
America/Tegucigalpa
America/Thule
America/Thunder_Bay
America/Tijuana
America/Toronto
America/Tortola
America/Vancouver
America/Virgin
America/Whitehorse
America/Winnipeg
America/Yakutat
America/Yellowknife
Antarctica/Casey
Antarctica/Davis
Antarctica/DumontDUrville
Antarctica/Macquarie
Antarctica/Mawson
Antarctica/McMurdo
Antarctica/Palmer
Antarctica/Rothera
Antarctica/South_Pole
Antarctica/Syowa
Antarctica/Troll
Antarctica/Vostok
Arctic/Longyearbyen
Asia/Aden
Asia/Almaty
Asia/Amman
Asia/Anadyr
Asia/Aqtau
Asia/Aqtobe
Asia/Ashgabat
Asia/Ashkhabad
Asia/Atyrau
Asia/Baghdad
Asia/Bahrain
Asia/Baku
Asia/Bangkok
Asia/Barnaul
Asia/Beirut
Asia/Bishkek
Asia/Brunei
Asia/Calcutta
Asia/Chita
Asia/Choibalsan
Asia/Chongqing
Asia/Chungking
Asia/Colombo
Asia/Dacca
Asia/Damascus
Asia/Dhaka
Asia/Dili
Asia/Dubai
Asia/Dushanbe
Asia/Famagusta
Asia/Gaza
Asia/Harbin
Asia/Hebron
Asia/Ho_Chi_Minh
Asia/Hong_Kong
Asia/Hovd
Asia/Irkutsk
Asia/Istanbul
Asia/Jakarta
Asia/Jayapura
Asia/Jerusalem
Asia/Kabul
Asia/Kamchatka
Asia/Karachi
Asia/Kashgar
Asia/Kathmandu
Asia/Katmandu
Asia/Khandyga
Asia/Kolkata
Asia/Krasnoyarsk
Asia/Kuala_Lumpur
Asia/Kuching
Asia/Kuwait
Asia/Macao
Asia/Macau
Asia/Magadan
Asia/Makassar
Asia/Manila
Asia/Muscat
Asia/Nicosia
Asia/Novokuznetsk
Asia/Novosibirsk
Asia/Omsk
Asia/Oral
Asia/Phnom_Penh
Asia/Pontianak
Asia/Pyongyang
Asia/Qatar
Asia/Qostanay
Asia/Qyzylorda
Asia/Rangoon
Asia/Riyadh
Asia/Saigon
Asia/Sakhalin
Asia/Samarkand
Asia/Seoul
Asia/Shanghai
Asia/Singapore
Asia/Srednekolymsk
Asia/Taipei
Asia/Tashkent
Asia/Tbilisi
Asia/Tehran
Asia/Tel_Aviv
Asia/Thimbu
Asia/Thimphu
Asia/Tokyo
Asia/Tomsk
Asia/Ujung_Pandang
Asia/Ulaanbaatar
Asia/Ulan_Bator
Asia/Urumqi
Asia/Ust-Nera
Asia/Vientiane
Asia/Vladivostok
Asia/Yakutsk
Asia/Yangon
Asia/Yekaterinburg
Asia/Yerevan
Atlantic/Azores
Atlantic/Bermuda
Atlantic/Canary
Atlantic/Cape_Verde
Atlantic/Faeroe
Atlantic/Faroe
Atlantic/Jan_Mayen
Atlantic/Madeira
Atlantic/Reykjavik
Atlantic/South_Georgia
Atlantic/St_Helena
Atlantic/Stanley
Australia/ACT
Australia/Adelaide
Australia/Brisbane
Australia/Broken_Hill
Australia/Canberra
Australia/Currie
Australia/Darwin
Australia/Eucla
Australia/Hobart
Australia/LHI
Australia/Lindeman
Australia/Lord_Howe
Australia/Melbourne
Australia/NSW
Australia/North
Australia/Perth
Australia/Queensland
Australia/South
Australia/Sydney
Australia/Tasmania
Australia/Victoria
Australia/West
Australia/Yancowinna
BET
BST
Brazil/Acre
Brazil/DeNoronha
Brazil/East
Brazil/West
CAT
CET
CNT
CST
CST6CDT
CTT
Canada/Atlantic
Canada/Central
Canada/East-Saskatchewan
Canada/Eastern
Canada/Mountain
Canada/Newfoundland
Canada/Pacific
Canada/Saskatchewan
Canada/Yukon
Chile/Continental
Chile/EasterIsland
Cuba
EAT
ECT
EET
EST
EST5EDT
Egypt
Eire
Etc/GMT
Etc/GMT+0
Etc/GMT+1
Etc/GMT+10
Etc/GMT+11
Etc/GMT+12
Etc/GMT+2
Etc/GMT+3
Etc/GMT+4
Etc/GMT+5
Etc/GMT+6
Etc/GMT+7
Etc/GMT+8
Etc/GMT+9
Etc/GMT-0
Etc/GMT-1
Etc/GMT-10
Etc/GMT-11
Etc/GMT-12
Etc/GMT-13
Etc/GMT-14
Etc/GMT-2
Etc/GMT-3
Etc/GMT-4
Etc/GMT-5
Etc/GMT-6
Etc/GMT-7
Etc/GMT-8
Etc/GMT-9
Etc/GMT0
Etc/Greenwich
Etc/UCT
Etc/UTC
Etc/Universal
Etc/Zulu
Europe/Amsterdam
Europe/Andorra
Europe/Astrakhan
Europe/Athens
Europe/Belfast
Europe/Belgrade
Europe/Berlin
Europe/Bratislava
Europe/Brussels
Europe/Bucharest
Europe/Budapest
Europe/Busingen
Europe/Chisinau
Europe/Copenhagen
Europe/Dublin
Europe/Gibraltar
Europe/Guernsey
Europe/Helsinki
Europe/Isle_of_Man
Europe/Istanbul
Europe/Jersey
Europe/Kaliningrad
Europe/Kiev
Europe/Kirov
Europe/Kyiv
Europe/Lisbon
Europe/Ljubljana
Europe/London
Europe/Luxembourg
Europe/Madrid
Europe/Malta
Europe/Mariehamn
Europe/Minsk
Europe/Monaco
Europe/Moscow
Europe/Nicosia
Europe/Oslo
Europe/Paris
Europe/Podgorica
Europe/Prague
Europe/Riga
Europe/Rome
Europe/Samara
Europe/San_Marino
Europe/Sarajevo
Europe/Saratov
Europe/Simferopol
Europe/Skopje
Europe/Sofia
Europe/Stockholm
Europe/Tallinn
Europe/Tirane
Europe/Tiraspol
Europe/Ulyanovsk
Europe/Uzhgorod
Europe/Vaduz
Europe/Vatican
Europe/Vienna
Europe/Vilnius
Europe/Volgograd
Europe/Warsaw
Europe/Zagreb
Europe/Zaporozhye
Europe/Zurich
Factory
GB
GB-Eire
GMT
GMT+0
GMT-0
GMT0
Greenwich
HST
Hongkong
IET
IST
Iceland
Indian/Antananarivo
Indian/Chagos
Indian/Christmas
Indian/Cocos
Indian/Comoro
Indian/Kerguelen
Indian/Mahe
Indian/Maldives
Indian/Mauritius
Indian/Mayotte
Indian/Reunion
Iran
Israel
JST
Jamaica
Japan
Kwajalein
Libya
MET
MIT
MST
MST7MDT
Mexico/BajaNorte
Mexico/BajaSur
Mexico/General
NET
NST
NZ
NZ-CHAT
Navajo
PLT
PNT
PRC
PRT
PST
PST8PDT
Pacific/Apia
Pacific/Auckland
Pacific/Bougainville
Pacific/Chatham
Pacific/Chuuk
Pacific/Easter
Pacific/Efate
Pacific/Enderbury
Pacific/Fakaofo
Pacific/Fiji
Pacific/Funafuti
Pacific/Galapagos
Pacific/Gambier
Pacific/Guadalcanal
Pacific/Guam
Pacific/Honolulu
Pacific/Johnston
Pacific/Kanton
Pacific/Kiritimati
Pacific/Kosrae
Pacific/Kwajalein
Pacific/Majuro
Pacific/Marquesas
Pacific/Midway
Pacific/Nauru
Pacific/Niue
Pacific/Norfolk
Pacific/Noumea
Pacific/Pago_Pago
Pacific/Palau
Pacific/Pitcairn
Pacific/Pohnpei
Pacific/Ponape
Pacific/Port_Moresby
Pacific/Rarotonga
Pacific/Saipan
Pacific/Samoa
Pacific/Tahiti
Pacific/Tarawa
Pacific/Tongatapu
Pacific/Truk
Pacific/Wake
Pacific/Wallis
Pacific/Yap
Poland
Portugal
ROC
ROK
SST
Singapore
SystemV/AST4
SystemV/AST4ADT
SystemV/CST6
SystemV/CST6CDT
SystemV/EST5
SystemV/EST5EDT
SystemV/HST10
SystemV/MST7
SystemV/MST7MDT
SystemV/PST8
SystemV/PST8PDT
SystemV/YST9
SystemV/YST9YDT
Turkey
UCT
US/Alaska
US/Aleutian
US/Arizona
US/Central
US/East-Indiana
US/Eastern
US/Hawaii
US/Indiana-Starke
US/Michigan
US/Mountain
US/Pacific
US/Pacific-New
US/Samoa
UTC
Universal
VST
W-SU
WET
Zulu
Using the MemoryStore
The MemoryStore provides an in-memory, table-based, data storage abstraction within the correlator. All EPL code running in the correlator in any context can access the data stored by the MemoryStore. In other words, all EPL monitors running in the correlator have access to the same data.
The MemoryStore can also store data on disk to make it persistent, and copy persistent data back into memory. However, the MemoryStore is primarily intended to provide all monitors in the correlator with in-memory access to the same data.
Data that the MemoryStore stores must be one of the following types: boolean, decimal, float, integer or string.
To use the MemoryStore, you need to add the MemoryStore bundle to your Apama project (see Adding the MemoryStore bundle to your project). This lets you create instances of MemoryStore events and then call actions on those events. Available actions include the following:
Creating stores that contain tables
Defining the schema for the rows in a table
Creating tables and associating a schema with each table
Storing, retrieving, updating, and committing rows of data
Copying tables to disk to make the data persistent
Overview of MemoryStore events
The MemoryStore defines the following events in the com.apama.memorystore package. Most of these events contain action fields that serve as the MemoryStore interface.
Storage — The event type that provides the interface for creating stores.
Store — A Store event represents a container for a uniquely named collection of tables.
Table — A Table event represents a table in a store. A table is a collection of rows. Each table has a unique name within the store. A table resides in memory and you can store it on disk if you want to.
Schema — A Schema event specifies a set of fields and the type of each field. Each Schema event represents the schema for one or more tables. Each table is associated with one schema. All rows in that table match the table’s schema.
Row — A Row event represents a row in a table. A row is an ordered and typed set of named fields that match the schema associated with the table that the row belongs to. Each row is associated with a string that acts as its key within the table. You can change the values of the fields in a row.
Iterator — Provides the ability to manipulate each row of a table in turn.
Finished — The MemoryStore enqueues a Finished event when processing of an asynchronous action is complete.
To use the MemoryStore, you need only add the MemoryStore bundle to your project. To do this, change to the directory of a project created using apama_project, and run:
apama_project add bundle "The MemoryStore"
Adding the MemoryStore bundle to your project makes the MemoryStore.mon file available to the monitors in your project.
Steps for using the MemoryStore
To use the MemoryStore, you must first add the The MemoryStore bundle to your project. After you have added the MemoryStore bundle, you write EPL that does the following:
Prepare and then open a store that will contain one or more tables.
Define the data schema for the rows that will belong to the table.
Prepare and then open a table in a store.
Get a new or existing row from the table.
Modify the row.
Commit the modified row to the table.
Repeat the three previous steps as often as needed.
Optionally, use an iterator to step through all rows in the table.
Optionally, store the in-memory table on disk.
Preparing and opening stores
The Storage event is used to prepare and open a store to which you can add tables. Storage events define actions that do the following:
Request preparation of a store.
Open a store that has been prepared.
Storage events contain no data. All Storage events are alike and exist only to provide the interface for preparing and opening stores. All actions on the Storage event are static; there is no need to create an instance of a Storage event.
If you do not require on-disk persistence, you can prepare a store in memory. If you do require on-disk persistence, you can specify the file that contains (or that you want to contain) the store. Depending on the action you call to open the store, the MemoryStore does one of the following:
Opens the store for read-write access.
Opens the store for read-only access.
Opens the store for read-write access. Create the store if it does not already exist.
Preparation of stores is asynchronous. Actions that prepare stores return an ID immediately. When the MemoryStore completes preparation it enqueues a Finished event that contains this ID. You should define an event listener for this Finished event. The Finished event indicates whether or not preparation was successful.
You can open a store only after receiving a Finished event that indicates successful preparation.
For example, the following code fragment declares a Storage type variable and a Store type variable. It then calls the prepareOrCreate() action on the Storage type variable and saves the returned ID in the Store type variable. The name of the new store is storename and the store will be made persistent by saving it in the example.dat file. Finally, this code fragment declares a Finished event variable and an event listener for a Finished event whose ID matches the ID returned by the preparation request.
using com.apama.memorystore.Storage;
using com.apama.memorystore.Store;
using com.apama.memorystore.Finished;
monitor Test {
Store store;
action onload() {
integer id := Storage.prepareOrCreate("storename", "/tmp/example.dat");
on Finished(id,*,*)as f
onStorePrepared(f);
...
}
}
After a store has been successfully prepared, you can open it:
action onStorePrepared(Finished f) {
if not f.success { log "Whoops"; die; }
store := Storage.open("storename");
All subsequent examples assume that the appropriate using statements have been added.
Any monitor instance can open a store after that store has been successfully prepared. However, monitor A has no information about whether or not monitor B has prepared a particular store.
Therefore, each monitor should prepare any store it needs, and then prepare any tables it needs within that store. There is no way to pass Store or Table events from one monitor to another. Multiple monitors can prepare and open the same store or table at the same time.
There are several different actions available for preparing a store:
Storage.prepareInMemory(string name) returns integer prepares an in-memory store with the name you specify. All tables are empty when prepared for the first time. Persistence requests are ignored and immediately return a successful Finished event.
Storage.prepare(string name, string filename) returns integer does the same thing as Storage.prepareInMemory and it also associates that store with the database file you specify. If there is data in the database file the MemoryStore loads the store with the data from the file when you prepare a table. Persistence requests write changes back to the file. The specified file must exist.
Storage.prepareOrCreate(string name, string filename) returns integer does the same thing as Storage.prepare() except that it creates the file if it does not already exist.
Storage.prepareReadOnly(string name, string filename) returns integer does the same thing as Storage.prepare and it also opens for read-only access the database file you specify. The MemoryStore will load the store with data from the file when you prepare the table. Persistence requests are refused and return a failure Finished event
Suppose a monitor instance calls one of the Storage.prepare() actions and the action is successful. Now suppose another monitor instance calls the same Storage.prepare() variant with the same table name and, if applicable, the same filename, as the previously successful call. The second call does nothing and indicates success immediately. However, if a monitor instance makes a Storage.prepare() call and specifies the same table name as was specified in a previously successful prepare() call, that call fails immediately if at least one of the following is different from the successful call:
The variant of the prepare() action called
The specified file name or store name (if applicable)
For example, suppose a monitor made the following successful call:
Storage.prepare("foo", "/tmp/foo.dat")
After this call, the only prepare call that can successfully prepare the same table is
If a monitor makes a call to prepare() that matches a prepare action that is in progress, the result is the same as the result of the prepare that is in progress.
Description of row structures
Schemas
A schema consists of an ordered list of the names and types of fields that define the structure of a row. For example, the following schema consists of one field whose name is times_run and whose type is integer:
A valid schema can be created from an event type using schemaFromAny(event). Types that are not supported in the event are converted to string types.
The schema does not include the row’s key. The key is always a string and it does not have a name. Each row in a table is associated with a key that is unique within the table. The key provides a handle for obtaining a particular row. The row does not contain the key.
Two schemas match when they list the same set of field names and types in the same order.
Tables
Table events define actions that do the following:
Retrieve a row by key. The returned object is a Row event.
Remove a row by key
Remove all rows
Obtain a sequence of keys for all rows in the table
Obtain an iterator to iterate over the rows in the table
Determine if any row in the table has a particular key
Store on disk the changes to the in-memory table
Modify a row by key
Modify all rows
Obtain the position in a schema of a specified field.
Obtain the name of the table
Obtain the name of the store that contains the table
Retrieval of a row from a table by key always succeeds. If the row already exists, the MemoryStore returns a Row event that provides a local copy of the row. The content of this Row event does not change if another user modifies the in-memory version of the row in the table. If the row does not already exist, the MemoryStore populates a Row event with default values and returns that with field values as follows:
boolean types are false
decimal types are 0.0d
float types are 0.0
integer types are 0
string types are empty ("")
Rows
Row events define actions that do the following:
Getters and setters. These actions modify only the local copy (your Row event) and not the in-memory version of the row. The in-memory version of the row is available to all monitors. If another user of the table retrieves the same row, that user receives a Row event that contains a copy of the in-memory version of the row; that user does not receive a copy of your modified, local version of the row:
Get and set boolean, decimal, float, integer, and string fields by name.
Generic get and set field by name actions which use the any type. These throw an exception if the underlying types do not match the expected field type.
Get and set all fields. These expect a prototype event whose fields and types match that of the table schema. An exception is thrown if the schemas do not match.
Commit a modified Row event. That is, you modify your local Row event, and commit the changes, which updates the shared row in the table. This makes the update available to all monitors.
Get the value of a row’s key.
Determine whether a row was present in the table when the local copy was provided.
Obtain the name of the table the row is in.
Obtain the name of the store the row’s table is in.
The Row.commit() action modifies only the in-memory copy of the row so it is a synchronous and non-blocking operation. If any other user of the table modifies the in-memory row between the time you obtain a Row event that represents that row and the time you try to commit your changes to your Row event, the Row.commit() action fails and the monitor instance that called Row.commit() dies. Therefore, if you are sharing the table with other users, you should call Row.tryCommit() instead of Row.commit(). If it fails you must retry the commit operation by retrieving the row again (that is, obtaining a new Row event that contains the latest content of the in-memory row), reapplying the changes, and then calling the Row.tryCommit() action. This ensures that you always make changes that are consistent and atomic within the shared version of the row.
However, it is not possible to make atomicity guarantees across rows or tables.
Preparing and opening tables
After you have an open store, you can add one or more tables to that store. You call actions on Store events to create tables. Store events define actions that do the following:
Prepare a table. You specify a table name and a schema or supply an event or type name to use as the name and schema. This call is asynchronous. The MemoryStore sends a Finished event that indicates success or failure. If the table does not exist, the MemoryStore creates an empty table.
Open a table that has been prepared
Store on disk the in-memory changes to tables.
If the store that contains the table is persistent (i.e. on-disk) and the table exists on disk then the on-disk schema must match the schema that you specify when you call the action to prepare the table. If the schemas do not match, the Finished event that the MemoryStore sends includes an error message.
If a monitor instance calls Store.prepare() with the same table name and schema as those of a previously successful Store.prepare() call, the call does nothing and indicates success immediately. If a monitor instance calls Store.prepare() and specifies the same table name but the schema does not exactly match, that call fails immediately. If a monitor makes a call to Store.prepare() that matches a preparation that is in progress, the result is the same as the result of the preparation that is in progress.
If the table you want to prepare is persistent and it has not yet been loaded into memory then the MemoryStore loads the table’s on-disk data into memory in its entirety. The MemoryStore sends the Finished event when loading the table is complete.
To use a table that is in memory, you must retrieve a handle to it from the store that contains it. Obtaining a handle to a prepared (loaded) table is a synchronous action that completes immediately and does not block. The calling monitor instance dies if you try to obtain a handle to a table that is not prepared or that is in the process of being prepared.
For example:
integer id := store.prepare("tablename", schema);
on Finished(id,*,*) as f onTablePrepared(f);
action onTablePrepared(Finished f) {
if not f.success { log "Whoops"; die; }
Table tbl := store.open("tablename");
Info
The term “table” is a reserved keyword. Consequently, you should not use “table” as a variable name.
Preparation of a table can fail for a number of reasons including, but not limited to, the following:
You call prepare() on an existing table and the schema of that table and the schema specified in the prepare() call do not match.
You call prepare() on a table that does not exist and the store has been opened read-only.
You call prepare() on a table that does not exist in a persistent store and the attempt to create a new table in the persistent store fails, perhaps because the disk is full.
The on-disk version of the table is corrupt in some way.
Using transactions to manipulate rows
In a monitor, any changes you make to Row events are local until you commit those changes. In other words, any changes you make actually modify the Row events that represent the actual rows in the table. After you commit the changes you have made to your Row events, the updated rows are available to all monitors in the correlator.
Info
When you modify a Row event and you want to update the actual row with your changes, you must commit your changes.
The Row event defines the following actions for committing changes:
Row.commit() — Tries to commit changes to Row events to the table. If nothing else modified the row in the table since you obtained the Row event that represents that row, the MemoryStore commits the changes and returns. The update is available to all monitors. If the row in the table has been modified, an exception is thrown, leaving the table unchanged.
Row.tryCommit() — Behaves like commit() except that it does not throw an exception upon failure. If the row in the table has been modified, this action returns false and leaves the table unchanged. If this action is successful, it returns true.
Row.tryCommitOrUpdate() — Behaves like tryCommit() except that when it returns false, it also updates your local Row event to reflect the current state of the actual row in the table. In other words, if the row in the table has been modified, this action does the following:
Leaves the actual row in the table unchanged.
Updates the local Row event that represents this row to reflect the current state of the table. Any local, uncommitted modifications are lost.
Returns false.
Row.forceCommit() — Commits the local Row event to the table even if the row in the table has been modified after you obtained the Row event.
Determining which commit action to call
If you are certain that you are the only user of a table and if it is okay for your monitor instance to be killed if you are wrong, you can use commit().
If you want to use a simple loop like the one below, or if you intend to give up if your attempt to commit fails, then use tryCommit().
boolean done := false;
while not done {
Row row := tbl.get("foo");
row.setInteger("a",123);
done := row.tryCommit();
}
However, the loop above calls tbl.get() every time around. If you think there might be a high collision rate, it is worth optimizing to the following, more efficient design:
Row row := tbl.get("foo");
boolean done := false;
while not done {
row.setInteger("a",123);
done := row.tryCommit();
if not done { row.update(); }
}
The tryCommitOrUpdate() action makes the example above a little simpler and considerably more efficient:
Row row := tbl.get("foo");
boolean done := false;
while not done {
row.setInteger("a",123);
done := row.tryCommitOrUpdate();
}
Alternatively, there is a packaged form of that loop that you might find more convenient:
The above example is equivalent to the previous one, both in behavior and performance. Which to use is a matter of context, style and personal preference.
If you want to simply overwrite the whole Row rather than updating the row based on the current value, then using forceCommit() would be more appropriate. It commits the local Row content to the table even if it was modified after you obtained the Row event:
To create a row in a table, call the get() or add() action on the table to which you want to add the row. The action declaration for the get() action is as follows:
action get(string key) returns Row
The Table.get() action returns a Row event that represents the row in the table that has the specified key. If there is no row with the specified key, this action returns a Row event that represents a row that contains default values. A call to the Row.inTable() action returns false. For example:
boolean done := false;
integer n := -1;
while not done {
Row row := tbl.get("example-row");
n := row.getInteger("times_run");
row.setInteger("times_run", n+1);
done := row.tryCommit();
}
send Result(
"This example has been run " +n.toString() +" time(s) before")
to "output";
The add() action does the same as the get() action, except that it does not check if the row that is to be added already exists in the table until commit() is called and it therefore never throws an exception. If you are sure that the row does not yet exist, you can use add() as this is faster than get().
To remove a row from a table, call the Table.remove() action on the table that contains the row. The action declaration is as follows:
action remove(string key)
The Table.remove() action removes the row with the specified key from the table. If the row does not exist, this action does nothing.
It is also possible to remove a row transactionally, by calling Table.get() and then Row.remove() and Row.commit(). This strategy lets you check the row’s state before removal. The Row.commit() action fails if the shared, in-memory row has been updated since the Table.get() action.
In some circumstances, using Row.remove() is essential to guarantee correctness. For example, when decrementing a usage counter in the row and removing the row when the count reaches zero. Otherwise, another correlator context might re-increment the count between it reaching zero and the row being removed.
Iterating over the rows in a table
Iterators have operations to step through the table and determine when the end has been reached. Provided an iterator is not at the table’s end, the key it is at can be obtained.
Iterator events define actions that do the following:
Step through the rows in a table.
Determine when the last row has been reached.
Obtain the key of the row that the iterator is at. The iterator must not be at the end of the table for this action to be successful.
Obtain a Row event to represent the row that the iterator is at.
The following sample code reads table content:
Iterator i := tbl.begin();
while not i.done() {
Row row := i.getRow();
if row.inTable() {
// Put code here to read the row in the way you want.
}
i.step();
}
The following sample code modifies table content:
Iterator i := tbl.begin();
while not i.done() {
Row row := i.getRow();
boolean done := false;
while row.inTable() and not done {
// Put code here to modify the row in the way you want.
done := row.tryCommitOrUpdate();
}
i.step();
}
Iterating through a table is always safe, regardless of what other threads are doing. However, if another context adds or removes a row while you are iterating in your context, it is undefined whether your iterator will see that row.
Furthermore, it is possible for another context to remove a row while your iterator is pointing at it. If this happens, a subsequent Iterator.getRow() returns a Row event that represents a row for which Row.inTable() is false.
If an EPL action loops, the correlator cannot perform garbage collection within that loop. (See Optimizing EPL programs.) Performing intricate manipulations on many rows of a large table could therefore create so many transitory objects that the correlator runs out of memory. If this becomes a problem, you can divide very large tasks into smaller pieces, each of which is performed in response to a routed event. This gives the correlator an opportunity to collect garbage between delivering successive events.
Requesting persistence
After changing a MemoryStore table, you can call the Table.persist() action to store the changes on disk. Note that you can call persist() only on tables in an on-disk store; you cannot call persist() on tables in in-memory stores.Updating a table on disk is an asynchronous action. The MemoryStore sends a Finished event to indicate success or failure of this action. The persistent form of the database that contains the tables is transactional. Consequently, if there is a hardware failure either all of the grouped changes are made or none of them are made.
Following is an example of storing a table on disk:
integer id := tbl.persist();
on Finished(id,*,*) as f onPersisted(f);
action onPersisted(Finished f) {
if not f.success { log "Whoops"; die; }
log "All OK" at INFO;
When you update a table, the MemoryStore copies only the changes to the on-disk table.
To improve performance, the MemoryStore might group persistence requests from multiple users of a particular store. This means that calling persist() many times in rapid succession is efficient, but this does not affect correctness. If the MemoryStore indicates success, you can be certain that the state at the time of the persist() call (or at the time of some later persist() call) is on disk.
You can call the Store.backup() action to backup the on-disk form of a store while it is open for use by the correlator. This is an asynchronous action that immediately returns an ID. The MemoryStore sends a Finished event that contains this ID to indicate success or failure of this action. Be sure to define an event listener for this event.
Monitoring status for the MemoryStore
The MemoryStore provides status values via the user status mechanism. It provides the following metrics:
memStoreCommitTimeEwmaLongMillis — A longer-term exponentially-weighted moving average (EWMA) of the time in milliseconds taken to commit to the MemoryStore (in-memory store and on-disk persistence).
memStoreCommitTimeEwmaShortMillis — A quickly-evolving exponentially-weighted moving average (EWMA) of the time in milliseconds taken to commit to the MemoryStore (in-memory store and on-disk persistence).
memStoreCommitTimeMaxInLastHour — The maximum commit latency in milliseconds since the start of the last 1 hour measurement period for the MemoryStore (in-memory store and on-disk persistence).
The above MaxInLastHour values are updated if either of the following conditions is true:
The latency of current commit transaction is greater than the existing maximum.
The existing maximum value was set more than 1 hour ago.
At any one time, only one correlator should be accessing a particular MemoryStore disk file.
To minimize the risk of data corruption in the event of a system failure, keep MemoryStore files on your local disk and not on a remote file server.
Do not create hard or symbolic links to MemoryStore files. Linking to the directory that contains a MemoryStore file is not a problem.
Using the Management interface
The Management interface defines actions that let you do the following:
Obtain information about the correlator
Connect the correlator to another component
Control logging
Manage user-defined status values
Actions in the Management interface are defined on several event types, which are documented in the API reference for EPL (ApamaDoc).
To use the Management interface, add the Correlator Management EPL bundle to your Apama project. See Creating and managing an Apama project from the command line. Alternatively, you can directly use the EPL interfaces provided in APAMA_HOME\monitors\Management.mon.
Obtaining information about the correlator
The Management interface provides actions for obtaining information about the correlator that the Management interface is being used in. These actions are defined in the com.apama.correlator.Component event.
There are also options of the engine_management tool that you can specify (see also Shutting down and managing components) to retrieve the same information from outside the correlator,
The correlator also logs all of these values to its log file at startup.
Connecting the correlator to another component
The com.apama.correlator.Component event provides actions that allows EPL to create connections to another component, in the same way as the engine_connect tool (see Configuring pipelining with engine_connect).
Controlling logging
You can configure logging using the Management interface. The com.apama.correlator.Logging event provides actions such as setApplicationLogFile, setLogFile and setApplicationLogLevel. These actions are the equivalent of using the engine_management options to configure logging (see also Shutting down and managing components).
The rotateLogs() action, which is also defined in the com.apama.correlator.Logging event, is used for closing the log files in use, opening new log files, and then sending messages to the new log files. This action applies to:
the main correlator log file,
the correlator input log file if you are using one, and
You can write an EPL monitor that triggers log rotation on a schedule. For example, the code below rotates logs every 24 hours at midnight:
using com.apama.correlator.Logging;
monitor Rotator {
action onload() {
on all at (0, 0, *, *, *) {
Logging.rotateLogs();
}
}
}
Managing user-defined status values
The Management interface provides actions for managing (set and return) the user-defined status values. These actions are defined in the com.apama.correlator.Component event and in the com.apama.correlator.EngineStatus event.
Status keys will have leading and trailing whitespace stripped. Keys may not be empty.
A user-defined status will only be changed if the new value differs from the current value when set using setUserStatus.
Note that the correlator status statements that appear in the log files will not have the user-defined status values, and will remain unaffected.
You can use the JSON plug-in to convert EPL objects to JSON strings, and vice versa. When converting from JSON strings, JSON objects are converted to dictionary<any, any> and JSON lists are converted to sequence<any>.
Equivalent functionality is available with the JSON codec connectivity plug-in. Use the JSON codec if the JSON is coming in as an entire message of a connectivity chain. Using a codec helps to separate the application logic from the connection over which the data is being sent, and allows the use of mapping codecs if needed. See The JSON codec connectivity plug-in for detailed information.
Use the JSON EPL plug-in if only one field of an event is JSON, or if the events are going to or are coming from a connection that is not using a connectivity chain.
Using the Base64 plug-in
You can use the Base64 plug-in to encode EPL strings as Base64 or to decode Base64 strings to EPL. When decoding Base64, the encoded data must be in UTF-8 character encoding.
Similar functionality is available with the Base64 codec connectivity plug-in. That allows arbitrary binary data from a transport to be stored in EPL in Base64 format. See The Base64 codec connectivity plug-in for detailed information.
Interfacing with user-defined EPL plug-ins
Although EPL is very powerful and enables complex applications, it is foreseeable that some applications might require additional specialized operations. For example, an application might need to carry out advanced arithmetic operations that are not provided in EPL. You can address this situation by writing custom EPL plug-ins using Apama’s C++, Java, or Python plug-in development kits. For detailed information on developing your custom EPL plug-in, see Developing EPL plug-ins.
Info
The correlator’s plug-in interface is versioned. If you upgrade the major or minor version of Apama, you may need to recompile your plug-ins against the new libraries to be compatible with the newer version of the correlator.
In order to access a function implemented in an EPL plug-in, you must import the plug-in. If the plug-in is written in Java, you must first inject the jar file containing the plug-in. For any plug-in, an EPL monitor or event must use the import statement to load it:
import"apama_math" as math;
For a Java plug-in, this will load the plug-in from the injected jar file. For a C++ plug-in, this will look for the Apama plug-in file libapama_math.so. This must be located on the standard library path (by default, in $APAMA_WORK/lib). It will then map it to the internal alias math.
Info
Insert the import statement in the monitor that uses the plug-in functions.
If the apama_math plug-in defines a method called cos that takes a single floating point value as an argument and returns a float value, this would be called from EPL as follows:
float a, b;
//... some other EPL
a := math.cos(b);
Standard float, integer and boolean types are passed by-value to external functions while string and sequence types (which map to native arrays in the plug-in) are passed by-reference. In addition, the chunk type can be used to “pass-through” data returned from one function call to another plug-in function, as shown in About the chunk type.
About the chunk type
The chunk type allows data to be referenced from EPL that has no equivalent EPL type. It is not possible to perform operations on data of type chunk from EPL directly; the chunk type exists purely to allow data output by one external library function to pass through to another function. Apama does not modify the internal structure of chunk values in any way. As long as a receiving function expects the same type as that output by the original function, any complex data structure can be passed around using this mechanism.
To use chunks with plug-ins, you must first declare a variable of type chunk. You can then assign the chunk to the return value of an external function or use the chunk as the value of the out parameter in the function call.
The following example illustrates this. The functions setRealVal and setImagVal set internal values of the chunk, while the functions getRealVal and getImagVal retrieve values from the chunk. The function addComplexNumber adds the second chunk to the first chunk. The source code for complex_plugin is in the correlator_plugin/cpp directory of the Apama Samples repository.
Monitor ComplexPlugin {
import"complex_plugin" as plugin;
action onload {
// Creates a first complex number chunk
chunk complexNumberFull := plugin.makeComplexNumberFull(4.0, -1.4);
printComplexNumber(complexNumberFull);
// Creates a second complex number chunk
chunk complexNumberEmpty := plugin.makeComplexNumberEmpty();
// Get the real and imaginary values of the number
plugin.setRealVal(complexNumberEmpty, 2.0);
plugin.setImagVal(complexNumberEmpty, 3.0);
printComplexNumber(complexNumberEmpty);
// Add the second complex number to the first complex number
plugin.addComplexNumber(complexNumberFull, complexNumberEmpty);
printComplexNumber(complexNumberFull);
}
action printComplexNumber(chunk complexNumber)
{
float real := plugin.getRealVal(complexNumber);
float imag := plugin.getImagVal(complexNumber);
string sign := "";
if(imag >= 0.0) {
sign := "+";
}
log real.toString() + sign + imag.toString() + "i";
}
}
Although the chunk type was designed to support unknown data types, it is also a useful mechanism to improve performance. Where data returned by external plug-in functions does not need to be accessed from EPL, using a chunk can cut down on unnecessary type conversion. For example, suppose the output of a localtime() method is a 9-element array of type float. While you could declare this output to be of type sequence<float>, there is no need to do so because the EPL never accesses the value. Consequently, you can declare the output to be of type chunk and avoid an unnecessary conversion from native array to EPL sequence and back again.
