Writing EPL plug-ins in Python

Creating a plug-in using Python

The APIs for writing plug-ins in Python are documented in the API reference for Python. The relevant classes are in the apama.eplplugin module.

To create a plug-in for EPL in Python, you have to create a class which inherits from apama.eplplugin.EPLPluginBase. Your class must provide a one-argument constructor and pass the argument verbatim to the EPLPluginBase constructor. For example:

import apama.eplplugin

class MyPluginClass(apama.eplplugin.EPLPluginBase):
   def __init__(self, init):
      super(CountPlugin, self).__init__(init)
...

The base class provides two member functions to derived classes:

A single instance of your class is created for each time it is listed in the configuration file. Functions which you want to expose to EPL are member functions on that instance. To export a function to EPL, you need to declare a member function on the class and decorate it to indicate the name and signature of the function in EPL using the EPLAction decorator:

@EPLAction("getCounter", "action<string> returns integer")
def lookupCounter(name):
   return counters[name].value()

You configure the plug-ins in the YAML configuration file for the correlator, in the eplPlugins stanza:

eplPlugins:
  counterPlugin:
    pythonFile: ${PARENT_DIR}/CounterPlugin.py
    class: CountPlugin

You load the plug-ins into EPL by using the import statement in the monitor or event which wants to use the plug-in:

import "counterPlugin" as plugin;

Complete simple example

This example implements a plug-in which simply keeps a single global count.

from apama.eplplugin import EPLPluginBase,EPLAction

class CountPlugin(EPLPluginBase):
  def __init__(self, init):
    super(CountPlugin, self).__init__(init)
    self.count = 0
  @EPLAction("action<integer>", "increment") # override name
  def incrementCount(self, number):
    self.count = self.count + number
  @EPLAction("action<> returns integer")
  def getCount(self):
    return self.count

eplPlugins:
  counterPlugin:
    pythonFile: ${PARENT_DIR}/CounterPlugin.py
    class: CountPlugin

event A {
  integer increment;
}
monitor CounterPlugin {
  import "counterPlugin" as counter;
  action onload() {
    monitor.subscribe("CounterChannel");
    on all A() as a {
      counter.increment(a.increment);
      print "Current count: "+counter.getCount().toString();
    }
    send A(1) to "CounterChannel";
  }
}

Method signatures and types

Only methods which take arguments and return values which can be converted into EPL types can be used in an EPL plug-in written in Python. For each action, you must provide the EPL signature for the method to the EPLAction decorator. Typing is not strictly enforced by Python, but is enforced by EPL when being used as a plug-in. EPL method signatures take the following forms:

action<integer>
action<sequence<string>, integer>
action<> returns integer
action<dictionary<string, string>, string> returns string

EPL to Python type conversion

Methods using args/kwargs

You can either explicitly specify all the arguments that your plug-in methods take or you can rely on *args-type argument handling.

You can pass any number of EPL arguments into a Python function expecting *args. This functions in the same way as passing the arguments from within Python.

Expanding a sequence as function parameters is not supported. In this case, *args would contain a single parameter, of type list. It is possible to achieve this using a wrapper function from within Python. For example:

def funcThatUsesArgs(*args):
  ...

@EPLAction("action<sequence<any>>")
def foo(d):
    funcThatUsesArgs(*d)

Apama does not invoke plug-in methods with argument names, so **kwargs patterns will not work. However, it is possible to use Python functions expecting **kwargs by using a wrapper function in the same way as with *args. For example:

def funcThatUsesKwargs(**kwargs):
  ...

@EPLAction("action<dictionary<string, any>>")
def foo(d):
    funcThatUsesKwargs(**d)

Exceptions

Plug-ins can raise exceptions from plug-in methods. They must be derived from exceptions.Exception as normal.

If a plug-in throws an exception, this will be turned into an exception in EPL which may be caught with the EPL try... catch statement. See also The try… catch statement.

If you do not catch the exception, then the calling monitor instance will be terminated with the message in the exception thrown from the plug-in.

Writing to the correlator log file

The EPLPluginBase class provides a member function getLogger to plug-ins. This can be used to write messages to the correlator’s log file.

Log messages are prefixed with the string plugins.*PluginName*, which is also the category that can be used to control the log level for this plug-in via the correlatorLogging section in the YAML configuration file (see Setting correlator and plug-in log files and log levels in a YAML configuration file).

def logMessage(msg):
   getLogger().info("Message is: %s", msg);

Documentation about the available functions and log levels can be found in the API reference for Python.

Sending events

Method calls on plug-ins are synchronous. Generally, they should be written not to take too long or hold up processing in the correlator. One technique to enable asynchronous behavior in the plug-in is to interact with EPL by sending events which can be handled asynchronously, potentially from a background thread which is processing events as well as from methods themselves.

The apama.eplplugin.Correlator class contains several methods for sending events into the correlator. You can send events either as the string representation of the event in Apama’s internal string format (for example, MyEvent(1.3, true)) or as a dictionary either with the event type specified explicitly or as an apama.eplplugin.Event type. In the dictionary case, the keys are strings corresponding to field names in the event, and the values are the value for those fields in the appropriate type/format for the type of the field.

You can select the destination of the event in two ways:

• Named channel. The preferred method is to deliver the event to a specific named channel. This will go to everything which has subscribed to that named channel. Subscribers can either be contexts, external receivers, connectivity chains or other plug-ins which are subscribed to receive events on that channel.

  Send as string:

  Correlator.sendTo("channelName", "MyEvent(42)")
  

  Send as object:

  event = {}
  event["number"] = 42
  Correlator.sendTo("channelName", event, type="MyEvent")
  

• Context. You can deliver the event to a specific context using an apama.eplplugin.Context object. Contexts can either be passed into a plug-in from an EPL context object, or they can be obtained with the apama.eplplugin.Correlator.getCurrentContext() method.

  Send as string:

  Correlator.sendTo(Correlator.getCurrentContext(), "MyEvent(42)")
  

  Send as object:

  apama.eplplugin.Event event;
  event.fields["number"] = 42
  event.type = "MyEvent"
  Correlator.sendTo(contextObject, event)
  

Using Python plug-ins

After you have created an EPL plug-in in Python, you must configure it in a Python-enabled correlator so that it is available for use in your Apama applications. Applications that use the plug-in also need to import the plug-in by name.

Enabling Python support in the correlator

To enable Python support in the correlator, you must use the --python command-line option of the correlator executable. See also Starting the correlator.

You can also enable Python support using the YAML configuration file for the correlator:

correlator:
  pythonSupport: true

If you are using a standard Apama installation, a copy of Python is provided in your installation. This Python will be used by default. If you are using the core installer, or wish to use a different version of Python, then you will need to override the location of your Python installation. You can do this by setting the AP_PYTHONHOME environment variable.

Adding a Python plug-in to the correlator

EPL plug-ins written in Python are made available to EPL via the YAML configuration file for the correlator (see also Configuring the correlator).

To configure a specific Python plug-in once you have enabled Python support, you need to add an eplPlugins section to the configuration file:

eplPlugins:
   myPluginName:
      pythonFile: ${PARENT_DIR}/plugin.py
      class: PluginClass
      pythonPath:
         - ${PARENT_DIR}/dependencies
      config:
          key: value

The plug-in name is an arbitrary string which will be used to refer to the plug-in from EPL. The following configuration options are available for each plug-in:

You can create multiple instances of the same plug-in with different names.

Importing a Python plug-in to EPL

Once a Python plug-in has been configured, it is available for import using the plug-in name defined in the configuration file. The correlator will make available all methods decorated with the EPLAction decorator to be called directly from EPL. For example, the following code imports a plug-in named TestPlugin and calls its dosomething method:

monitor m {
   import "TestPlugin" as test;
   action onload()
   {
       test.dosomething();
   }
}

If the plug-in has been incorrectly configured, the correlator will try to load the plug-in as a C++ plug-in and may give an error such as Error opening plug-in library libTestPlugin.so: libTestPlugin.so: cannot open shared object file: No such file or directory. If this happens and you were trying to load a plug-in written in Python, then check the name in your configuration file and make sure that it matches the name you are trying to import.

Python plug-ins and correlator persistence

Since Python plug-ins provide no way to persist data stored inside the plug-in, or in chunks from a Python plug-in, it is not permitted to import a Python plug-in from a persistent monitor or to use an event which imports a Python plug-in from a persistent monitor. You can use Python plug-ins from non-persistent monitors in a persistent correlator.

Installing Python modules

The standard (full) installation of Apama includes a copy of Python which is used by default in the Apama environment. It provides a pip (pip3 on Linux) wrapper to the Python interpreter that is shipped with Apama.

To install a Python module, run the following command in the Apama Command Prompt or apama_env wrapper (see also Setting up the environment using the Apama Command Prompt).

Windows:

pip install module_name

Linux:

pip3 install module_name

Sample plug-ins written in Python

Apama provides sample EPL plug-ins written in Python, located in the samples\correlator_plugin\python directory of your Apama installation.