With the Apama Data Player you can play back previously saved event data as you develop your application. During playback, you can analyze the behavior of your application. Or, if you modify the saved event data, you can analyze how your application performs with the altered data. Apama plays back event data that has been stored in standard data formats.
The Data Player relies on Apama Database Connector (ADBC) adapters that are specific to standard ODBC and JDBC database formats as well as the proprietary Apama Sim format. These adapters run in the Apama Integration Application Framework (IAF), which connects the data sources to the correlator.
The Apama Data Player consists of both the Query Editor and the Data Player Control.
In addition to the normal operations for running an application in Apama Plugin for Eclipse, in order to play back event data in the Data Player, you need to perform a few other steps. Broadly, these steps are:
Configure the Apama project to use the appropriate ADBC adapter for the data source and database and specify the event types that will be played back along with the appropriate IAF mapping.
Launch the project so it can establish a connection to the data source.
Specify a Data Player playback query to determine what data from the database you want to play back.
Use the Data Player control to specify the following: how fast you want to play back the data; over what time range; and what throttling period to use.
Run the playback session.
If you want to use the Apama Data Player in your project, you need to add and configure the Apama database connector adapter that is appropriate to the data source used by the project: ODBC, JDBC, or Sim.
To add and configure an ADBC adapter
There are two ways of adding an ADBC adapter to a project.
If you are creating a new Apama project:
Select File > Project > New > Apama.
Give the new project a name, and click Next. If you are adding an ADBC adapter to an existing project:
In the Project Explorer, right-click the Connectivity and Adapters node and select Add Connectivity and Adapters.
In the Connectivity and Adapters dialog, create a new name for the adapter instance or accept the default instance name. Apama Plugin for Eclipse prevents you from using a name that is already in use.
Select the ADBC adapter bundle that is appropriate to the kind of data source your application will use. Click OK.
When you add a data source-specific adapter, the ADBC Adapter (Common Apama database connector adapter) bundle will be added to the project automatically.
Configure the instance of the new adapter as described in Configuring an ADBC adapter.
To create queries most efficiently, the project needs to be running so that you can see what data sources, databases, and existing queries are available.
To create a new run configuration
Select the project for which to create the run configuration.
In the Apama Plugin for Eclipse menu bar, select Run > Run Configuration.
In the Run Configuration wizard, select Apama Application and click the New launch configuration button
(
).
On the Apama Project tab, specify the following:
Specify the Name of the run configuration.
Select or accept the Project.
Select the Enable DataPlayer check box. If this box is not checked, the Query Editor and Data Player control are disabled.
Select Generate time events from data if you want the correlator to use external time events (starting the correlator with the -Xclock option). The Generate time events from data check box is available only if you checked the Enable DataPlayer check box. For details about the format of correlator &TIME events, see Externally generating events that keep time (TIME events).
When Generate time events from data is checked, the time field specified in the Data Player playback query must be a float value that represents a number of a seconds since the epoch. The data player transforms these values into &TIME events.
When Generate time events from data is disabled (unchecked), the Data Player’s Speed and Play to controls are disabled.
Click Run to save and launch the project. Or click Apply to save the configuration without running the project.
You create and modify Data Player queries with the Data Player Query Editor. The information for the Data Player queries is stored in the project’s dataplayer_queries.xml file.
To create or modify a Data Player query
In the Project Explorer view, expand the project and then expand the config folder. Double-click the dataplayer_queries.xml file to open the Data Player Query Editor. If the project is running, you will be able to make selections from the Datasource, Database, and Query drop-down lists. If the project is not running, most of the controls are disabled.
If the project is not running, the Data Player Query Editor will report that it is offline. In this mode, you cannot select data sources or databases and all the controls are disabled.
If you are creating a new Data Player query, you can also right-click the name of the project and select New > Data Player Query from the pop-up menu. Use this method if you are working on an imported project that does not yet contain a dataplayer_queries.xml file. In the Configuration field, enter the name of the query and click Finish. The query will be added to the project. You can then edit the query in the Data Player editor.
Click the Add Query button (
). The New Query dialog appears.
Provide a name for the new Data Player query and click OK.
In the General Settings section, specify the following properties for the Data Player query:
Description — Provide a description of the Data Player query.
Datasource — Select an available data source from the drop-down list.
Database — Select an available database from the drop-down list.
Return type — The choices are Native or Wrapped. When Native is selected, each matching event will be sent as-is to the correlator. When Wrapped is selected, each matching event will be “wrapped” in a container event. The container event will be named using the event name. Event wrapping allows events to be sent to the correlator without triggering application listeners. A separate user monitor can listen for wrapped events, modify the contained event, and reroute it such that application listeners can match on it.
For example, you want to map the event Tick from the namespace com.apama.demo via the adapter, and your definition of Tick is as follows:
event Tick{
string symbol;
integer qty;
float price;
}
In order to define the event return type as Wrapped - Return events in container event, you have to define an event in the default package such as the following:
event HistoricalTick{
integer queryId;
string serviceId;
float timestamp;
com.apama.demo.Tick tick;
}
The Tick event will then be wrapped in a HistoricalTick event.
Flush between events — Controls whether the correlator’s queues are flushed between every event from the database or not. This is important for correct behavior in multi-threaded or multi-context applications. The recommended value is 2 (default), which flushes the correlator’s queues twice, but may need to be higher if the application contains a long sequence of contexts that send events from one to the next. This will reduce the maximum speed that events can be processed, so for single-context applications, you may want to set this to 0 (or equivalently, leave blank).
In the Query section, the Use query template check box specifies whether you want to use a Data Player query template (checked) or a raw query (unchecked).
If you are using a query template:
Select a Data Player query template from the drop-down list. The choices are the canned Data Player queries supplied with the Apama installation: findAll, findEarliest, findLatest, and getCount. You can add your own Data Player query templates; see Creating Data Player query templates.
If the Data Player query template uses replaceable parameters, they will be displayed in the Name and Value table in the Query section of the editor.
Double-click a cell in the Value column and type the entry to use for the Data Player query, for example, the name of a database table or column.
If you are using a raw query, in the Query String text area, specify the statement to execute as follows:
For ODBC and JDBC data sources, use SQL syntax.
For an Apama Sim data source, use the following keywords (keywords are case insensitive):
TIME with =, <, >, >=, and <= comparators and the time. Use one or two TIME statements. If two TIME statements are used, only data that matches both statements will be returned.
INCLUDE plus the event type to retrieve.
EXCLUDE plus the event type to not retrieve.
Examples of this syntax are:
TIME=12345
TIME=12345;INCLUDE=(com.apama.something)
TIME=12345;INCLUDE=(com.apama.something);EXCLUDE=(com.apama.something.abc)
TIME>12345;TIME<20000
In the Event Type field, specify unmapped-sql when the database table contains an eventString column that contains Apama events with string value. However, if the database table contains individual columns that are mapped to event fields in the ADBC adapter, then specify the name of the specific event in the Event Type field.
The value that you specify in the Event Type field becomes the value for _ADBCType in the ADBC adapter.
Given the above example of the Tick event that has been mapped in the adapter, you would have to specify com.apama.demo.Tick as the event type.
In the Time Column field, specify the time column in your table. For example, if your table contains a TransactionTime column, then you could specify TransactionTime in the Time Column field. Ensure that the value in the time column is a float value that represents a number of seconds since the epoch.
In the Advanced Settings section, specify any Extra parameters.
Right-click in the Name column and select Add from the pop-up menu (or double-click in the Name column).
Specify the name of the extra parameter.
In the Value column, specify the value of the extra parameter.
Save the Data Player query.
When you save the Data Player query, the Data Player query name is added to the Data Player Control view’s drop-down query list.
The Query Editor tool bar contains the following buttons:
) Configure credentials — This displays the Credential for selected data source dialog where you add a user name and password for the current Data Player query if they are required for accessing the data source and select the scope of the credentials from a drop-down list. You can specify if the credentials apply globally, to a specific data source, or to a specific Data Player query. Credentials entered here are shared by the Data Player Query Editor and the Data Player.
) Clone query — Select this if you want to make a copy of the current Data Player query. The Clone Query dialog prompts you for the name of the copy. You can then edit the specifications for the Data Player query.) Add query — Select this if you want to add a new Data Player query. The New Query dialog prompts you for the name of the new Data Player query.
) Rename query — Select this if you want to rename the Data Player query. The Rename Query dialog prompts for the new name of the Data Player query.
) Delete query — Select this if you want to delete the Data Player query. The Delete Query dialog prompts you to confirm that this is what you want to do.The Data Player Control view is enabled when the project is launched if Enable DataPlayerhas been checked in the project’s Run Configuration. The tab for this view is located in the lower right of the Apama Developer perspective next to the Console and Tasks views. If the Data Player Control view is not shown, select Window > Show View > Data Player Control from the Apama Plugin for Eclipse menu.
The individual controls available on title bar of the Data Player Control view are:
The Data Player playback controls are:
The Data Player displays the following status information:
Total Events — Shows the total events that have been played back during the current playback operation.
Last Event Time — Shows the recorded time of the last event played back.
Last Event — Shows the contents of the last event played back.
When a playback session has run to completion or if you click the Stop button and Generate time events from data has been enabled, the Data Player control displays the Query Done dialog. You can move the dialog box around if it covers up status information needed in order to decide whether to restart or stop the project.
Restart — Click Restart if you want to run another playback session. In the Terminate launch dialog click No. You can then display the Data Player control again and start another playback session.
Stop — Click Stop if you want to terminate all processes associated with the playback session.
If you want to create a new Data Player query template, you need to specify them in the project’s query template file. The Data Player query template file is found in the project’s bundle_instance_files folder and is named ADBC-queryTemplates-SQL.xml if you are using an ODBC or JDBC data source or ADBC-queryTemplates-Sim.xml if you are using a Sim data source.
To add a Data Player query template
In the Project Explorer view, expand the project’s bundle_instance_files folder and then expand either the ADBC_for_ODBC, ADBC_for_JDBC, or ADBC_for_Sim folder as appropriate.
Double click the query templates file, either ADBC-queryTemplates-SQL.xml or ADBC-queryTemplates-Sim.xml. The file opens in the text editor. The following is the standard findLatest Data Player query template for ODBC and JDBC data sources.
<query
name="findLatest"
description="Get the row with the latest time."
implementationFunction="substitution"
inputString="select * from ${TABLE_NAME} order by
${TIME_COLUMN_NAME} desc limit 1">
<parameter
description="Name of a table to query"
name="TABLE_NAME"
type="String"
default=""/>
<parameter
description="Name of the time column"
name="TIME_COLUMN_NAME"
type="String"
default="timeField"/>
</query>
The Data Player query templates are defined in the <query> element. The available attributes you can specify are:
name — A short name that will be displayed in the Query Template drop-down list.description — A short description of the Data Player query.implementationFunction — Currently Data Player query templates support the substitution function. This allows a token to be used as a place holder in a SQL query.inputString — For ODBC and JDBC data sources, use standard SQL syntax, from the example above:inputString="select * from ${TABLE_NAME} order by
${TIME_COLUMN_NAME} desc limit 1">
In the above example, $(TABLE_NAME) and $(TIME_COLUMN_NAME) are parameters that will be replaced with a database table and column name when you create a Data Player query from the template (see parameter, below).
For Sim data sources use the TIME keyword plus the comparators =, ><>=, <= plus the time. You can use one or two TIME expressions. Note, because you are adding the expression to an xml file, you need to use > and < instead of the characters > and <. Do not use spaces before or after > and <.
The inputString expression can also use the INCLUDE, and EXCLUDE keywords, for example:
TIME=12345;INCLUDE=(com.apama.something);EXCLUDE=(com.apama.something.abc)
TIME>12345;TIME<20000
parameter — For ODBC and JDBC data sources you can specify parameters and default values for the Data Player query template. This allows you to create different Data Player queries from the same template that, for example, query different tables or different columns in the database. From the above example:
<parameter
description="Name of a table to query"
name="TABLE_NAME"
type="String"
default=""/>
<parameter
description="Name of the time column"
name="TIME_COLUMN_NAME"
type="String"
default="timeField"/>
When you create a Data Player query from a template that has the parameters shown above in the Data Player Query Editor, you will specify the names of the database table and column to use with the Data Player query.
Save the Data Player query template file when you finish modifying it.
When you are ready to test your application, you can use the Data Player command-line interface to write scripts and unit tests to exercise the API layers. In some cases, it might be easier to play back events to the correlator using the command-line interface as compared to using the Data Player GUI in Apama Plugin for Eclipse.
To use the command-line interface to the Data Player, you must have already used the GUI interface in Apama Plugin for Eclipse to define Data Player queries and Data Player query configurations in Apama Plugin for Eclipse. When you use the command-line interface, you specify the Data Player query names and Data Player query configurations that you created in Apama Plugin for Eclipse.
For more information on using the Data Player command line interface, see Using the Data Player command-line interface.