Making application data available to clients

DataViews are an Apama abstraction that allows an application to present a read-only view onto some of their data for easy consumption by external clients that use Apama’s Scenario Service API (see Scenario Service API), such as Dashboard Builder dashboards.

External clients using the Scenario Service can be notified when instances are added and deleted, and when the outputs of a specific instance change. However, they cannot create, delete or edit instances themselves, as DataViews are read-only views and instance management is under the control of the application.

This topic is about the DataView Service, which provides an EPL API for creating/deleting/updating DataView definitions, instances and instance outputs such that they can be consumed by Scenario Service clients. The DataView Service is not the only way Apama exposes DataViews to the Scenario Service. MemoryStore tables, for example, can be configured to expose their rows as DataView instances (see Exposing in-memory or persistent data as DataViews).

The DataView Service uses two central concepts:

  • DataView definition

    A DataView definition specifies a unique DataView name, a set of field names and field types (each type is one of string, decimal, float, integer, and boolean), and optionally a set of key fields.

  • DataView item

    Each DataView item is associated with a DataView definition, and specifies values for the defined fields.

Info
The topics below briefly describe the event types that are used for managing the DataView definitions and items. For detailed information on the fields that are available for these events, see the com.apama.dataview package in the API reference for EPL (ApamaDoc).

Note that a DataView definition is not intended to serve as a central data structure for your application, but rather is intended merely to expose your application’s data to remote client applications.

The programming interface is defined by the DataViewService_Interface.mon file in the monitors directory of your Apama installation directory. It defines the API for working with DataView definitions and DataView items.

Using the DataView Service, you can create DataViews in only the main context. You cannot create them in any contexts you create.

Metadata properties can be specified for a DataView by adding keys with the prefix DataViewDefinition.EXTRA_PARAMS_METADATA_PREFIX to the extraParams dictionary of DataViewAddDefinition when adding the new DataView definition.

Adding the DataView Service bundle to your project

To use the DataViewService, you have to add the DataView Service bundle to your Apama project. Adding this bundle ensures that the following EPL files are loaded before any monitors that use them. These monitors are in the monitors directory of your Apama installation:

  • ScenarioService.mon
  • DataViewService_Interface.mon
  • DataViewService_Impl_Dict.mon
Info
The DataViewService is designed primarily to interact with other EPL applications that reside in the same correlator. However, it can also be used with multiple correlators. See Using multiple correlators for further information.

The description below explains how to add the bundle using Apama Plugin for Eclipse, but you can also add it using the apama_project command-line tool as described in Creating and managing an Apama project from the command line.

To add the DataView Service bundle to your Apama project

  1. In Apama Plugin for Eclipse, go to the Apama Developer perspective.

  2. In the Project Explorer, right-click the EPL Bundles node and select Add Bundle.

  3. Select the DataView Service bundle and click OK.

Creating DataView definitions

Use the following event types to create DataView definitions.

DataViewAddDefinition

Create and route an event of this type in order to create a DataView definition. The response is provided by a DataViewDefinition or DataViewException event.

DataViewDefinition

These events are responses to DataViewAddDefinition events. They indicate the successful creation of a DataView definition. The contents of the fields are exactly those of the DataViewAddDefinition event to which this is a response, except possibly for extraParams.

DataViewException

These events occur under exceptional circumstances in response to DataViewAddDefinition or DataViewDeleteDefinition events, or any circumstance under which a DataView cannot be identified.

Here is an example of creating a DataView definition and handling DataViewException events:

using com.apama.dataview.DataViewAddDefinition;
using com.apama.dataview.DataViewException;
...
DataViewAddDefinition add := new DataViewAddDefinition;
add.dvName := "Weather";
add.dvDisplayName := "Weather";
add.fieldNames := ["location","temperature","humidity","visibility"];
add.fieldTypes := ["string","integer","integer","integer"];
add.keyFields := ["location"];
route add;
on all DataViewException() as dvException {
log "*** Weather monitor error: " +
   dvException.toString() at ERROR;
}

Deleting DataView definitions

Use the following event types to delete DataView definitions.

DataViewDeleteDefinition

Create and route events of this type in order to delete a DataView definition. The response is provided by a DataViewDefinitionDeleted or DataViewException event.

DataViewDefinitionDeleted

These events are responses to DataViewDeleteDefinition events. They indicate the successful deletion of a DataView definition.

Creating DataView items

Use the following event types to create DataView items.

DataViewAddItem

Create and route an event of this type to create a DataView item. This item must not exist already. A response is provided by a DataViewItem or DataViewException event.

Here is an example that creates and routes a DataViewAddItem event, and handles the DataViewItem response by logging the addition of the item:

using com.apama.dataview.DataViewAddItem;
using com.apama.dataview.DataViewItem;
...
string location;
integer temp;
integer humidity;
integer visibility;
...
DataViewAddItem item := new DataViewAddItem;
item.dvName := "Weather";
item.fieldValues :=
   [location,temp.toString(),humidity.toString(),
   visibility.toString()];
route item;
on DataViewItem (dvName="Weather") as added {
   log("Weather monitor - DataViewItem: " +
      added.dvItemId.toString());
}

DataViewAddOrUpdateItem

Create and route an event of this type to create a DataView item if it does not already exist, or update a DataView item if it already exists. A response is provided by a DataViewItem or DataViewException event.

This will only work when keyFields are used. Any attempts to change the owner of an existing item will be rejected with a DataViewItemException.

DataViewItem

These events are responses to DataViewAddItem events. They indicate the successful creation of a DataView item. The contents of the fields are exactly those of the DataViewAddItem event to which this is a response, except possibly extraParams, and with the addition of the dvItemId field.

DataViewItemException

These events occur under exceptional circumstances in response to DataViewDeleteItem, DataViewUpdateItem or DataViewUpdateItemDelta events.

Deleting DataView items

Use the following event types to delete DataView items.

DataViewDeleteItem

Create and route an event of this type to delete a DataView item. A response is provided by a DataViewItemDeleted, DataViewException or DataViewItemException event.

Here is an example that creates and routes a DataViewDeleteItem event and handles the DataViewItemDeleted response by logging the deletion of the item:

using com.apama.dataview.DataViewDeleteItem;
using com.apama.dataview.DataViewItemDeleted;
string location;
...
DataViewDeleteItem delete := new DataViewDeleteItem;
delete.dvName := "Weather";
delete.dvItemId := -1;  // Set the ID to -1 when using keyFields
delete.keyFields := [location];
route delete;
on DataViewItemDeleted (dvName="Weather") as deleted {
   log("Weather monitor - DataViewItemDeleted:
      "+deleted.dvItemId.toString());
}

DataViewItemDeleted

These events are responses to DataViewDeleteItem events. They indicate the successful deletion of a DataView item.

DataViewDeleteAllItems

Create and route an event of this type to delete all DataView items associated with a specified DataView definition. A response is provided by a DataViewAllItemsDeleted, DataViewException or DataViewItemException event.

DataViewAllItemsDeleted

These events are responses to DataViewDeleteAllItem events. They indicate the successful deletion of all items associated with a given DataView definition.

Updating DataView items

Use the following event types to update DataView definitions.

Info
In addition to the event types listed below, you can also use the DataViewAddOrUpdateItem event to either create new DataView items or to update existing ones. See Creating DataView items.

DataViewUpdateItem

Create and route an event of this type to update a data item by specifying a sequence of new filed values. If the update does not succeed, a response is provided by a DataViewItemException event.

Here is an example of creating and routing a DataViewUpdateItem event:

using com.apama.dataview.DataViewUpdateItem;
...
string location;
integer temp;
integer humidity;
integer visibility;
...
DataViewUpdateItem update := new DataViewUpdateItem;
update.dvName := "Weather";
update.dvItemId := -1;  // Set the ID to -1 when using keyFields
update.fieldValues :=
   [location,temp.toString(),humidity.toString(),visibility.toString()];
route update;

DataViewUpdateItemDelta

Create and route an event of this type to update a data item by specifying a dictionary of field-position/field-value pairs. If the update does not succeed, a response is provided by a DataViewItemException event.

Here is an example of creating and routing a DataViewUpdateItemDelta event:

using com.apama.dataview.DataViewUpdateItemDelta;
...
string location;
integer temp;
integer humidity;
integer visibility;
...
DataViewUpdateItemDelta update := new DataViewUpdateItemDelta;
update.dvName := "Weather";
update.dvItemId := -1;  // Set the ID to -1 when using keyFields.
update.fieldValues :=
   {0:location,1:temp.toString(),2:humidity.toString(),
      3:visibility.toString()};
route update;

Looking up field positions

Use the following event types to look up the numerical position of a given field-name for a given DataView definition.

DataViewGetFieldLookup

Create and route an event of this type to request a helper dictionary that supports lookup of field position for a given field name. The response is provided by a DataViewFieldLookup or DataViewException event.

DataViewFieldLookup

These events are responses to DataViewGetFieldLookup events. They contain a dictionary that supports lookup of the field position for a given field name.

Using multiple correlators

The DataViewService is designed to primarily interact with other EPL applications that reside in the same correlator. Therefore, the DataViewService implementation does not send any events. You can inject the following optional additional monitors, which are in the monitors directory of your Apama installation, to send the events when that is required:

  • DataViewService_ServiceEmitter.mon
  • DataViewService_ApplicationEmitter.mon

This enables Dashboard Builder clients to visualize the state of a number of applications, each of which is running in a separate correlator, and each of which may fail-over to another correlator. Since configuring all of the dashboards to know about each of these correlators might be difficult and fragile, you can designate an additional single correlator as the “view correlator”, which holds the DataViewService and ScenarioService to which any client dashboard can connect.

With this architecture, the individual applications in the separate correlators need to send DataViewService request events to a channel that has been connected to the view correlator. These applications can either send the events directly, or with the Application Emitter injected they can route the events and the extra monitor will send them to the channel. The DataViewService in the view correlator routes its responses (as normal), but the Service Emitter monitor will then send those events out on the com.apama.dataview channel so that the originating correlators can receive them.

Note that these two emitters are entirely optional, and are not required for most deployments. Moreover, you normally do not inject these two monitors into the same correlator. Also, there is no bundle in Apama Plugin for Eclipse that provides these monitors.