Where personal data is held within the Apama platform

Most deployments of Apama deal with personal data only in customer-defined data fields, which are largely under the control and responsibility of the customer who writes and deploys the Apama application.

In Apama applications, customer-defined data is usually held “in memory” in EPL event fields and monitor variables, by connectivity plug-ins, or by EPL plug-ins such as the MemoryStore.

Customer-defined application data may also be stored “at rest” in the following places:

  • Correlator log files (the main log file, and also any additional files defined by eplLogging and correlatorLogging configuration). These files include logging performed by the customer’s application and by standard Apama connectivity and EPL plug-ins and the correlator itself. For example, the contents of Apama events are often logged (either in full, or truncated) if an error occurs during processing or sending of the event, and data from events or other EPL data structures may be logged as part of correlator error messages.
  • Correlator input log. If enabled, it contains the contents of all events sent into the correlator.

We strongly advise against allowing any personal data to exist in the application logic itself (the EPL source files), and this documentation assumes that this principle is being followed.

In addition to the customer-defined data mentioned above, there are a small number of situations where the Apama platform could potentially be considered to directly handle personal data. You should establish whether in your own environment any of the “users” listed below represent the “personal data” of an identifiable human protected by legislation, and which merely represent machine-to-machine communication, or system administrators who have accepted the logging of their user name and IP address as part of their terms of employment.

Product area

Potential “personal data”

Where data could be stored

Correlator

User identifiers and IP addresses for direct connections to/from Apama server processes (typically only for machine-to-machine communication between server processes, or monitoring and management by system administrator accounts). These are logged to provide an audit trail in case of an attack or accidental mistake by a system administrator.
  • correlator main log file
  • correlator input log

HTTP server connectivity plug-in

User identifiers and IP addresses of clients that connect to the HTTP server, as specified in HTTP header. These are written to the log file. Along with other HTTP headers they are also present in the message metadata. Thus they can optionally be mapped to fields in an Apama event, using a connectivity codec such as the mapper codec. See also The HTTP Server Transport Connectivity Plug-in.
  • correlator main log file
  • correlator input log file
  • correlator in-memory state

HTTP server connectivity plug-in, only if authentication is enabled

User identifiers of clients who are permitted to connect to the HTTP server (with a secure hash of the passwords). See also The HTTP Server Transport Connectivity Plug-in.
  • HTTP server authentication password file, which is stored on disk in plaintext and contains un-encrypted usernames and hashed salted passwords, see Authentication. As the file is completely under the user’s control, you can use standard tools included with your operating system to set access control for protecting this data as needed. Users can be deleted from the file using a text editor or the httpserver_passman provided by Apama as described in the documentation.