Shutting down and managing components

The correlator implements an interface with which it can be asked to shut themselves down, provide its process identifier, and respond to communication checks.

The tool for this is called engine_management, located in the bin directory. For historical reasons this functionality is also available as component_management.

Some of the functions of the engine_management tool can also be performed from within EPL. For more information, see Using the Management interface.

To use the correlator’s management tool, run the following command:

engine_management [ options ]

When you run this command with the –h option, the usage message for this command is shown.

Description of engine_management

Use the engine_management tool to connect to a running component. Once connected, the tool can shut down the component or return information about the component. The engine_management tool can connect to the correlator.

The engine_management tool sends output to stdout.

Options

The engine_management tool takes the following options. These options are also available for the component_management tool. All options are optional.

correlator.exe options
engine_inject some_EPL_files

engine_management -W 10

Management requests

The options in the tables below replicate -r (or --dorequest) request types of the same name.

The following options are specific to the correlator:

Exit values

The engine_management tool returns the following exit values:

Viewing garbage collection and application events information

The information in this topic applies to the correlator only.

You can enable logging of verbose garbage collection and application events in different ways and at different times, as described below.

Using the engine_management tool to enable garbage collection and application event logging for a running correlator

A handy way to view garbage collection (GC) information for a running correlator is to execute the following command:

engine_management -r verbosegc on

This command enables logging of garbage collection events, and is particularly useful in production environments. The additional garbage collection information goes to the correlator log, where the garbage collection messages are logged at DEBUG level and are signposted by a prefix, <apama.verboseGC.MonitorName>, where MonitorName indicates the monitor where garbage collection occurred.

To disable logging of garbage collection information, execute the following command:

engine_management -r verbosegc off

The above commands provide an alternative to the following command, which provides a great deal of detailed output (such as context-state changes, event triggering, spawning, routing, etc.) in addition to garbage collection information:

engine_management -r applicationEventLogging on

Again, this output goes to the correlator log, with the messages being logged at DEBUG level. Any additional log messages to the garbage collection messages are signposted by the prefix <apama.applicationEvents>.

To turn this off, execute the following command:

engine_management -r applicationEventLogging off

See Shutting down and managing components for more information on the -r (or --dorequest) option of the engine_management tool.

Enabling garbage collection and application event logging at correlator startup

You can also enable garbage collection logging at correlator startup. To do so, execute the following command:

correlator --loglevel apama.verboseGC=DEBUG

This enables verbose garbage collection logging for all monitors.

If you only wish to enable garbage collection logging for a particular monitor, append the name of the monitor (for example, “MyMonitor”) to apama.verboseGC as follows:

correlator --loglevel apama.verboseGC.MyMonitor=DEBUG

This enables verbose garbage collection logging only for the monitor which has the name “MyMonitor”.

Similarly, you can also enable application event logging at start time by starting a correlator with the following command:

correlator --loglevel apama.applicationEvents=DEBUG

Note:

If you enable application event logging at start time in this fashion, this will not enable verbose garbage collection logging (as is the case when enabling it using the engine_management request). Furthermore, it is not possible to enable application event logging on a per-monitor basis, as it is for garbage collection logging.

See Starting the correlator for more information on the --loglevel (or -v) option of the correlator command.

Using a YAML configuration file to enable garbage collection and application event logging at correlator startup

Another way to enable garbage collection logging or application event logging at start time is through a YAML configuration file. For more information, see the descriptions of the following categories in Setting correlator and plug-in log files and log levels in a YAML configuration file:

• apama.verboseGC
• apama.verboseGC.MonitorName
• apama.applicationEvents

Setting EPL log files and log levels dynamically

This topic describes how to configure logging for individual EPL packages. It applies to the correlator only. For information about configuring the log level of the whole correlator and plug-ins running inside it, see Setting correlator and plug-in log files and log levels in a YAML configuration file.

You can configure per-package logging in several ways:

• Dynamically, using the following options of the engine_management tool as described in this topic:
  • --setApplicationLogFile
  • --setApplicationLogLevel
• Statically, in a YAML configuration file when starting the correlator. See Setting EPL log files and log levels in a YAML configuration file for detailed information.
• Dynamically from within EPL. See Using the Management interface for detailed information.

In EPL code, you can specify log statements as a development or debug tool. By default, log statements that you specify in EPL send information to the correlator log file. If a log file was not specified when the correlator was started, and you have not executed the engine_management tool to associate a log file with the correlator, log statements send output to stdout.

In place of this default behavior, you can specify different log files for individual packages, monitors and events. This can be helpful during development. For example, you can specify a separate log file for a package or monitor you are implementing, and direct log output from only your development code to that file.

Also, you can specify a particular log level for a package, monitor, or event. The settings of log files and log levels are independent of each other. That is, you can set only a log level for a particular package, monitor or event, or you can set only a log level for a particular element. The topics below provide information for managing individual log files and log levels.

See also Rotating correlator log files.

Note:

Regularly rotating log files and storing the old ones in a secure location may be important as part of your personal data protection policy. For more information, see Recommendations for logging by Apama application code.

Tree structure of packages, monitors, and events

Packages, monitors and events form a tree as illustrated in the figure below. For each node in the tree, you can specify a log file and/or a log level. Nodes for which you do not specify log settings inherit log settings from their parent node.

The root of the tree is the default package, which contains code that does not explicitly specify a package with the package statement. Specified packages are intermediate nodes. Packages can nest inside each other. Monitors and events in specified packages are leaf nodes. If you specify an event type in a monitor, that event is a leaf node and its containing monitor is an intermediate node.

For example, suppose you specify packageA.log as the log file for packageA. The packageA.log file receives output from log statements in MonitorE and MonitorK. If EventF contains any action members that specify log statements, output would go to the packageA.log file.

Now suppose that you set ERROR as the log level for the default package and you set INFO as the log level for PackageB. For log statements in MonitorG, PackageH, and MonitorL, the correlator compares the log statement’s log level with INFO. For log statements in the rest of the tree, the correlator compares the log statement’s log level with ERROR. For details, see the table in Managing EPL log levels.

Managing EPL log levels

To set the log level for a package, monitor or event, invoke the engine_management tool as follows:

engine_management --setApplicationLogLevel [node=]logLevel

To obtain the log level for a particular node, invoke the tool as follows:

engine_management --getApplicationLogLevel [node]

If you do not specify a node, the tool returns the log level for the default package. To remove the log level for a node, so that it takes on the log level of its parent node, invoke the tool as follows. Again, if you do not specify a node, you remove the log level for the default package. The default package then takes on the log level in effect for the correlator. The default correlator log level is INFO.

engine_management --unsetApplicationLogLevel [node]

To manage the log level for an event that you define in a monitor, see Managing event logging attributes.

After the correlator identifies the applicable log level, the log level itself determines whether the correlator sends the log statement output to the appropriate log file. The following table indicates which log level identifiers cause the correlator to send the log statement to the appropriate log file.

See also Log levels determine results of log statements.

Managing EPL log files

To specify a log file for a package, monitor or event, invoke the engine_management tool as follows:

engine_management --setApplicationLogFile  [node=]logFile

To obtain the path of the log file for a particular node, invoke the tool as follows:

engine_management --getApplicationLogFile [node]

If you do not specify a node, the tool returns the log file for the default package. To disassociate a log file from its node, so that the node uses the log file of its parent node, invoke the tool as follows. Again, if you do not specify a node, you disassociate the log file from the default package. The correlator log file is then in effect for the default package. If a log file has not been specified for the correlator, the default is stdout.

engine_management --unsetApplicationLogFile [node]

Managing event logging attributes

If you specify an event type in a monitor, that event does not inherit the logging configuration from the enclosing monitor. It is expected that this will change in a future release. To explicitly set logging attributes for an event type defined in a monitor, invoke the engine_management tool and specify an unqualified event type name. Do not specify an enclosing scope, such as com.apamax.myMonitor.NestedEventType. For example:

engine_management --setApplicationLogFile NestedEventType=foo.log
engine_management --setApplicationLogLevel NestedEventType=DEBUG

Rotating correlator log files

Rotating a correlator log file refers to closing a log file being used by a running correlator and opening a new log file to be used instead from that point onwards. This lets you archive log files and avoid log files that are too large to easily view.

Each site should decide on and implement its own correlator log rotation policy. You should consider the following:

• How often to rotate log files.
• How large a correlator log file can be.
• What correlator log file naming conventions to use to organize log files.

There is a lot of useful header information in the main log file being used when the correlator starts. If you need to provide log files to Apama technical support, you should be able to provide the log file that was in use when the correlator started, as well as any other log files that were in use before and when a problem occurred.

Note:

Regularly rotating log files and storing the old ones in a secure location may be important as part of your personal data protection policy. For more information, see Protecting and erasing data from Apama log files.

To rotate the correlator log file and also rotate any other log file the correlator is using (input log file, EPL log files), see Rotating all correlator log files.

To rotate only the main correlator log file, see Rotating specified log files.

Note:

Some people use the term “log rolling” instead of “log rotation”.

Rotating all correlator log files

The information in this topic applies to the correlator only.

To invoke rotation of all log files that the correlator is using, you can do the following:

• Invoke the engine_management tool and specify --rotateLogs.

  This rotates the main correlator log file, the correlator input log file if it is being generated, and any EPL log files that are being generated. When you invoke this management request then the correlator closes each log file it was using.

  If the log filename specification declared ${START_TIME}, ${ROTATION_TIME} and/or ${ID}, then the correlator starts new log files with updated names according to the log filename specification; for example, if ${ID} was specified, then the ID portion of a log filename would be incremented by 1.

• In EPL, create a monitor that uses the Management interface EPL plug-in to trigger log rotation on a schedule. See Using the Management interface.

• On UNIX only, you can write a cron job that periodically sends a SIGHUP signal to Apama processes.

  The standard UNIX SIGHUP mechanism causes Apama processes to re-open their log files. If the log file names were specified with ${ROTATION_TIME} and/or ${ID}, then the re-opened files have names that contain the rotation time and/or the incremented ID.

If you want a log filename to always be the same and so did not declare ${START_TIME}, ${ROTATION_TIME} or ${ID} in the log filename specification, then the correlator starts new log files that have the same names as the log files it closed. Log files are appended to if the names are the same.

Rotating specified log files

Run one of the following utilities to rotate a particular log file. Write a cron job that periodically runs the utilities.

• The following command instructs the correlator to close its component-specific log file and start using a new log file that has the name you specify. If the name of the file contains blanks, be sure enclose it in quotation marks.

  engine_management --setLogFile log-filename
  

• The following command instructs the correlator to use the specified file as the log file for the specified node, which can be a package, monitor, or event. See also Setting EPL log files and log levels dynamically.

  engine_management --setApplicationLogFile [node=]log-filename
  

  If you use separate log files for particular packages, monitors, or events you might want to rotate those log files at the same time that you rotate the main correlator log file. This keeps your Apama log files in sync with each other. See Rotating all correlator log files.

A log file is never overwritten. If you rotate a log file and specify the same name, then Apama appends messages to the content already there.

Apama does not support automatic log file rotation based on log file size.

The only way to rotate the correlator input log is to rotate all correlator log files. See Rotating all correlator log files.