8. Logging and Auditing

xGT provides two types of logging output: security audit logging and general logging. Each is configured separately, and the logs will be written to different locations. General logging provides a variety of information about the state of the xGT server and operations being performed by it. The recorded events can be helpful in monitoring xGT’s usage and in diagnosing bugs or other issues.

The security audit consists of a record of authentication and access control events. It is used to track accesses of data in the xGT server and may be helpful in analyzing security incidents. Events logged in the security audit include:

  • Authentication events from the Python client.

  • Requests to run OpenCypher queries or other transactions on the server.

  • Security access control violations, such as a user attempting to read or modify data for which the user does not have permission.

8.1. General Logging

General logging in xGT is configured in the /etc/xgtd/xgtd.conf configuration file as described in Configuring the Server. The keys logging.filepath and logging.prefix control the log output location. General logging is divided into components, which group the log messages by type (logging component). A logging.level.X key sets the logging level of a component X.

Available logging levels are debug, info, warning, and fatal in order from least to most severe. Only messages of the same level as the chosen logging level or a more severe level than the chosen level will be logged. For example, to log only messages of severity greater or equal to warning for the component compiler, add the following line to the configuration file:

{
  "logging.level.compiler" : "warning"
}

The logging level fatal causes the xGT server to terminate. This level is only used during startup. Consequently, setting the logging level to fatal means that nothing will be logged after the server finishes its boot process.

8.1.1. Logging Components

The available components are:

  • xgt: General log messages about the xGT server.

  • config: All lookups of and changes to xGT config settings.

  • compiler: Operations taken by the compiler while processing a request.

  • io: Ingest and egest frames.

  • exception: Each exception thrown by the xGT server. This is logged at the DEBUG level.

  • query_optimization: Operations performed during query optimization.

  • metrics: Any errors in the computation of metrics used for query optimization. These are at the WARNING level.

  • runtime: Information about the parallel runtime, including threads, tasks, and memory use.

  • security: Security related log messages, including failure to load groups or labels at server startup. Note that this is separate from security audit logging.

8.2. Security Audit Logging

Security audit logging in xGT uses Apache log4cxx and must be configured with a log4cxx configuration file. The path to the log4cxx configuration file must be supplied in the /etc/xgtd/xgtd.conf configuration file using the key audit.config_file:

{
  "audit.config_file" : "path/audit.xml"
}

For more information on configuring xGT, see Configuring the Server. The rest of this section focuses on the configuration of the audit system in the file indicated by audit.config_file.

xGT sends security audit log messages to a single logger named xgtdaudit. The log4cxx configuration file must set up this logger by defining its logging level and by specifying one or more appenders which define where the messages sent to the logger are written:

<logger name="xgtdaudit" additivity="false">
    <level value="INFO"/>
    <appender-ref ref="appender1"/>
    <appender-ref ref="appender2"/>
</logger>

The level defines the severity of messages that will be recorded for the logger. Possible values are TRACE, DEBUG, INFO, WARN, ERROR and FATAL in order from least to most severe. If a logger is assigned level X, it will only log messages of severity equal to or greater than X. For example, if the xgtdaudit level is set to WARN, only messages marked WARN, ERROR, or FATAL will be written.

xGT writes all audit messages at the INFO level.

By default, log4cxx sends all messages to the root logger as well. The additivity="false" setting prevents messages logged to the xgtaudit logger from also being sent to the root logger.

8.2.1. Appenders

The audit messages sent to a logger can be output to multiple destinations, each defined by an appender. The log4cxx configuration file should define one or more appenders. The following example defines a rolling file appender that rolls after a file size of 10 megabytes and keeps 100 backup files.

<appender name="rollingFile" class="org.apache.log4j.RollingFileAppender">
    <param name="File" value="debug.log"/>
    <param name="MaxFileSize" value="10MB"/>
    <param name="MaxBackupIndex" value="100"/>
    <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="%d{ISO8601} [%p] %c (%t) {%x} - %m%n"/>
    </layout>
</appender>

Using the PatternLayout allows the output format to be customized using conversion characters. In the above example, the message consists of and ISO8601 formatted date (%d), the message level (%p), the logger name (%c), the name of the thread that generated the logging event (%t), the nested diagnostic context associated with the thread (%x), and the log message (%m%n).

Instead of defining a custom layout, pre-formatted JSON messages may be logged by choosing JSONLayout:

<layout class="JSONLayout"/>

The following appender types are supported by log4cxx.

Name

Description

Usage Notes

ConsoleAppender

Writes log messages to stdout or stderr.

The output target is specified by target=System.out or target=System.err.

FileAppender

Writes log messages to a file.

The file name is specified by File.

RollingFileAppender

Writes log messages to a file and rolls over when the file reaches a chosen size.

The roll-over file size is specified by MaxFileSize. The number of backup files kept is specified by MaxBackupIndex.

DailyRollingFileAppender

Writes log messages to a file and rolls over at a chosen frequency.

The roll-over frequency is specified by DatePattern. It may be yearly, monthly, daily, hourly, or every minute. To roll every minute, use DatePattern=’.’yyyy-MM-dd-HH-mm. To roll daily use ‘.’yyy-MM-dd. The number of backup files kept is specified by MaxBackupIndex.

SocketAppender

Sends messages to a remote log server.

NTEventLogAppender

Appends messages to NT EventLog.

SyslogAppender

Sends messages to a remote syslog daemon.

More detail is available at Apache’s appender documentation.

8.2.2. Audit Fields

xGT adds the following fields to each message in the audit log:

Name

Description

level

The priority of the entry. xGT logs all audit messages at the INFO level.

logger

The source logger name. This is always “xgtdaudit”.

message

A detailed message explaining what was logged.

timestamp

The server timestamp when the entry is written.

context_map

The Mapped Diagnostic Context (MDC) that includes simple key-value pairs. Used for information like username, ip address, or other information.

The context_map contains the following subfields:

Name

Description

code

The category of the audit event.

identity

The username that initiated the action.

ipAddress

The IP address that initiated the action.

xGT uses the following audit codes:

Name

Description

AUTH

User authentication attempt, successful or not.

KEEPALIVE

Heartbeat request from the client to the server.

CLOSE

Connection termination.

QUERY

Query job submission.

CANCEL

Job cancellation.

ACCESSOR

Attempt to access a frame a user doesn’t have permission to see.

SHUTDOWN

Server shutdown request.

PUTDATA

Insert of data into a frame. Does not include queries inserting into results frames.

GETDATA

Request for data from a frame or job immediate results.

8.2.3. Filtering Audit Output

A site may want to log only a subset of the audit events. The log4cxx library supports filtering of events.

Two useful filters are the MapFilter and StringMatchFilter.

Name

Description

MapFilter

Filters using exact matching based on subfields in the event’s context_map.

StringMatchFilter

Filters by matching for a substring in the event’s message.

8.2.3.1. MapFilter

The MapFilter filters events based on key value pairs in the context_map. A user might want to only log events from one or more categories, or they might want to exclude events from one or more categories.

Consider the case where only events from one or more categories are desired to appear in the log file. There needs to be a filter section for each category code to accept followed by a filter to deny everything else. Here’s an example appender that accepts audit events with the codes QUERY, PUTDATA, or GETDATA and rejects all other events.

<appender name="auditJson" class="org.apache.log4j.RollingFileAppender">
    ...
    <filter class="log4cxx.MapFilter">
        <param name="code" value="QUERY"/>
        <param name="AcceptOnMatch" value="true"/>
    </filter>
    <filter class="log4cxx.MapFilter">
        <param name="code" value="PUTDATA"/>
        <param name="AcceptOnMatch" value="true"/>
    </filter>
    <filter class="log4cxx.MapFilter">
        <param name="code" value="GETDATA"/>
        <param name="AcceptOnMatch" value="true"/>
    </filter>
    <filter class="DenyAllFilter" />
</appender>

The filter xml is placed inside the appender the filter affects. A true value for AcceptOnMatch sends the event to the appender if the filter matches the code. The DenyAllFilter rejects any events that don’t match any of the other filters.

Now, consider the case where events from all categories except for one or more are desired to appear in the log file. There needs to be a filter section for each category code to deny. Here’s an example appender that accepts all audit events except those with the codes CLOSE and KEEPALIVE.

<appender name="auditJson" class="org.apache.log4j.RollingFileAppender">
    ...
    <filter class="log4cxx.MapFilter">
        <param name="code" value="CLOSE"/>
        <param name="AcceptOnMatch" value="false"/>
    </filter>
    <filter class="log4cxx.MapFilter">
        <param name="code" value="KEEPALIVE"/>
        <param name="AcceptOnMatch" value="false"/>
    </filter>
</appender>

A false value for AcceptOnMatch doesn’t send the event to the appender if the filter matches the code. All other events are sent to the appender.

These two examples used the code from the context_map, but any other key in the map could be filtered on as well.

8.2.3.2. StringMatchFilter

The StringMatchFilter filters audit events based on the message. Consider the case where events whose message contains the word “data” should be filtered. Here’s what the filter section would look like in the config file.

<appender name="auditJson" class="org.apache.log4j.RollingFileAppender">
    ...
      <filter class="log4cxx.StringMatchFilter">
          <param name="StringToMatch" value="data"/>
          <param name="AcceptOnMatch" value="false"/>
      </filter>
</appender>

8.2.4. Sample log4cxx Configuration

The configuration file below shows the default audit logging behavior for xGT. It outputs xgtdaudit logger messages in JSON format with level greater than or equal to INFO to a file that rolls every hour and to the console. The file names are relative to the location from which the xGT server is running. The root logger is disabled because xgtd doesn’t send anything to it.

<?xml version="1.0" encoding="UTF-8"?>
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

    <root>
        <level value="OFF"/>
    </root>

    <appender name="auditJson" class="org.apache.log4j.RollingFileAppender">
        <param name="File" value="/var/log/xgtd/audit.log"/>
        <param name="MaxFileSize" value="10MB"/>
        <param name="MaxBackupIndex" value="10"/>
        <layout class="JSONLayout"/>
    </appender>

    <logger name="xgtdaudit" additivity="false">
        <level value="INFO"/>
        <appender-ref ref="auditJson"/>
    </logger>

</log4j:configuration>

Apache provides sample configuration files here <https://logging.apache.org/log4cxx/configuration-samples.html>.