20    Logging

20.1   Overview

CDMI™ logging is divided into three functional areas, each with differing levels of detail. These areas are

   object logging,

   security logging, and

   data management logging.

CDMI does not define the format of log messages. It is anticipated that future logging standards will address this area.

CDMI clients may access log data by creating a logging queue that indicates the scope of log messages that they wish to receive, as described in 20.5. If the user has sufficient permissions to create a logging queue, all log messages to which he or she has subscribed shall be enqueued into the queue, which may be accessed for processing and archival storage.

If multiple logging queues are defined, each queue shall get the log entry for a subscribed event. If no logging queues are defined that subscribe to a given log message or class of log messages, these messages do not have to be retained by the cloud storage system.

20.2   Object Logging

If logging is supported by the cloud storage system, all operations performed on CDMI objects (data objects, container objects, domain objects, queue objects, and capability objects) shall be persistently stored into all defined logging queues.

Log messages shall contain a minimum of the following information, in a format specified by the implementor:

   a timestamp in ISO-8601 format (see 5.14);

   the domain in which the operation was performed;

   the operation being performed;

   the URI of the object against which the operation was performed;

   the principal of the entity by which the operation was performed; and

   the result of the operation.

Operations logged should include operations performed to a CDMI-exported file system.

20.3   Security Logging

All security-sensitive events, including session establishment, authentication and authorization, and domain modifications and delegation shall be logged as security events. Security logging includes user and domain management, credential-related actions (i.e., revocation list validation) and should include out-of-band operations that affect the security of a cloud storage system (e.g., modifications of security properties of a CDMI domain via an administrative GUI).

If the cloud storage system supports a queue type of cdmi_logging_queue and a cdmi_logging_class of cdmi_security_logging as shown in 20.5, this indicates that the system supports audit logging. Consequently, the system-wide capability of cdmi_security_audit specified in Table 101 of 12.1.3 shall be set to "true". Otherwise, cdmi_security_audit shall not be present.

20.4   Data Management Logging

In addition to log messages associated with the alteration of metadata when changing data system metadata, logging should also include all conditions where the specified or actual data system metadata for objects change. For example, if the number of requested replicas was changed by a client, this change shall generate a log message indicating this change. A corresponding change in the actual number of replicas by the system shall also generate a log message.

This class of logging shall also contain object holds and retention policy log messages.

20.5   Logging Queues

Logging queues allow CDMI clients to get detailed logging information about the actions related to the operation of a cloud storage system. As queue data is persistent, no session state needs to be retained by the client. If different logging queues are used for different clients, then each client operates independently from the others (e.g., an analysis application may retrieve information about actions performed in a specific domain or set of objects using a logging queue that is uniquely configured to its specific needs).

Logging queues differ from notification queues (see Clause 21) in that the information provided is at a much more detailed level than notifications and is typically restricted to a smaller, privileged subset of clients.

When a client wishes to receive logging information, it may first check if the system is capable of providing logging by checking for the presence of the cdmi_logging capability in the root container capabilities. If this capability is not present, creating a logging queue shall be successful, but no logging entries shall be enqueued into the logging queue.

When creating a logging queue, the metadata described in Table 120 shall be provided. Attempts to change metadata in this table shall result in an  HTTP status code of 403 Forbidden. Once a logging queue has been created, with the exception of cdmi_queue_type, the metadata items in this table cannot be changed. cdmi_queue_type can only be removed, indicating to the system that the logging queue shall no longer receive log messages and shall be treated as a regular CDMI queue object.

Table 120 - Required Metadata for a Logging Queue

Metadata Name

Type

Description

Requirement

cdmi_queue_type

JSON String

The queue type indicates how the cloud storage system shall manage the queue object. The type of cdmi_logging_queue is defined for logging queues.

Mandatory

cdmi_logging_class

JSON Array of JSON Strings

Contains a JSON array that indicates which log messages are to be enqueued. Defined values are:

   cdmi_object_logging - Receive logging messages related to object operations;

   cdmi_datasystem_logging - Receive logging messages related to data system metadata state changes; and

   cdmi_security_logging - Receive logging messages related to security events.

Clients may include the desired classes of log messages in the cdmi_logging_class JSON array. If all log messages are desired, an empty JSON array shall be used.

Mandatory

cdmi_scope_specification

JSON Array of JSON Objects

The scope specification determines the set of objects for which associated log messages shall be enqueued. If logging is desired for all objects, include an empty JSON array. For security logging, the scope specification is ignored. See Clause 18 for how to construct a scope specification.

Mandatory

EXAMPLE 1   An example of the metadata associated with a logging queue is as follows:

{

   "metadata" : {

       "cdmi_queue_type" : "cdmi_logging_queue",

       "cdmi_logging_class" : [

           "cdmi_object_logging",

            "cdmi_security_logging"

       ],

       "cdmi_scope_specification" : [

           {

               "domainURI" : "== /cdmi_domains/MyDomain/"

            }

       ]

    }

}

When logging messages are dequeued from a logging queue, the contents of each queue value shall contain a JSON object and have a value MIME type of "application/json". This JSON object contains one or more JSON strings or objects, each representing a single log message.

Log messages are only included in a logging queue if the user who created the logging queue is able to access the object associated with the log message, (i.e., user has any ACE from 16.1.5).

EXAMPLE 2   If the logging queue was created by the administrator, then all matching objects, without restriction, are included in the results. If the logging queue was created by user "jdoe", then only logging messages for objects that "jdoe" is allowed to access are included in the results.

Table 121 describes the system-created metadata that provides details on the status of the logging queue.

Table 121 - Logging Status Metadata

Metadata Name

Type

Description

Requirement

cdmi_logging_status   

JSON String

A string indicating the state of the logging queue. Defined values are:

   Processing - Indicates that the logging queue is scanning for results;

   Halted - Indicates that new log messages will no longer be enqueued;

   Current - Indicates that the logging queue contained all log messages that can be found at this time; and

   Error - Indicates that the logging queue metadata is not valid, or other errors were encountered that prevented logging messages from being enqueued. Arbitrary vendor-defined text may follow the string "Error".

Mandatory

20.6   Logging Security Considerations

The timestamp accuracy and integrity of the log entries depend on the accuracy and integrity of the clock that is used to set their timestamp values. Accurate timestamps are essential to troubleshooting, forensic analysis of distributed attacks, dispute resolution, and proof of time-sensitive transactions. In essence, debugging, security, audit, and authentication are founded on the basis of event correlation (i.e., knowing exactly what happened in what order and on which side), and these security considerations depend on good time synchronization.

While specifying the accuracy and integrity of timekeeping is not within the scope of this international standard, to demonstrate that log timestamps are trustworthy, timestamps should be traceable to a standard time, and it should be demonstrated that system time may not be arbitrarily changed.