Data Model and I/O

Introduction

The purpose of this chapter is to describe the model that is used as the basis for MINOS persistable records and the I/O related management of those records.

Persistency and ROOT

Persistence describes the process of making objects permanent beyond their present application for reuse by some later application, such as when a C++ object is written to a file.

The problem of persisting C++ objects to file and retrieving those objects back in memory is not trivial. Objects may have complex structure (hierarchy of inheritance, and data members that are objects or pointers to objects) and C++ has weak built-in support for object persistency.

The solution to this problem is to supplement C++ with a framework for use of its I/O facilities. In the case of MINOS, ROOT provides this framework. In particular, ROOT provides the following persistency tools:

• Automatic Streaming mechanism for user-defined classes. The Streaming mechanism is used to convert object data to a stream of bytes (data buffer) to be stored on file and vice versa.
• File management (TFile) to organize storage of object data buffers in a file with Keys to facilitate random and sequential access of objects according to a user provided Key Name.
• Data structures (TTree) to coordinate sets of objects. The ROOT TTree is a generalized ntuple that can be used to store objects as well as basic data types. Synchronized and partial read of members of a set are supported.
• Remote access to files (TNetFile along with a ROOT supplied server daemon) to support a ``Distributed Database'' environment in which data may be distributed over multiple files (some remote).
• Automatic Class Schema Evolution such that a class definition can change, for example data members may be added and removed to produce a new version of the class, but it is still possible to read objects of an older version of this class from a data file.
• Persistable Pointers such that the relationship between objects written out to different trees, possibly even in different files, can be reconstructed on input.

Data Model

The data model is defined by the classes that are used to store the data, the data structures that are used to store the data objects, and the organizational scheme that is used to store the data in the data structures and the data structures in the files.

Two considerations that have gone into defining a MINOS data model are:

• A desire to make maximal use of the persistency tools provided by ROOT. For example the ROOT TTree data structure allows partial input of data members of a set (analogous to reading only a selected subset of columns of an ntuple for a particular analysis). To maximize the use of this aspect of the ROOT TTree data structure it is desirable to store frequently accessed smaller data objects on a separate branch (column) from larger data objects.
• A desire to avoid overspecializing or hardwiring record class definitions in the input and output stream management code.

Definitions

The MINOS data model makes use of the following terms:

• A Record consists of a single header and a collection of data blocks, and constitutes a single entry in a ROOT TTree. To take advantage of the partial input feature of the TTree, the header is stored on a separate branch from the collection of data blocks.
• A Stream consists of a tree in a single file or possibly sequence of multiple files. A Stream name may be the same as a Tree name, but need not be. For example, Stream ``Cand1'' may refer to TTree ``Cand'' in File1, while Stream ``Cand2'' refers to TTree ``Cand'' in File2.

More on Records

To be supplied.

Input Stream Management

Figure 11.1: Diagram illustrating layout of MINOS records, trees and files for a typical far detector run.

Figure 11.1 illustrates the organization of MINOS records in ROOT TTrees and the organization of ROOT TTrees in data files. The diagram also illustrates how the records from the different streams are synchronized and loaded into the MomNavigator (Mom) according to each record's VldContext by the input stream manager.

In particular, note that:

• Multiple records with coincident VldContext are loaded in the same Mom object as a single record set.
• Some trees, for example the DaqMonitor tree, may contain multiple records of identical VldContext that will also be loaded into the same Mom object as part of a single record set.

Identifying the Source of a Data Record

All records are stamped on input with the streamname, filename, treename, and tree index from which they came. (This information is stamped on the record whether it is received through the dispatcher or through offline file access.) The data is stored in a temporary (not persisted) Registry in the RecMinos base class. The following code shows how to access this data given a pointer ``mom'' to a MomNavigator object:

   // Iterate over all objects contained in Mom
   TIter mitr = mom -> FragmentIter();
   TObject* object;
    while ( (object = mitr.Next()) ) {
        RecMinos* record = dynamic_cast<RecMinos*>(object);
        if ( record ) {
           char* streamname = 0;
           if ( record -> GetTempTags().Get("stream",streamname) )
              cout << "stream " << streamname << endl;
           char* treename = 0;
           if ( record -> GetTempTags().Get("tree",treename) )
             cout << "tree " << treename << endl;
           int treeindex = 0;
           if ( record -> GetTempTags().Get("index",treeindex) )
             cout << "index " << treeindex << endl;
           char* filename = 0;
           if ( record -> GetTempTags().Get("file",filename))
           cout << " file " << filename << endl;
        }
     }

Output Stream Management

To be supplied.

Miscellanea

Rules for writing a Persistable Class

Since ROOT forms the framework for MINOS persistency, it defines the rules that the user must follow when defining a class to be persisted to file. What follows is an abbreviated list of rules to remember when writing a persistable class. A much more detailed description of defining a class can be found in the ROOT documentation.

• The class header file should include a line invoking the ROOT provided

        ClassDef(ClassName,ClassVersionID) // Class Title
  

  macro. An illustration of use is shown in an excerpt from RecoBase/CandStripList.h:

        class CandStripList : public TObject {
          ...
          ClassDef(CandStripList,1)  //  Concrete Event Candidate Class   
        };
  

  The ClassDef statement should be included just before the closing brace. The class version number should be incremented whenever the data members of the class change, for example when the data member order is changed, a new member is added or an old member is removed, or the type of a data member changes. It should also be incremented if the list of base classes that the defined class inherits from changes.

  Note that setting the version number in the ClassDef to 0 means that the objects of this class are not to be persisted.

• The class implementation file should invoke the ClassImp(ClassName) macro. An example can be found in RecoBase/CandStripList.cxx:

        ...
        #include "MessageService/MsgService.h"
        
        ClassImp(CandStripList)
        ...
  

• Activate the automatic schema evolution capabilities provided with ROOT by specifying the '+' option after the classname in the LinkDef.h file, e.g.:

        #pragma link C++ class CandStripList+;
  

  This should be done even with the first version of the class. Don't wait until the second version of the class to activate this or extra work will have to be done to read objects of older versions of this class stored in old data files!

• If the class is to be the top level record class persisted to a ROOT TTree, it should inherit from the Record/RecMinos class and have a header that inherits from the Record/RecMinosHdr class. This is because the input stream manager uses the VldContext stored in the base header class for sorting the records from the different trees on input.