The job control package provides the interface to the data for reconstruction and analysis modules. The package defines the basic unit for reconstruction and analysis, the ``JobCModule'' and provides several standard modules which allow users to organize the various modules into reconstruction/analysis jobs.
JobCModules allow physicist programmers to plug their reconstruction and analysis code into the framework. A JobCModule consists of a header (.h) and an associated (.cxx) file. A module calls algorithms which interface with the data to perform reconstruction. Several JobCModules are provided by the framework which perform basic reconstruction and analysis of MINOS data. These can be organized into reconstruction and analysis jobs using ROOT macros. Instructions for configuring the loon executable using ROOT macros are given in Section 9.1.
In addition to the modules provided by the framework, users can provide their own JobCModules and plug them into loon. Instructions for how to do this are provided in Section 9.2
The loon executable is configured via a ROOT C++
macro. The basic command is:
loon
-[options] macro.C -o out.root
in1.root in2.root in3.root ...
where ``macro.C'' is the ROOT macro which configures the job, and
``out.root'' is the default name to use for data output via the
output module. The list of input data files are listed as
``in1-in3.root''. The possible command line options are:
-b :
Run in batch mode without graphics.
-n :
Do not execute ROOT logon and logoff macros.
-q :
Quit the job after completing the commands in the macro.
-h :
Print usage help summary.
-H[module] :
Print help for a specified module.
-d[url]: Specify database URL.
-u[user]: Specify database user.
-p[passwd]: Specify database password.
-t[time]:
Set the time limit for the job (eg. -t'10 minutes').
-r[n]:
Specify the maximum number of records to evaluate before
quitting.
-o[out]:
Set the default name to use for output file.
The most common options are ``-bq'' which specify batch running of
loon. loon can also be run interactively (just
leave off the ``-bq'' option). In this mode the user will be given
essentially a ROOT prompt and can execute C++ statements, ROOT macros and commands.
Job control macros provide several functions. The macros are used to 1) organize the analysis and reconstruction modules into ``paths'' which define the sequence the modules are applied to the input data, 2) alter the configuration of jobs modules and framework services, and 3) trigger module commands. Several example job control macros are provided in the Demo package which is a good source to see working examples of macros and JobCModules.
The basic function of the job control macro is to define the list of
reconstruction and analysis steps to perform on the data. The
following example macro defines a very simple job path:
{
JobC j;
j.Path.Create("Demo",
// if MC "RerootToRawDataModule::Get "
"DigitListModule::Get "
"DigitListModule::Reco "
"UserAnalysis::Ana "
"CheezyDisplayModule::Ana "
"EventDump::Ana");
j.Path("Demo").Run();
}
The first step in the macro, JobC j, creates a new job. A new
reconstruction/analysis path, named ``Demo'' is created within this
job using the C++ command j.Path.Create. The first
argument to this command in the name of the path (``Demo'' in this
case) which is used to reference this path later in the macro. The
second argument is a list of modules (EventDump eg.) and methods
(Ana, eg.) to add to the path. The example, shows the typical
``event preparation'' steps (RerootToRawDataModule::Get does the
data format conversion for Monte Carlo events, DigitListModule
builds the candidate digit lists which hold the hit strip data), as
well as some modules provided by the framework to print and display
information about events. Finally, this macro runs all the data from
the input files through the ``Demo'' path via the command, j.Path("Demo").Run() .
While the ``Create'' command is the most common, the ``Path'' module
provides several other commands to configure job paths. The complete
list can be viewed in source file JobCPathModule.h which defines
the interface to the ``Path'' module; several commands are listed
here:
j.Path("Demo")
// return the path named ``Demo''
j.Path.Delete("Demo");
// delete the job path named ``Demo''
j.Path.Add("Demo","Module::Method");
// Append to a job path.
j.Path.Attach("A","B");
Link the end of path A to the beginning of path B
j.Path.Report();
// Print a summary of all active job paths.
j.Path.Reset();
// Delete all active paths
Job paths can be run in several ways:
j.Path("Demo").Run();
// Run until all input data has been processed
j.Path("Demo").Run(10);
// Run until 10 records have been read in
j.Path("Demo").RunNin(10);
// Same as Run(10)
j.Path("Demo").RunNpass(10);
// Run until 10 records pass all filters set in the path. Filters
are discussed in Section 9.1.1.
j.Path("Demo").RunNfail(10);
// Run until 10 records have been failed by the path.
To get a summary of the run, use the Report() command:
j.Path("Demo").Report();
This produces output like this:
Demo(10 in 1 out 9 filt.) 1) +DigitListModule::Get n=10 ( 10/ 0) t=( 179.53/ 7.13) 2) +DigitListModule::Reco n=10 ( 10/ 0) t=( 56.71/ 1.46) 3) +UserAnalysis::Ana n=10 ( 1/ 9) t=( 0.01/ 0.00) 4) +EventDump::Ana n=1 ( 1/ 0) t=( 0.01/ 0.00)This report indicates that the path ``Demo'' has processed 10 records, 1 of which passed the analysis, 9 of which were filtered. The path is made of 4 nodes (a Get, a Reco, and two Ana's). The number of event processed by each node is listed. In this example, the node UserAnalysis::Ana saw 10 records, passed 1 of those and filtered 9. The last two columns list the total cpu and system time (in seconds) spent inside the node.
The handling of filter settings can be configured in several
ways. First, the user can opt to turn off all the filters in a path
off. In this case processing will continue even for events which
fail. All the filters in a path can be switched on and off using a
command like:
j.Path("Demo").SetAllFilters(false);
// Turn all filters off
j.Path("Demo").SetAllFilters(true);
// Turn all filters on
Further, filtering at specific points in the analysis chain can be
configured. For example,
j.Path("Demo").Node("MyModule::Ana").FilterOn();
j.Path("Demo").Node("MyModule::Ana").FilterOff();
Users can also reverse the meaning of the pass/fail decisions made by
elements of the analysis chain. For example, if one wanted to output
(ie. ``pass'') all events which failed the reconstruction path
``Demo'':
j.Path("Demo").ReverseAllFilters();
Alternately, the filters can be switched one-by-one, eg.:
j.Path("Demo").Node("MyModule::Ana").ReverseFilter();
Users can view the status of filters set in a given path with the
``Path.Report()'' command:
1) +DigitListModule::Get n=10 ( 10/ 0) t=( 179.53/ 7.13) 2) DigitListModule::Reco n=10 ( 10/ 0) t=( 56.71/ 1.46) 3) !UserAnalysis::Ana n=10 ( 1/ 9) t=( 0.01/ 0.00) 4) EventDump::Ana n=1 ( 9/ 0) t=( 0.01/ 0.00)The markers to the left of each node label that the filter is on (``+''), on and reversed (``!''), or off (blank). In this example, the filter for the UserAnalysis::Ana node has been reversed. As you can see from the report, 9 events were failed by this node, and 9 events were processed by the next node in the path. In the case that a node makes no pass/fail decision all records are passed regardless of the on/off/reverse setting.
There are several things that can be configured from a job control macro. These include the modules which perform the reconstruction as well as various services provided by the loon program.
Each job module carries with it a set of configuration parameters. To
see the set of configuration parameters using the ``Report'' command:
loon [0] gSystem->Load("libDemo.so");
loon [1] JobC j;
loon [2] j.Path.Create("A","EventDump::Ana");
loon [3] j.Path("A").Mod("EventDump").Report();
EventDump configured with: Registry: `EventDump.config.default', 13 entries.
[U|L] DumpAll = 0
[U|L] DumpCandDigits = 0
[U|L] DumpFragments = 1
[U|L] DumpObjectTable = 0
[U|L] DumpRawDaqSnarlHeader = 1
[U|L] DumpRawData = 0
[U|L] DumpRawHeader = 1
[U|L] First = 0
[U|L] Freq = 1
[U|L] Last = 999999999
[U|L] PrintSeparator = 1
[U|L] RawBlockList =
[U|L] Wait = 0
Here the ``EventDump'' module has been placed into the path ``A''. The
``EventDump'' module is currently configured using a configuration
names ``EventDump.config.default'', which has 13 entries with the
values shown. Any of these parameters can be changed using a ``Set''
command:
loon [4] j.Path("A").Mod("EventDump").Set("Freq=10");
loon [5] j.Path("A").Mod("EventDump").Report();
EventDump configured with: Registry: `EventDump.config.default', 13 entries.
[U|L] DumpAll = 0
[U|L] DumpCandDigits = 0
[U|L] DumpFragments = 1
[U|L] DumpObjectTable = 0
[U|L] DumpRawDaqSnarlHeader = 1
[U|L] DumpRawData = 0
[U|L] DumpRawHeader = 1
[U|L] First = 0
[U|L] Freq = 10
[U|L] Last = 999999999
[U|L] PrintSeparator = 1
[U|L] RawBlockList =
[U|L] Wait = 0
From the report you should be able to see that the frequency parameter
``Freq'' has been changed from 1 to 10.
Modules support a second interface (``Cmd'' instead of ``Set''). These commands vary from module to module. They should be documented in the help for the module. For example, j.Path("").Mod("Module").Help();.
Several of the framework services can be configured. For example,
message print thresholds can be altered using the ``Msg'' module. Here
are some examples:
j.Msg.SetLevel("Plex","Error");
// Raise the threshold from emitting messages from the Plex
package so that only error messages are printed.
j.Msg.SetLevel("MyPackage","Verbose");
// Lower the threshold for ``MyPackage'' so that all debugging
messages are printed.
j.Msg.Attach("MyPkg::Error","mypackage.err");
// Attach the file ``mypackage.err'' to the ``MyPkg::Error''
stream.
j.Msg.Format("MyPkg::Debug","File,Line",off);
// Remove the file name and line number tag from messages printed
to ``MyPkg::Debug'' stream.
j.Msg.Stats();
// Print statistics about message stream use. Useful to see if
any warnings or errors were produced during your run.
This section reviews some basic modules provided by the framework and how to use them.
loon [0] JobC j;
loon [1] j.Input.Report();
INPUT configured with: Registry: `INPUT.config', 9 entries.
[U|L] DDSDataSource = Daq
[U|L] DDSKeepUpMode = FileKeepUp
[U|L] DDSMaxSyncDelay = 15
[U|L] DDSOffLine = 0
[U|L] DDSPort = 9090
[U|L] DDSServer = daqdds.minos-soudan.org
[U|L] DDSTimeOut = 120
[U|L] Format = input
[U|L] Streams = DaqMonitor,DaqSnarl,LightInjection
As you can see from the report the input module has several
configuration parameters. Most of these are for controlling the
interface to the DataDispatcher (DDS), which is not the most common
use of the input module. Generally, the input module talks to files on
a local disk. The two parameters you need to set in that case are
``Format'' which is in general either ``input'' for detector data,
``reroot'' for Monte Carlo. If you want to take data from the
DataDispatcher you would j.Input.Set("Format=dds"). The second important
parameter is the setting of which data streams you would like to
subscribe to. For example, if you only wanted to see ``DaqSnarl''
records you would j.Input.Set("Streams=DaqSnarl");
The input module has the following commands:
j.Input.Next(n)
// Advance n records in the stream (default is 1).
j.Input.Prev(n)
// Go back n records in the stream (default is 1).
j.Input.GoTo(run,event)
// Go to specified run event number.
j.Input.List()
// List all files atteched to the input module
j.Input.AddFile("filename")
// Add a file to the list
j.Input.RemoveFile("filename")
// Remove a file from the list
j.Input.NextFile(n)
// Advance n files in the list (default is 1)
j.Input.PrevFile(n)
// Go back n files in list (default is 1)
j.Input.GoToFile(n)
// Go to the ith file in the list
j.Input.GoToFile("file")
// Go to the file ``file'' in the list
j.Input.Select("stream","cut")
// Set a selection cut on a stream
When a JobC object is created the Input module automatically loads files from the loon command line.
This section will outline how to make a job module and link it into loon.
To use the generation program, type:
gen_module
.
The program will prompt the user for several fields:
Once complete, gen_module will write the .cxx and .h files for the class. These classes will require some tweaking before they compile. For example, the Config() method will need to be hooked up before it will work. Also, comments will need to be added (search for the tag ``FILL_IN''. Oh, and you'll have to fill in your analysis and reconstruction code...
Once the module compiles, you'll need a GNUmakefile if you already have one. For most packages this is simple:
# # GNUmakefile for simple package # ROOTCINT := YES LIB := lib$(PACKAGE) LIBCXXFILES := $(wildcard *.cxx) ######################################################################## include SoftRelTools/standard.mk include SoftRelTools/arch_spec_root.mk #A ``gmake'' should get your module compiled into the package library and the library installed in your test release.
The next step is to attach your module to the loon
program. This step is done by updating a file which lives in the SRT
test and/or base release which stores the map from job module names to
the libraries that provide them. To update this file, use this
command:
makemodulemap
That should be all there is to it. When you ask to add a module to a
job path, 
will consult this file
( $SRT_PRIVATE_CONTEXT/tmp/jobmodules.txt), load the required
library, and build your module.
![[*]](/usr/share/latex2html/icons/crossref.png){kind=icon}
.
User modules must inherit from the base class JobCModule as shown in line 019 in the figure. The user module can optionally provide implementations for one or more of the following methods:
JobCResult Module1::Ana(const MomNavigator* mom) {
// Simplest case - no errors, no pass/fail decisions
return JobCResult::kAOK;
//
// This would also work:
//
// JobCResult r;
// return r;
}
//......................................................................
JobCResult Module2::Ana(const MomNavigator* mom) {
//
// Another simple case: A single pass/fail decision
//
if (this->PassesCuts(mom)) return JobCResult::kPassed;
// else
return JobCResult::kFailed;
}
//......................................................................
JobCResult Module3::Ana(const MomNavigator* mom) {
//
// A more complicated case: Apply selection criteria and flag error
// conditions
//
JobCResult r; // The result of this method's analysis of mom
SomeDataObject* d = this->GetSomeDataObject(mom);
// Do some error checking on the object d (for example)
if (d->GetSomeNumber() > this->GetValidRange()) {
r.SetError(); // Flag this condition as an error
r.SetFailed(); // Fails selection criteria
}
// Apply selection criteria
if (d->PassesCuts()) {
r.SetPassed();
}
else {
r.SetFailed();
}
// Return final result, could contain both ``failed'' and ``error''
return r;
}
However, in some cases, you may need more specialized than the Set() command. In this case, you will want to talk to the module using the Cmd() command. This interface allows users to issue a large variety of commands to the module. For example,
loon [0] JobC j;
loon [1] j.Path.Create("A","AModule::Ana");
loon [2] j.Path("A").Mod("AModule").Cmd("Draw timeHisto chargeHisto");
This command might cause histogram windows to pop-up containing time
and charge information.
If you find that you need to be able to talk to your module with more flexibility than is provided via Set() you will need to implement the HandleCommand method. This method receives a parsed version of the string command typed by the user as the argument to the Cmd() method. This parsed version is called a JobCommand.
JobCommands are of the form /c1/c2/c3/... o1 o2 o3 ... where c1, c2, c3, ... specify the command path (through menus and sub-menus) and o1, o2, o3... are the options associated with the command. Module can navigate the command path and options using the methods JobCommand::PopCmd(), JobCommand::PopOpt(), JobCommand::PopIntOpt(), JobCommand::PopFloatOpt(). For example,
UserModule::HandleCommand(JobCommand *cmd)
{
std::string c1 = cmd->PopCmd();
if (c1 == "Draw") {
std::string opt = cmd->PopOpt();
if (opt == "Top") this->DrawTopView();
else if (opt == "Side") this->DrawSideView();
else { ... print warning... }
}
if (c1 == "Dump") {
std::string c2 = c2 = cmd->PopCmd();
if (c2 == "Summary") {
std::string opt = cmd->PopOpt();
if (opt == "Muon") {...}
else if (opt == "Electron") {...code...}
}
else if (c2 == "Stats") {
...code...
}
}
}
handles the commands:
j.Path("A").Mod("UserModule").Cmd("Draw Top);
j.Path("A").Mod("UserModule").Cmd("Draw Side");
j.Path("A").Mod("UserModule").Cmd("Dump/Summary Muon");
j.Path("A").Mod("UserModule").Cmd("Dump/Summary Electron");
j.Path("A").Mod("UserModule").Cmd("Dump/Stats");
////////////////////////////////////////////////////////////////////////