How to use the new TRestDetector interface?

[jgalan]

Hi @nkx

could you give some early guidelines on how to use the new gDetector?

I am now processing a Geant4 MonteCarlo simulation, but I get a message saying

-- Warning : TRestDetector::ReadFile: no detector parameters stored in file

I imagine this is because no gDetector was generated by restG4 and it is not available at the .root output file. So, what would be the best way to proceed? Should restG4 create the TRestDetector object and write it to the output .root file?

Then processes will the later be used to update the members of the detector class? I.e. if I use TRestHitsToSignalProcess and I explicitly specify the sampling parameter, the sampling should be replaced at gDetector instance? If sampling is not given, then the sampling value will be retrieved from gDetector?

If this works like that we could have a new TRestDetectorSetupProcess where we define all the parameters we wish at the process and they are used to update the gDetector values…

It is this the way you though about TRestDetector?

[nkx]

I have updated the code to make restG4 to save gDetector parameters. Hopefully now there is no warning message when opening g4 file.

A simple use of gDetector is to provide parameters for processes. If there is sampling parameter stored in the input root file, if there is no sampling parameter inside rml, we can get it in processes like we always do, by calling TRestMetadata::GetParameter(string, string). This method is rewritten to get more than rml parameters. It finds the parameter from command line arguments, rml definitions, and gDetector parameters in sequence. Method GetDblParameterWithUnits(string, double), ReadAllParameters() also support to read gDetector parameters. For example:

// in restManager.cxx
int main(int argc, char* argv[]) {
    gDetector->SetParameter("sampling", "1s");
    ...
}

// inside process's code
void TRestHitsToSignalProcess::InitFromConfigFile() {
    cout<<fSampling<<endl;
    // gets sampling rate from command line arguments, rml definitions, and gDetector parameters in sequence
    fSampling = GetDblParameterWithUnits("sampling", 0.1); 
    cout<<fSampling<<endl;  // here we get fSampling=1000000. If we remove the line in restManager.cxx, we will get fSampling=0.1
    ...
}

void TRestSignalZeroSuppresionProcess::InitFromConfigFile() {
    cout<<fSampling<<endl;
    // gets all parameters in one line
    ReadAllParameters();
    cout<<fSampling<<endl;  // here we get fSampling=1000000
}

@jgalan

Then processes will the later be used to update the members of the detector class? I.e. if I use TRestHitsToSignalProcess and I explicitly specify the sampling parameter, the sampling should be replaced at gDetector instance?

Processes are more likely to get parameters. We don’t add all the parameters from rml to gDetector. If you really want to set parameter for gDetector, you can write it manually:

void TRestHitsToSignalProcess::InitFromConfigFile() {
    fSampling = GetDblParameterWithUnits("sampling", 0.1); 
    if (gDetector->GetParameter("sampling") == PARAMETER_NOT_FOUND_STR) {
        gDetector->SetParameter("sampling", ToString(fSampling) + "us");
    }
    ...
}

[jgalan]

@nkx

I have updated the code to make restG4 to save gDetector parameters. Hopefully now there is no warning message when opening g4 file.

I compiled at sultan for testing it and I got a crash. It is strange, we usually had no problems compiling at sultan when the pipeline was passing without errors The compiler at sultan and at lfna are very similar.

jgalan@lfna:~$ g++ --version
g++ (Debian 6.3.0-18+deb9u1) 6.3.0 20170516

and

jgalan@sultan:~/git/REST_v2/build$ g++ --version
g++ (Debian 6.3.0-18+deb9u1) 6.3.0 20170516

crash report

jgalan@sultan:~/git/REST_v2/packages/restG4/examples/01.NLDBD$ restG4 
Warning in <UnknownClass::SetDisplay>: DISPLAY not set, setting it to 155.210.94.139:0.0

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f9fdd2f2ffa in __GI___waitpid (pid=14741, stat_loc=stat_loc
entry=0x7ffe841ae740, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f9fdd27a0ab in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2  0x00007f9fea3e2917 in TUnixSystem::StackTrace() () from /home/jgalan/apps/root-6.20.00/install/lib/libCore.so
#3  0x00007f9fea3e5064 in TUnixSystem::DispatchSignals(ESignals) () from /home/jgalan/apps/root-6.20.00/install/lib/libCore.so
#4  <signal handler called>
#5  0x00007f9fddab4b4a in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007f9fe4b6c69b in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > >::_M_get_insert_unique_pos(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestProcesses.so
#7  0x00007f9fded0742c in std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) [clone .isra.113] () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#8  0x00007f9fded08cd2 in REST_Reflection::TRestReflector::TRestReflector(void*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#9  0x00007f9fdecfef00 in TRestDetector::SetParameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#10 0x00007f9fdecfe77b in TRestDetector::SetDetectorName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#11 0x00007f9fded39e5c in __TRestDetector_Init::__TRestDetector_Init() () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#12 0x00007f9fdecf2c9e in __static_initialization_and_destruction_0(int, int) [clone .constprop.82] () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#13 0x00007f9fec2d679a in call_init (l=<optimized out>, argc=argc
entry=1, argv=argv
entry=0x7ffe841b1398, env=env
entry=0x7ffe841b13a8) at dl-init.c:72
#14 0x00007f9fec2d68ab in call_init (env=0x7ffe841b13a8, argv=0x7ffe841b1398, argc=1, l=<optimized out>) at dl-init.c:30
#15 _dl_init (main_map=0x7f9fec4ec170, argc=1, argv=0x7ffe841b1398, env=0x7ffe841b13a8) at dl-init.c:120
#16 0x00007f9fec2c7c5a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#17 0x0000000000000001 in ?? ()
#18 0x00007ffe841b2445 in ?? ()
#19 0x0000000000000000 in ?? ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f9fddab4b4a in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007f9fe4b6c69b in std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > >::_M_get_insert_unique_pos(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestProcesses.so
#7  0x00007f9fded0742c in std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, TDataType*> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&) [clone .isra.113] () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#8  0x00007f9fded08cd2 in REST_Reflection::TRestReflector::TRestReflector(void*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#9  0x00007f9fdecfef00 in TRestDetector::SetParameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#10 0x00007f9fdecfe77b in TRestDetector::SetDetectorName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#11 0x00007f9fded39e5c in __TRestDetector_Init::__TRestDetector_Init() () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#12 0x00007f9fdecf2c9e in __static_initialization_and_destruction_0(int, int) [clone .constprop.82] () from /home/jgalan/git/REST_v2/install/lib/libRestTools.so
#13 0x00007f9fec2d679a in call_init (l=<optimized out>, argc=argc
entry=1, argv=argv
entry=0x7ffe841b1398, env=env
entry=0x7ffe841b13a8) at dl-init.c:72
#14 0x00007f9fec2d68ab in call_init (env=0x7ffe841b13a8, argv=0x7ffe841b1398, argc=1, l=<optimized out>) at dl-init.c:30
#15 _dl_init (main_map=0x7f9fec4ec170, argc=1, argv=0x7ffe841b1398, env=0x7ffe841b13a8) at dl-init.c:120
#16 0x00007f9fec2c7c5a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#17 0x0000000000000001 in ?? ()
#18 0x00007ffe841b2445 in ?? ()
#19 0x0000000000000000 in ?? ()
===========================================================

@nkx

    cout<<fSampling<<endl;
    // gets all parameters in one line
    ReadAllParameters();
    cout<<fSampling<<endl;  // here we get fSampling=1000000

Ok, here we get 1000000 because <parameter name="sampling" value="1us" /> is not defined at the RML?

@nkx

Processes are more likely to get parameters. We don’t add all the parameters from rml to gDetector. If you really want to set parameter for gDetector, you can write it manually:

Ok, but perhaps if would be interesting that we are allowed to initialise the parameters of TRestDetector at the RML.

So, that we can create a new section in our restG4 RML file.

<TRestDetector sampling="40ns" shaping="1us" .... />

That will initialise the values from gDetector.

[nkx]

@jgalan

I compiled at sultan for testing it and I got a crash. It is strange,

I find out the problem. The initialization sequence of global variables is random if we place them in different cpp file. If they have dependency on each other, it will cause seg.fault. I have placed them all in one file now.

@jgalan

Ok, here we get 1000000 because <parameter name="sampling" value="1us" /> is not defined at the RML?

Yes. If you define it in rml, then you will get fSampling=1. rml parameters is in prior.

@jgalan

Ok, but perhaps if would be interesting that we are allowed to initialise the parameters of TRestDetector at the RML.

We can always define such shared parameters in the <globals section. Maybe some specification is useful in rml to flush those parameters to gDetector and make them saved to file? A problem is that gDetector is not a metadata class. It doesn’t follow TRestMetadata’s startup logic, which means it cannot read rml itself, which means we cannot use <TRestDetector section. Maybe we can add the new process TRestDetectorSetupProcess, or directly modify metadata TRestDetectorSetup. Then we can add it in rml to set global parameters to gDetector. For example:

<TRestManager name="CoBoDataAnalysis" title="Example" verboseLevel="info" >

  <TRestRun name="SJTU_Proto" >
    <parameter name="inputFile" value="auto" />
    <parameter name="runNumber" value="auto"/>//change this value to "auto" to enable database
    <parameter name="inputFormat" value="R[runNumber]_[runTag]_Vm[meshVoltage].aqs"/>
    <parameter name="outputFile" value="RUN[RunNumber].root" />

    <addMetadata name="PandaReadout_MxM" file="meta.root"/>
    <TRestGas name="Xenon-TMA 1Pct 10-10E3Vcm" pressure="10atm" electricField="1000V/cm" file="server"/>
    <TRestDetectorSetup name="P3Detector" mass="140kg"/>
  </TRestRun>

...

  <globals>
    <parameter name="sampling" value="0.2us" />
    <parameter name="pointThreshold" value="3" />
    <parameter name="pointsOverThreshold" value="3" />
  </globals>

In this rml, we set the parameter “mass”, “sampling”, “pointThreshold” and “pointsOverThreshold” to gDetector. How do you think? We can discuss tomorrow.

[jgalan]

@nkx

In this rml, we set the parameter “mass”, “sampling”, “pointThreshold” and “pointsOverThreshold” to gDetector. How do you think?

It is not crazy to pass arbitrary parameters at the globals section. But perhaps to be hosted by a new class with free parameters? I.e. TRestGlobalParameters? Lets discuss tomorrow.

[jgalan]

Hi @nkx, what is the state or evolution of TRestDetector?

I am starting to look back at the TREX-DM data to be analysed with v2.3.X. However, I need to identify the members inside the AQS filename with some values at the metadata members.

If I go to pipeline/trexdm/ and I run restManager --c 01_raw.rml --f R01928_tagTest_Vm380.aqs

Then, where it is stored the value of the mesh voltage?

I guess I need to add a TRestDetectorSetup definition?

Additionally, when I open the resulting file I get “unknown protocol”. I dont know what that means

[nkx]

You need to add a <TRestDetector section under <TRestRun to initialize detector feature storage. Then, you need to use “inputFormat” parameter to define the datamember.

Input file: R01928_tagTest_Vm380.aqs
Format: R[fRunNumber]_[fTag]_Vm[TRestDetector::fAmplificationVoltage]

The logic we agreed previously is implemented. If the field value is fXXX, it will try to fill TRestRun’s datamember. If the field value is TRestXXX::fXXX, it will try to fill the datamember of stored metadata object.

[jgalan]

Ok, great, I will try. What about the “unknown protocol”?

[nkx]

@jgalan

What about the “unknown protocol”?

I guess it is due to the old dataURL file. Now in the latest commit I removed that file in user space.

Updating dataURL at user HOME space

I found the following problem. When I updated the contents of data/dataURL file to complain with the new class names at v2.3 I found that they were not having any effect. This is because startup.cpp will copy that file to the user space at $HOME/.rest/ if it does not exist already. In the next make install, at the installation location /install/path/data/dataURL this file will be updated. However, it won’t be updated at the user space that is the one that is really used. So, do we need this fi…