System : Linux
REST version : v2.3.15 commit 74400602
Root version : 6.26/10
Geant4 version : 11.0.3
Hi everyone,
I have questions concerning the multi-threading mode of restG4.
For simplicity I’m using the example found here.
1- In the restG4 Readme it’s written “Using the same seed and same number of threads produces the same results.”
As you can see in basicSim.rml, the number of seed is setup.
But, when I’m running 2 consecutive times restG4 basicSim.rml -t 2, I have two different results.
See here the two TotalDepositedEnergy spectra of the two restG4 runs:
2- Another thing that could be related (?): you see the number of entries in the histograms are 1001.
However, in basicSim.rml we have set <parameter name="nRequestedEntries" value="1000"/>.
During the restG4 command runs, we have the terminal output:
========================== Begin of Run Action ========================
Events to be simulated: 2147483647
Requested number of entries in file: 1000
=======================================================================
25.60% | 256 events stored / 2568 processed (1.28e+02/s) | 256 entries / 1000 requested | 2.00 seconds elapsed
53.80% | 538 events stored / 5059 processed (1.31e+02/s) | 538 entries / 1000 requested | 4.10 seconds elapsed
69.60% | 696 events stored / 6426 processed (1.14e+02/s) | 696 entries / 1000 requested | 6.10 seconds elapsed
89.50% | 895 events stored / 8391 processed (1.10e+02/s) | 895 entries / 1000 requested | 8.10 seconds elapsed
G4WT1 > Stopping Run! We have reached the number of requested entries (1000)
100.10% | 1001 events stored / 9271 processed (9.91e+01/s) | 1001 entries / 1000 requested | 10.10 seconds elapsed
============================= Run Summary =============================
1001 events stored out of 9271 simulated events
=======================================================================
It’s like the simulation doesn’t stop right away after reaching the requested number of event.
I think problems 1 and 2 could be related because this number of extra events is not constant with successive runs of restG4 basicSim.rml -t n.
Of course I tested without the multithreading option and restG4 behaves as expected: same seed = same result.
I also have the impression that the higher the number of thread requested, the higher the number of these “extra events”.
Indeed, one output when running restG4 basicSim.rml -t 6:
========================== Begin of Run Action ========================
Events to be simulated: 2147483647
Requested number of entries in file: 1000
=======================================================================
40.00% | 400 events stored / 3901 processed (1.94e+02/s) | 400 entries / 1000 requested | 2.06 seconds elapsed
71.80% | 718 events stored / 6935 processed (1.77e+02/s) | 718 entries / 1000 requested | 4.06 seconds elapsed
G4WT0 > Stopping Run! We have reached the number of requested entries (1000)
100.60% | 1006 events stored / 9463 processed (1.66e+02/s) | 1006 entries / 1000 requested | 6.06 seconds elapsed
100.60% | 1006 events stored / 9463 processed (1.25e+02/s) | 1006 entries / 1000 requested | 8.06 seconds elapsed
100.60% | 1006 events stored / 9463 processed (1.00e+02/s) | 1006 entries / 1000 requested | 10.06 seconds elapsed
============================= Run Summary =============================
1006 events stored out of 9463 simulated events
=======================================================================
Yes you are right, when using the requested number of events as a stop condition, once this number is reached, the simulation is signaled to stop, but since communication between threads is not immediate, extra events may have been simulated in other threads and cannot (easily) be removed, so this is why extra events appear. This means that the requested number of entries parameter may generate additional events in the multithreading mode.
However the random seed should work for multithreading mode (perhaps not exactly when requesting a number of events, but should only differ in event number). I will do some checks, could you try to use a single thread (-t 1) and do the same simulation without multithreading, and see if the seed works in this case?
no multithreading: as you can see in my original post, I already tried without multi-threading by running the command restG4 basicSim.rml, and indeed it behaved as expected, meaning that one seed corresponds to one set of simulated events.
single thread: restG4 basicSim.rml -t 1: the simulation outputs are different from one another with the same seed.
Personally I use the multithreading mode a lot but not for production results. The MT mode is faster in order to get a understanding of the simulation, but in order to generate the final simulation I would recommend to use the serial mode a submitting multiple jobs into a cluster if available. As you can see it’s difficult to validate the multithreading implementation is fully reliable, but it can generally be trusted.
