Error generating new gas files or accessing gas definitions from a rml file

Elisa · February 19, 2020, 3:44pm

REST version : v2.2.14
REST commit : c72d76f9

I am trying to generate new gas mixtures. I have read the documentation of the TRestGas.cxx class where it is explained how to do it, I have also checked this tutorial, and finally the definitions of the gases already available at the server here: https://sultan.unizar.es/gasFiles/gases.rml

I created a new rml file where I define the gas mixtures as:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<gases>
<TRestGas name="Xenon-CO2 1Pct 10-10E3Vcm" title="Xenon-CO2 Mixture (1Pct CO2)">
    <parameter name="pressure" value="1"/>
    <parameter name="temperature" value="293.15"/>
    <parameter name="maxElectronEnergy" value="400"/>
    <parameter name="W_value" value="22.12"/>
    <parameter name="nCollisions" value="10"/>
    <eField Emin="10" Emax="1000." nodes="20"/>
    <gasComponent name="xe" fraction="0.99"/>
    <gasComponent name="co2" fraction="0.01"/>
</TRestGas>
</gases>

But when I try to create a new instance of a TRestGas with this rml file, I get the following error:

root [0] TRestGas *gas1 = new TRestGas("/home/eruizchl/rest_g4data/gasMixtures/gasDefinitions.rml", "Xenon-CO2 1Pct 10-10E3Vcm", true)
    -- Error : Filename : null
    -- Error : Config File does not exist. Right path/filename?
    Press a KEY to continue ...

I have no problem using the “server” option and accessing the already generated gas files, but for some reason I cannot use a gas definition from my local computer. I cannot use the rml that comes with REST installation either (REST_v2/data/definitions/gases.rml).

Any thoughts?

jgalan · February 19, 2020, 5:20pm

Hi,

this needs some re-validation since it seems to be not working as usual. I guess the problem is connected to the fact that LoadFromConfigFile inside the TRestGas constructor is called with null string value.

To solve that I have added fConfigFileName = cfgFileName; inside the constructor so that it gets the right input RML file, and loads it…

    if (strcmp(cfgFileName, "server") == 0) {
        LoadConfigFromFile(StringToElement("<TRestGas name=\"" + name + "\" file=\"server\"/>"), NULL);
    } else {
        fConfigFileName = cfgFileName;
        LoadConfigFromFile(fConfigFileName, name);
    }

However, once I re-compile and test by executing

TRestGas *gas2 = new TRestGas("gases.rml", "Xenon-TMA 1Pct O2 200.0ppm 10-10E3Vcm", true );

and I get the following output

-- Debug : Entering ... TRestGas::ConstructFilename( )
-- Debug : Constructed filename : xe_99.0-n(ch3)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas
-- Debug : TRestGas::InitFromConfigFile. ConstructFilename. fGasFilename = xe_99.0-n(ch3)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas
-- Debug : Entering ... TRestGas::FindGasFile( name=xe_99.0-n(ch3)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas )
-- Error : download failed!
Gas definition does NOT exist in database?
File name: https://sultan.unizar.es/gasFiles/xe_99.0-n\(ch3\)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas
Please specify a local file
-- Info : Trying to find the gasFile locally
-- Debug : TRestGas::InitFromConfigFile. FindGasFile. fGasFilename = /programas/REST/testing/data/download/xe_99.0-n(ch3)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas
-- Warning : TRestGas gasFile generation is enabled, but the gasFile already exists!!
-- Warning : fGasGeneration should be disabled to remove this warning.
-- Warning : If you really want to re-generate the gas file you will need to disable the gasServer.
-- Warning : And/or remove any local copies that are found by SearchPath.

where the gas mixture does not exist at sultan.unizar.es/gasFiles/, but it does in the local RML.

And surprisingly, It tells me that it found the file!! I tried to track the problem a little and this seems to happen when gDataBase->get_metadatafile is called inside FindGasFile method.

That call generates an empty file at /programas/REST/testing/data/download/xe_99.0-n(ch3)3_1.0-O2_200.0ppm-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_800.gas. It will be found then by TRestGas, and it will not launch the gas generation because he thinks the gas file is present.

@nkx I do not know exactly what is the reason to introduce the external database, when the idea was to centralise all gas file definitions into a common gas file database? Anyway, there are some issues with it, perhaps the origin of the problem is at DownloadRemoteFile?

nkx · February 19, 2020, 6:11pm

I see, wget -O will create an empty file even the source url is invalid. We shall remove the file if empty and if wget failed

Elisa · February 20, 2020, 10:34am

I implemented the small change, and now it seems to find the config file, but in my case no file is generated in such folder. I don’t even have the “data” folder inside the folder “testing”.

I don’t get what you are trying to tell, sorry! I don’t know what this “wget -0” is. The file that I am imputing should be valid, at least it is following the same structure as the gases.rml file that was already in REST. And I am not getting any empty file as a result…

jgalan · February 20, 2020, 11:56am

I can see at the GitLab repository that Kaixiang has already pushed that change to the repository.

/programs/REST/testing/ is my $REST_PATH you should have a $REST_PATH/data.

I think I found another solution, perhaps just for the case when we want to generate a gas file.

Then, it starts the gas generation, what do you think?

I imagine the case where we use “server” and the gas file does not exist in sultan is not covered here. Since the download will fail. In any case, it will fail and we will realize that it has to generate the gas. I hope.

That change is in the following commit.

Please @nkx could you enable clang auto-formatting on your development environment? I just changed one line in the previous commit, all other changes are re-formating according to clang rules defined at $REST_SOURCE/.clang-format.

Elisa · February 20, 2020, 3:27pm

Ok thanks! but anyway, I don’t get any downloaded file in $REST_PATH/data/downloads either.

I applied the new commit, but the gas is still not generated. I get these warnings:

root [0] TRestGas *gas1 = new TRestGas("/home/eruizchl/REST_v2/install/master/data/definitions/gasesElisa.rml", "Xenon-CO2 1Pct 10-10E3Vcm", true);
-- Warning : TRestGas gasFile generation is enabled, but the gasFile already exists!!
-- Warning : fGasGeneration should be disabled to remove this warning.
-- Warning : If you really want to re-generate the gas file you will need to disable the gasServer.
-- Warning : And/or remove any local copies that are found by SearchPath.

I checked and there isn’t any copy of any gas file at the searchPath file that I secified at the rml file, or at the $REST_PATH/data/downloads. Also, I always make sure to do make install when I do changes in my config files so they are applied at the installed version.

Which leaves me with a couple questions:

You can specify a searchPath at the globals section of the config file. Is this the path where REST searches for already generated .gas files? Shall it point to a specific folder?
When you generate a new gas file, is it generated at any folder in particular? How do you specify it?
It seems that the gasServer needs to be disabled in order to generate new files… how do you do it?

Thank you!

jgalan · February 20, 2020, 10:13pm

Ok, anyway it seems to find it. Adding this line after the output “but the gasFile already exists” should give a clue where it finds the file.

I pushed the change to the repo.

Gas server can be fully disabled using

    <globals>
        <parameter name="gasServer" value="none" />
    </globals>

as described inside file data/definitions/gases.rml.

But it would be preferrable to use always server. Just find the file that is blocking the gas generation. With the new output it should come out in screen.

nkx · February 21, 2020, 3:46am

I already enabled it. But sometimes the formatting is not automatic. I need to manually call for auto-formatting

jgalan · February 21, 2020, 7:54am

Perhaps the best would be that it is automatically done when we do “git commit”. Not sure if there is a solution for that.

Perhaps an alias of “git push” command could do that previously?

Elisa · February 21, 2020, 12:55pm

Thank you Javier! Now it is working

Elisa · February 25, 2020, 12:11pm

I have another question about this topic. When I generate the gas file, at some point I get this message:

ssh_dispatch_run_fatal: Connection to 155.210.94.115 port 22: Connection timed out
lost connection
-- Error : virtual int TRestDataBase::set_metadatafile(int, std::string)
-- Error : problem copying gases definitions to remote server
-- Error : Please report this problem at http://gifna.unizar.es/rest-forum/
error
gasUser@sultan.unizar.es's password:

Since I didn’t know the password, I pressed enter to skip this step, but it seems that this way the file is not saved locally either… Is it mandatory to upload the new gas file to the server? In that case, what is the password of gasUser?

Thank you!

jgalan · February 25, 2020, 3:48pm

Hi Elisa, most likely I need to grant you permissions to upload to the server. I can do that for any PC you wish to use to upload gases to the server.

You have several systems associated to your GitLab account, which one do you want to have write access to the gas server?

jgalan · February 25, 2020, 4:01pm

If, gasServer is set to none as mentioned in a previous post, then it should write the gas file and do not try to upload the gas to the server.

As it can be seen in TRestGas::GenerateGasFile() method, it writes the file at fGasOutputPath.

fGasOutputPath is a parameter that is retrieved in TRestGas::InitFromConfigFIle. By default is wherever you launch TRestGas.

jgalan · February 27, 2020, 9:50pm

Hi Elisa, did you succeed to generate and upload a gas file?

I found myself problems with the default Garfield version that is installed using the script ./installGarfield.sh, and I found that the following Garfield commit: 21891449, worked for me.

I updated ./installGarfield.sh, on the development v2.2.14_dev branch, so that it installs by default that Garfield commit

Be aware also, that in the attempt to upload a gasFile to the server you will be asked to add it to your known_hosts list, so you need to answer yes there, only for the first time.

Elisa · February 28, 2020, 10:23am

Hi Javier,

Sorry I didn’t read the first comment. I would like to have permissions with the “eruizchl@etap-accuracy” PC, please

By switching the gasServer to none I was able to successfully generate the new gas mixture. About the garfield installation, I think in the end the admin installed it manually, not using the installation script, so I am not entirely sure what commit did he use.

When you confirm me I have permissions, I’ll try to generate a new mixture with the server option on to make sure I can do it. Let me know!

jgalan · February 28, 2020, 10:39am

Yes, that PC got already permissions to upload. You may try.

Make sure to get the latest commit, I uploaded it right now.

Elisa · March 4, 2020, 10:08am

I tried to generate a gas file with the server option activated, and it both generated the gas and uploaded it to the server. I got these errors at the end of the process:

Writing gas file :
-----------------
Path : /home/eruizchl/REST_v2/install/master/data/download
Filename : xe_99.0-co2_1.0-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_400.gas
error
gases.rml
error
xe_99.0-co2_1.0-E_vs_P_10.0_1000.0_nodes_20-nCol_10-maxE_400.gas

… but I don’t know what they mean because everything worked fine.

I checked the server gas file and the gas was added here as well. However, it saves some local parameters I used such as

<TRestGas name="Xenon-CO2 1Pct 10-10E3Vcm" title="Xenon-CO2 Mixture (1Pct CO2)">
...
<searchPath value="/home/eruizchl/REST_v2/install/master/data/gasFiles/>
<parameter name="gasOutputPath" value="/home/eruizchl/REST_v2/install/master/data/download"/>
</TRestGas>

I understand these options are not general so I should not use them to generate the gas, correct?

jgalan · March 4, 2020, 11:20am

Probably is nothing then, I cannot find that “error” output on my latest pull to the code. Probably was some intermediate output message. Let us know if it appears in future, i.e. with the latest v2.2.14_dev version, it should not.

No, you should avoid them when generating the gasFiles to be uploaded to the server. I believe those parameters go generally in the <globals> section, you should remove them and provide a full/relative path to your local RML definition when calling TRestGas.

I have manually removed the searchPath statement from the gasServer repository. But if you remove it next times from your local RML, the gas generation should be working normally, and it will generate a clean definition.