Support #11021
closedpb algo data miner
100%
Description
I have the attached pb (logs) when I execute this algo:
Seems to be linked to the following R script:
cat("setting working directory\n")
working_directory_init=getwd()
cat("setting working directory",working_directory_init,"\n")
setwd(working_directory_init)
system("chmod -R 777 ./")*
Thanks,
Paul
Files
Updated by Paul Taconet over 7 years ago
I do not know. If you try to execute the algo through the DM you will see that it does not work (it returns an error).
The execution of the script seems to stop at its very beginning, ie after the line system("chmod -R 777 ./")
Updated by Giancarlo Panichi over 7 years ago
- Assignee changed from Giancarlo Panichi to Paul Taconet
Please @paul.taconet@ird.fr , add some logs to your code because I can not figure out where the algorithm stops, and republish it.
Updated by Paul Taconet over 7 years ago
Ok, I have added logs and republished the script
Updated by Paul Taconet over 7 years ago
Following the logs, the problem seems to be located in these lines of the R script (i.e. where I set the default values for the parameters):
cat("setting values to parameters\n")
include_IOTC<-TRUE
include_ICCAT<-TRUE
include_IATTC<-TRUE
include_WCPFC<-TRUE
include_CCSBT<-TRUE
datasets_year_release<-"2017"
iccat_include_type_of_school<-TRUE
iattc_raise_flags_to_schooltype<-TRUE
iattc_dimension_to_use_if_no_raising_flags_to_schooltype<-"flag"
mapping_map_code_lists<-TRUE
mapping_csv_mapping_datasets_url<-"http://data.d4science.org/ZWFMa3JJUHBXWk9NTXVPdFZhbU5BUFEyQnhUeWd1d3lHbWJQNStIS0N6Yz0"
mapping_keep_src_code<-FALSE
gear_filter<-"NULL"
unit_conversion_convert<-FALSE
unit_conversion_csv_conversion_factor_url<-"http://data.d4science.org/Z3V2RmhPK3ZKVStNTXVPdFZhbU5BTTVaWnE3VFAzaElHbWJQNStIS0N6Yz0"
unit_conversion_codelist_geoidentifiers_conversion_factors<-"areas_conversion_factors_numtoweigth_ird"
raising_georef_to_nominal<-TRUE
aggregate_on_5deg_data_with_resolution_inferior_to_5deg<-TRUE
disaggregate_on_5deg_data_with_resolution_superior_to_5deg<-"none"
disaggregate_on_1deg_data_with_resolution_superior_to_1deg<-"none"
spatial_curation_data_mislocated<-"remove"
overlapping_zone_iattc_wcpfc_data_to_keep<-"IATTC"
SBF_data_rfmo_to_keep<-"CCSBT"
metadata_title<-"Global monthly catch of tuna, tuna-like and shark species (%year_start%-%year_end%) aggregated by %spatial_resolution%° squares"
metadata_contact_owner<-"ird@ird.fr"
metadata_contact_metadata<-"paul.taconet@ird.fr@@julien.barde@ird.fr@@emmanuel.blondel@fao.org"
metadata_contact_pointofcontact<-"paul.taconet@ird.fr@@julien.barde@ird.fr@@emmanuel.blondel@fao.org"
metadata_contact_principalinvestigator<-"emmanuel.chassot@ird.fr"
metadata_contact_publisher<-"ird@ird.fr"
metadata_contact_processor<-"paul.taconet@ird.fr"
metadata_description<-"This dataset lists global catch of tuna, tuna-like and shark species from %year_start% to %year_end%. Catches are stratified by month, species, gear, vessel flag reporting country, fishing mode (i.e. type of school used), area (mainly 5° square) and unit of catch (weight or number). This dataset was computed using public domain catch-and-effort datasets released by the five tuna Regional Fisheries Management Organizations: the Indian Ocean Tuna Commission (IOTC), the International Commission for the Conservation of Atlantic Tunas (ICCAT), the Inter-American Tropical Tuna Commission (IATTC), The Western and Central Pacific Fisheries Commission (WCPFC), the Commission for the Conservation of Southern Bluefin Tuna (CCSBT)."
metadata_source<-"IRD"
metadata_supplemental_information<-"IRD"
cat("setting values to parameters OK\n")
Updated by Emmanuel Blondel over 7 years ago
If i remember well, there is a weird adhoc "workaround" to make Dataminer understand the boolean parameters... https://wiki.gcube-system.org/gcube/Statistical_Algorithms_Importer:_R_Project_FAQ#Managing_Boolean_values Maybe it is what you are looking for...
Updated by Paul Taconet over 7 years ago
- File RLOGb8345554-8a50-446c-bf59-aec05145675a.txt RLOGb8345554-8a50-446c-bf59-aec05145675a.txt added
Thanks Emmanuel, the pb was actually the one you have pointed out.
I have another question: I would like to use within my script the functions to interact with the WS (the ones located here: https://wiki.gcube-system.org/gcube/Workspace_Interaction_From_R). For this I need to retrieve somewhere the username and the token.
When I look at the logs (see attached) I see that the first lines are:
gcube_token<<-"0b999a4f-f160-4697-9166-b68e0071f128-843339462"
gcube_username<<-"paultaconet"
gcube_context<<-"/d4science.research-infrastructures.eu/gCubeApps/RPrototypingLab"
So I have assumed that these variables are created and filled with the right values. However when I try to use them in my script, it seems they do not exist (see logs attached)
Error in push_R_script_to_server(metadata_and_parameterization, virtual_repository_with_R_files, :
object 'gcube_username' not found
Can you indicate me how I can retrieve the gcube username and token and use them in the scripts that I import in the SAI?
Thanks,
Paul
Updated by Paul Taconet over 7 years ago
The line in my R script is:
metadata_and_parameterization<-push_R_script_to_server(metadata_and_parameterization,virtual_repository_with_R_files,gcube_username,gcube_token)
here I need as values for gcube_username and gcube_token the username and the token of the user that is executing the algo through the Data miner in the VRE
Updated by Giancarlo Panichi over 7 years ago
- Tracker changed from Incident to Support
Please @paul.taconet@ird.fr read the documentation for this case:
Updated by Paul Taconet over 7 years ago
Thanks for the documentation.
However, the logs indicate that the object "gcube_username" does not exist (object 'gcube_username' not found) (https://support.d4science.org/attachments/download/2105/RLOGb8345554-8a50-446c-bf59-aec05145675a.txt , while according to the documentation, this object should exist.
Updated by Gianpaolo Coro over 7 years ago
Hi Paul, I guarantee the variables are there because they are used extensively by other algorithms (also some we are currently running every day).
However, global variables are deleted if you use cleaning instruction such as rm(list=ls(all=TRUE)) at the beginning of your code. These instructions are not necessary on the DataMiner since when an algorithm runs it is always executed in a completely new environment.
Updated by Paul Taconet over 7 years ago
- File RLOGe73ac71e-ec9a-4672-9ff3-a7f9fc3bf5f2.txt RLOGe73ac71e-ec9a-4672-9ff3-a7f9fc3bf5f2.txt added
Thanks GianPaolo. I removed the rm(list=ls(all=TRUE)) at the beginning of my code and I do not have that pb anymore.
However, I have a new pb linked to this.
This is what I do in my script:
- First I source an R file that stores a given function: source(paste(repository_R_scripts,"push_R_script_to_server.R",sep="/")) (the script is here: https://raw.githubusercontent.com/ptaconet/rtunaatlas_scripts/master/tunaatlas_world/workflow_etl/scripts/push_R_script_to_server.R)
in this function, I have (among others) the following lines:
source("http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r")
username<-vre_username
token<-vre_token
uploadWS(virtual_repository_with_R_files,RFileName,overwrite=T)
RFileURL <- getPublicFileLinkWS(paste(virtual_repository_with_R_files,RFileName,sep="/"))
- Then I execute the function with the following parameters: metadata_and_parameterization<-push_R_script_to_server(metadata_and_parameterization,virtual_repository_with_R_files,gcube_username,gcube_token)
In the logs (see attached) I get: Error in authenticate(username, token, type = "basic") :
object 'username' not found
It seems that the error comes from when I source the workspace_interaction script, i.e. : source("http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/RConfiguration/RD4SFunctions/workspace_interaction.r")
Can you indicate where I am doing something wrong?
Thanks,
Paul
Updated by Gianpaolo Coro over 7 years ago
Hi Paul,
I think you just miss to use a "<<-" instead of "<-":
Since they are global variables, I would use directly gcube_username and gcube_token. Thus, I would write
username<<-gcube_username
token<<-gcube_token
before calling the WS functions. Anyway, also in your code (with the vre_* variables) this should work. You should be done with the lines above.
Overall, I'm referring to the instructions in this page: https://wiki.gcube-system.org/gcube/Workspace_Interaction_From_R
Updated by Paul Taconet over 7 years ago
Thanks GianPaolo.
I have modified my code as you indicated and it is working now.
Thank you!
Updated by Pasquale Pagano over 7 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100