Project

General

Profile

Actions

Task #5452

closed

Task #5448: Dataminer - Need to align R configuration with RStudio simulation environment

Is it possible to have a simple way for users to check that RStudio & Dataminer configs are aligned?

Added by Emmanuel Blondel over 8 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Category:
High-Throughput-Computing
Start date:
Oct 07, 2016
Due date:
Nov 04, 2016
% Done:

100%

Estimated time:
(Total: 0.00 h)
Infrastructure:
Development, Production

Description

Is it possible to have a simple way for users to check that RStudio & Dataminer configs are aligned?
As suggestion, could we have a simple Dataminer process, that execution will return the complete R config under Dataminer vs RStudio? This could include:

  • R session Info
  • dependent non-R libraries
  • available packages

This would be really useful for users, and to fasten troubleshooting of algorithms


Subtasks

Actions #1

Updated by Pasquale Pagano over 8 years ago

  • Tracker changed from Support to Task
  • Category set to High-Throughput-Computing
  • Assignee set to Gianpaolo Coro
  • Target version changed from UnSprintable to Dataminer and StatMan provisioning
Actions #2

Updated by Gianpaolo Coro over 8 years ago

Hi Emmanuel, your request is right and a process will be implemented to give you this information.
For the time being, please consider that we added an RStudio instance to the RPrototypeLab, which runs on a DataMiner service. This service is the one that hosts the deployed process and is a copy of the other Dataminers. Thus, in this case there is perfect alignment.

In my opinion, it would be very useful for you to switch to this VRE for your developments.

Actions #3

Updated by Pasquale Pagano over 8 years ago

  • % Done changed from 0 to 40

If I correctly understood, in one VRE, RPrototypingLab, this issue is not present. A more general solution has to be provide in order to check the alignment also in the other VREs.

Actions #4

Updated by Emmanuel Blondel over 8 years ago

Gianpaolo Coro wrote:

Hi Emmanuel, your request is right and a process will be implemented to give you this information.
For the time being, please consider that we added an RStudio instance to the RPrototypeLab, which runs on a DataMiner service. This service is the one that hosts the deployed process and is a copy of the other Dataminers. Thus, in this case there is perfect alignment.

In my opinion, it would be very useful for you to switch to this VRE for your developments.

All developments were done in this VRE. I've been using the RPrototypingLab VRE for Rstudio, SAI and Dataminer, as suggested by @pasquale.pagano@isti.cnr.it for taking advantage of the computing resources, and this where i had to do troubleshoot the algorithm which led to spot out the issue of R system locales misalignment #5290 ... That's why users need some quick way to check the R configuration, and exclude any problem of config when they debug algorithms.

Actions #5

Updated by Emmanuel Blondel over 8 years ago

Pasquale Pagano wrote:

If I correctly understood, in one VRE, RPrototypingLab, this issue is not present. A more general solution has to be provide in order to check the alignment also in the other VREs.

Maybe yes, maybe not: we don't have a way to check and assert that it is perfectly aligned. Until last friday, i was assuming R configs were aligned until i look deeper and spot out issue #5290 (but i spent time to figure out it).

Actions #6

Updated by Gianpaolo Coro over 8 years ago

Hi Emmanuel, are you sure you are not using the RStudioLab or the StockAssessment VRE? IRD was using these two.

The RStudio on the RPrototypeLab is exactly on the same machine of DataMiner, so there is no possibility to misalignment. In fact, if you go in the VRE (https://i-marine.d4science.org/group/rprototypinglab/rprototypinglab) and click on the RStudio button, in the browser address bar you will see "http://dataminer1-proto.d4science.org/". Instead, in the other VREs, you would see "http://rstudio.d4science.org/"

Please, let me know if you see this too.

Actions #7

Updated by Emmanuel Blondel over 8 years ago

I confirm myself and Levi have been running GRID-ARENDAL algorithm in RPrototypingLab VRE RStudio (for running in parallel the algorithm) + DataMiner. Possibly what you say about URLs seems something new (at least after friday, i see that there were changes in this VRE°, but what is sure is that i run all recent developments of GRID-ARENDAL in this new VRE (including those when i had to deal with sys.locale and encoding issues!). But you confirmed it in #5290 :

*"the system Locale is equal to "C" on DataMiner, whereas it is UTF-8 on RStudio. The DataMiners Locale should be aligned to RStudio."

Actions #8

Updated by Gianpaolo Coro over 8 years ago

The Dataminer has been recently restarted several times in the RPrototypingLab thus the locale has been reloaded. In the ticket I was referring to the other RStudio machine with respect to the Dataminers. Thus, on the RStudio in the RPrototypingLab the locale was already aligned. Maybe at that time the Dataminer had not been restarted yet.

Anyway, can you confirm that now the locales are OK?

Actions #9

Updated by Emmanuel Blondel over 8 years ago

Maybe not, certainly. I'm categoric on the tests i did and where i did it, on RPrototypingLab RStudio and Dataminer, on friday 30/09. The benefit is that it will avoid other users to deal with the same encoding issues (but troubleshooting has a cost). For now i can't confirm the locales are OK, better to confirm it by launching the process to be implemented in #5461.

Actions #10

Updated by Gianpaolo Coro over 8 years ago

  • Due date set to Nov 04, 2016

due to changes in a related task

Actions #11

Updated by Gianpaolo Coro over 8 years ago

  • Status changed from New to Closed

As reported in #5448 and #4371, DataMiner provisioning has been fully automatised, with DM services acting beyond a proxy service, which cannot be invoked directly. Further, R packages installation has been centralised and automatised in order to have the DMs always aligned with RStudio. I guess at this point this ticket has been satisfied in another way.
Misalignment should then considered issue when encountered.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)