Project

General

Profile

Actions

Task #5451

closed

Task #5448: Dataminer - Need to align R configuration with RStudio simulation environment

Ensure that dependent libraries invoked by R have same versions in Dataminer / RStudio environment

Added by Emmanuel Blondel over 8 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Category:
High-Throughput-Computing
Start date:
Oct 07, 2016
Due date:
% Done:

100%

Estimated time:
Infrastructure:
Development, Pre-Production, Production

Description

Non exhaustive inventory of libraries (to complete by R-related system admins probably):

  • CURL
  • GDAL
  • libxml
  • GEOS
Actions #1

Updated by Pasquale Pagano over 8 years ago

  • Tracker changed from Support to Task
  • Category set to High-Throughput-Computing
  • Assignee set to Andrea Dell'Amico
  • Target version changed from UnSprintable to Dataminer and StatMan provisioning
Actions #2

Updated by Andrea Dell'Amico over 8 years ago

The dataminer all have the same set of R packages installed.
On the rstudio/rstudio-dev servers the R installed packages are different.

I need a bit of work to merge the two lists, but after that the packages will always be the same everywhere.

Actions #3

Updated by Andrea Dell'Amico over 8 years ago

  • Status changed from New to In Progress
Actions #4

Updated by Andrea Dell'Amico over 8 years ago

  • % Done changed from 0 to 50
  • Infrastructure Pre-Production added

The R packages update is running right now.

I'm listing the debian packages that provide libraries needed by some of the packages, the R packages installed from debian packages, the R packages installed from CRAN mirrors, and the R packages installed from github.
Other packages, like curl/libcurl/libxml, are always installed on our system or were installed as dependencies from other packages.
It's a raw list directly taken from the provisioning configuration.

r_distribution_required_packages:
  - gdal-bin
  - dans-gdal-scripts
  - libgdal1-dev
  - libgeos-dev
  - libspatialite-dev
  - proj-bin
  - proj-data
  - libproj-dev
  - texlive
  - texlive-base
  - texlive-binaries
  - texlive-latex-base
  - texlive-latex-extra
  - texlive-formats-extra
  - texlive-xetex
  - libnetcdf-dev
  - netcdf-bin
  - mono-runtime
  - libmono-microsoft-visualbasic10.0-cil
  - gsl-bin
  - libgsl0-dev
  - mesa-common-dev
  - libglu1-mesa-dev
  - postgis
  - postgresql-client
  - jags

r_base_packages_list:
  - r-recommended
  - r-base
  - r-base-dev
  - r-base-html
  - r-base-core
  - r-doc-html
  - r-cran-boot
  - r-cran-cluster
  - r-cran-foreign
  - r-cran-kernsmooth
  - r-cran-lattice
  - r-cran-mgcv
  - r-cran-nlme
  - r-cran-rpart
  - r-cran-survival
  - r-cran-mass
  - r-cran-class
  - r-cran-nnet
  - r-cran-spatial
  - r-cran-codetools
  - r-cran-matrix

r_plugins_list_to_install:
  - { name: 'data.table', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'doBy', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'multcomp', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mvtnorm', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'survival', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'splines', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'TH.data', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'Matrix', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'abind', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'baysemix', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'coda', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'R2jags ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'R2WinBUGS', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rjags ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'runjags ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'maptools', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'sp', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'compiler', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'datasets', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'graphics', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'grDevices', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'grid', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'methods', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'nnet', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'parallel', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rpart', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'stats', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'stats4', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'tools', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'utils', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'withr', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'Rcpp ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'plyr ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'digest ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'httr ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'stringr ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'reshape2 ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'R2HTML ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'snow ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'RcppEigen ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'minqa ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'lme4 ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'RCurl ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'memoise ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'whisker ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'evaluate', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'devtools ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rgdal ', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rgeos', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'knitr', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rworldmap', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'spam', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'maps', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'fields', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'highlight', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'dichromat', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'colorspace', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'munsell', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'labeling', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'scales', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'gtable', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'ggplot2', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'jsonlite', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'magrittr', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'gridExtra', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'viridis', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'base64enc', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'htmltools', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'htmlwidgets', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'plotly', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mapdata', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mapplots', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mnormt', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'numDeriv', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'sn', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mclust', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'tclust', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'modeltools', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'flexmix', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'prabclus', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'diptest', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'DEoptimR', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'robustbase', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'kernlab', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'trimcluster', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'fpc', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'RJSONIO', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'cowplot', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'DBI', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'RPostgreSQL', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'raster', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'xtable', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'reshape', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'kobe', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'shiny', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'ncdf4', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'lubridate', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'dplyr', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'tidyr', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'gam', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rgl', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'readxl', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'geosphere', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'shapefiles', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'rmarkdown', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'igraph', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'curl', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'caret', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'XML', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'gdata', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'earth', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'plotrix', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'bitops', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'sandwich', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'googleVis', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'gsl', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'copula', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'mgcv', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'triangle', repo: '{{ r_cran_mirror_site }}' }
  - { name: 'FLa4a', repo: 'http://flr-project.org/R' }

r_plugins_from_sources:
  - { name: 'ncdf', version: '1.6.9', source: 'ncdf_1.6.9.tar.gz', url: 'https://cran.r-project.org/src/contrib/Archive/ncdf/ncdf_1.6.9.tar.gz' }

r_plugins_from_github:
  - { plugin_name: 'RFigisGeo', github_user: 'openfigis' }
  - { plugin_name: 'rsdmx', github_user: 'opensdmx' }
  - { plugin_name: 'R.methodsS3', github_user: 'HenrikBengtsson' }
  - { plugin_name: 'R.oo', github_user: 'HenrikBengtsson' }
  - { plugin_name: 'digest', github_user: 'eddelbuettel' }
  - { plugin_name: 'R.utils', github_user: 'HenrikBengtsson' }
  - { plugin_name: 'R.cache', github_user: 'HenrikBengtsson' }
  - { plugin_name: 'repmis', github_user: 'christophergandrud' }
  - { plugin_name: 'pryr', github_user: 'hadley' }
  - { plugin_name: 'RColorBrewer', github_user: 'cran' } 
  - { plugin_name: 'rCharts', github_user: 'ramnathv' } 
  - { plugin_name: 'IRDTunaAtlas', github_user: 'juldebar' } 
  - { plugin_name: 'FLCore', github_user: 'flr' } 

Actions #5

Updated by Andrea Dell'Amico over 8 years ago

  • Status changed from In Progress to Feedback
  • % Done changed from 50 to 100

The R packages and their dependencies are now aligned on all the dataminer and rstudio servers.

Actions #6

Updated by Andrea Dell'Amico over 8 years ago

We are going towards a continuous packages synchronization, see #4371.

Actions #7

Updated by Andrea Dell'Amico over 8 years ago

  • Status changed from Feedback to Closed
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)