Task #422
closedTask #421: Create an automatic installation package for DataMiner
Report libraries and configuration components for DataMiner
100%
Files
Related issues
Updated by Gianpaolo Coro almost 10 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 50
Here I report the passages required to build a fully working development-environment DataMiner installation from scratch.
1 - Install a SmartGears-enabled tomcat service, possibly on the 80 port or with a redirect to the 80 port. Use devsec as starting scope.
2 - Download the official 52 WAR application from the following link and put it under webapps:
http://build.dev.52north.org/jenkins/view/Processing/job/52n-wps_GIT-DEV_nightly_builds/lastSuccessfulBuild/org.n52.wps$52n-wps-webapp/artifact/org.n52.wps/52n-wps-webapp/3.3.2-20150523.010021-9/52n-wps-webapp-3.3.2-20150523.010021-9.war
3 - Substitute the 52n-wps-server-3.3.2-X.jar with the corresponding jar on our Maven gcube-externals repository:
(Repository: "gCube Externals") <dependency> <groupId>rapidminer-custom</groupId> <artifactId>52n-wps-server</artifactId> <version>3.3.2</version> </dependency>
4 - add the following maven library along with its dependencies to the wps/WEB-INF/lib/ folder of the wps application:
(Repository: "gCube Snapshots") <dependency> <groupId>org.gcube.dataanalysis</groupId> <artifactId>dataminer</artifactId> <version>1.0.0-SNAPSHOT</version> </dependency>
5 - create a folder named "persistence" under wsp/
6 - create a folder named "ecocfg" under wsp/
7 - copy all the files available at this SVN link into the ecocfg folder:
https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/cfg
8 - copy the PARALLEL_PROCESSING folder at this SVN link into the ecocfg folder (thus creating the PARALLEL_PROCESSING folder under ecocfg):
https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/PARALLEL_PROCESSING
9 - copy the following xml file into the wps/config folder:
https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/wpscfg/wps_config.xml
10 - substitute the hostname and the port inside the following tag of the previous xml file, with the correct indication of the hostname of the machine and of the port (80) of the tomcat:
<Server protocol="http" hostname="localhost" hostport="8080" includeDataInputsInResponse="false" computationTimeoutMilliSeconds="3600000" cacheCapabilites="false" webappPath="wps" repoReloadInterval="0.0" minPoolSize="10" maxPoolSize="20" keepAliveSeconds="1000" maxQueuedTasks="100">
11 - Put the gcube-app.xml and web.xml files, stored in the following SVN folder, into the WEB-INF wps application folder:
https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration/ghn
12 - a reference example of configured and working wps application can be found at this link:
http://goo.gl/rtbHpW
Updated by Pasquale Pagano almost 10 years ago
Can you add those instructions also on the gCube Admin guide?
Updated by Gianpaolo Coro almost 10 years ago
- % Done changed from 50 to 60
The installation instructions with a "related links" section is available at this address (linked by the Admin guide page)
https://wiki.gcube-system.org/index.php/DataMiner_Installation
These are meant to be used by the parent Task #421 to build an Ansible script that automates the installation.
Updated by Gianpaolo Coro almost 10 years ago
I forgot to indicate that there is required software to be installed on the machine to run all the algorithms.
This software and the related installation are reported in ticket #207
Updated by Gianpaolo Coro almost 10 years ago
- Related to Task #207: Document the R installation on the statistical-manager nodes. added
Updated by Gianpaolo Coro over 9 years ago
- Assignee changed from Gianpaolo Coro to Andrea Dell'Amico
- % Done changed from 60 to 80
The DataMiner installation Wiki is complete and needs to be tested on a new machine.
Updated by Gianpaolo Coro over 9 years ago
- Related to Task #1215: Install Dataminer in production environment added
Updated by Andrea Dell'Amico over 9 years ago
- Related to Task #1502: Automate the dataminer installation as a loadbalanced service added
Updated by Andrea Dell'Amico over 9 years ago
Gianpaolo Coro wrote:
2 - Download the official 52 WAR application from the following link and put it under webapps:
http://build.dev.52north.org/jenkins/view/Processing/job/52n-wps_GIT-DEV_nightly_builds/lastSuccessfulBuild/org.n52.wps$52n-wps-webapp/artifact/org.n52.wps/52n-wps-webapp/3.3.2-20150523.010021-9/52n-wps-webapp-3.3.2-20150523.010021-9.war
This war does not exist anymore. It does not seem an official release URL, it's named after what it seems a jenkins nightly build.
Updated by Andrea Dell'Amico over 9 years ago
I see that the latest official WPS release is 3.3.1: http://52north.org/downloads/category/15-wps
I'll look if I'm able to find the version you used.
Updated by Andrea Dell'Amico over 9 years ago
@gianpaolo.coro@isti.cnr.it The 52 north's Jenkins instance keeps the latest build only. The available build of the 3.3.2 development version is this one:
http://build.dev.52north.org/jenkins/job/52n-wps_GIT-DEV_nightly_builds/lastStableBuild/org.n52.wps$52n-wps-webapp/artifact/org.n52.wps/52n-wps-webapp/3.3.2-20150911.003446-11/52n-wps-webapp-3.3.2-20150911.003446-11.war
But I think that relying on development builds taken from jenkins is dangerous.
Updated by Andrea Dell'Amico over 9 years ago
The diff between the march and the september snapshots are visible here: http://build.dev.52north.org/jenkins/job/52n-wps_GIT-DEV_nightly_builds/lastStableBuild/changes and in more detail here: https://github.com/52North/WPS/compare/master...dev (Jenkins reports the changes from the latest build). It's only the quickstart.md file, that was added and then modified more than once.
I wasn't able to find a branch for the 3.3.1 version on github so the only way to compare the two versions is to download the 3.3.1 source and manually diff between the github dev branch.
Updated by Andrea Dell'Amico over 9 years ago
Andrea Dell'Amico wrote:
@gianpaolo.coro@isti.cnr.it The 52 north's Jenkins instance keeps the latest build only. The available build of the 3.3.2 development version is this one:
http://build.dev.52north.org/jenkins/job/52n-wps_GIT-DEV_nightly_builds/lastStableBuild/org.n52.wps$52n-wps-webapp/artifact/org.n52.wps/52n-wps-webapp/3.3.2-20150911.003446-11/52n-wps-webapp-3.3.2-20150911.003446-11.war
Never mind. Jenkins still lists that one as the latest build, but it's not available anymore.
Updated by Gianpaolo Coro over 9 years ago
@andrea.dellamico@isti.cnr.it I would like to work with 3.3.2, since I have seen the code and there as some functions I need there. If you strongly advice not to rely on that version, we could try with 3.3.1 anyway ans see what happens.
Updated by Andrea Dell'Amico over 9 years ago
Usint a pre 3.3.2 release is OK by me, the problem is using temporary builds that are volatile. Anyone who wants to follow your wiki guide would be stuck even before starting, right now.
We don't have a reliable download URL for the binary. Until an official 3.3.2 will be out a temporary solution is maybe put somewhere the war you downloaded and get it from there. Do we have a place? A workspace, maybe? @roberto.cirillo@isti.cnr.it @pasquale.pagano@isti.cnr.it?
Updated by Pasquale Pagano over 9 years ago
Yes, if a place has to be selected the storage with a public URI is advised.
Updated by Andrea Dell'Amico over 9 years ago
@gianpaolo.coro@isti.cnr.it do we have a pristine binary war that we can use?
Updated by Gianpaolo Coro over 9 years ago
Yes I have, I will provide it via Workspace pulic link asap and we will use that for the installation.
Updated by Gianpaolo Coro over 9 years ago
Here is the link to the war file:
http://data.d4science.org/uri-resolver/id?fileName=wps-3.3.2.war&smp-id=565d67b7e4b0eacf4a0fc5ad&contentType=application%2Fx-tika-java-web-archive
Updated by Andrea Dell'Amico over 9 years ago
- Related to Task #1503: Install a dev dataminer VM added
Updated by Andrea Dell'Amico over 9 years ago
Gianpaolo Coro wrote:
Here is the link to the war file:
http://data.d4science.org/uri-resolver/id?fileName=wps-3.3.2.war&smp-id=565d67b7e4b0eacf4a0fc5ad&contentType=application%2Fx-tika-java-web-archive
Thanks.
Updated by Andrea Dell'Amico over 9 years ago
All the subversion links need to be changed from
https://svn.d4science.research-infrastructures.eu/gcube/trunk/
to
https://svn.research-infrastructures.eu/public/d4science/gcube/trunk/
to allow non authenticated users. This is also true for the wiki page.
Updated by Andrea Dell'Amico over 9 years ago
@gianpaolo.coro@isti.cnr.it About the dataminer installation, maybe you want to add to the wiki the procedure to install the jar files from our maven repository and their dependencies?
mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:copy-dependencies -Dartifact=org.gcube.dataanalysis:dataminer:1.0.0-SNAPSHOT -DoutputDirectory=/home/gcube/tomcat/webapps/wps/WEB-INF/lib
But it needs a POM. I generated a fake one and uploaded it to the target machine, into the directory from where the command is executed:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.gcube</groupId> <artifactId>dataminer</artifactId> <packaging>jar</packaging> <version>1.0.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>org.gcube.dataanalysis</groupId> <artifactId>dataminer</artifactId> <version>1.0.0-SNAPSHOT</version> </dependency> </dependencies> </project>
Updated by Andrea Dell'Amico over 9 years ago
- File pom.xml pom.xml added
- Status changed from In Progress to Closed
- % Done changed from 80 to 100
- Infrastructure Pre-Production, Production added
The working pom.xml file that installs the dataminer dependencies correctly is attached.