Project

General

Profile

Actions

Task #5590

closed

DataMiner as Generic Worker

Added by Gianpaolo Coro over 8 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Urgent
Category:
High-Throughput-Computing
Start date:
Nov 03, 2016
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Infrastructure:
Production

Description

DataMiner should substitute the Generic Workers already from r4.1.0. The old GW should be shutdown. Three DataMiner are required for the moment(with the machine features of the previous GWs). A new cluster is required too for these DMs named:

dataminer.genericworkers.d4science.org

Subtasks 2 (0 open2 closed)

Incident #5638: Check data availability through URI ResolverRejectedRoberto CirilloNov 03, 2016

Actions
Incident #5645: Service EP Retrieval issueClosedLucio LeliiNov 03, 2016

Actions

Related issues

Related to D4Science Infrastructure - Incident #5634: Wrong IC client library versions on NexusClosedGabriele GiammatteoNov 03, 2016

Actions
Actions #1

Updated by Andrea Dell'Amico over 8 years ago

Gianpaolo Coro wrote:

dataminer.genericworkers.d4science.org

dataminer-genericworkers.d4science.org

Actions #2

Updated by Andrea Dell'Amico over 8 years ago

Can you confirm that these are the VMs that we can destroy?

node3.d4science.org
node[11:13].d4science.org
node16.d4science.org
node18.d4science.org
node23.d4science.org
node24.d4science.org
node[27:38].d4science.org
node[43:45].d4science.org
node[47:56].d4science.org
node[60:61].d4science.org
node64.d4science.org
Actions #3

Updated by Gianpaolo Coro over 8 years ago

yes!

Actions #4

Updated by Andrea Dell'Amico over 8 years ago

So, 3GB of ram, 2 CPUs, 12 GBs of disk (but I'll make them bigger, the R packages and dataminer occupy a lot of space).

Same list of R packages? and quickrank too?

Actions #5

Updated by Gianpaolo Coro over 8 years ago

Yes to all.

Actions #6

Updated by Andrea Dell'Amico over 8 years ago

The expanded list of generic workers that will be terminated:

node3.d4science.org
node11.d4science.org
node12.d4science.org
node13.d4science.org
node16.d4science.org
node18.d4science.org
node23.d4science.org
node24.d4science.org
node27.d4science.org
node28.d4science.org
node29.d4science.org
node30.d4science.org
node31.d4science.org
node32.d4science.org
node33.d4science.org
node34.d4science.org
node35.d4science.org
node36.d4science.org
node37.d4science.org
node38.d4science.org
node43.d4science.org
node44.d4science.org
node45.d4science.org
node47.d4science.org
node48.d4science.org
node49.d4science.org
node50.d4science.org
node51.d4science.org
node52.d4science.org
node53.d4science.org
node54.d4science.org
node55.d4science.org
node56.d4science.org
node60.d4science.org
node61.d4science.org
node64.d4science.org
Actions #7

Updated by Andrea Dell'Amico over 8 years ago

One more question: can I have the list of scopes where those servers will be available?

Actions #8

Updated by Andrea Dell'Amico over 8 years ago

  • Status changed from New to In Progress

Hostnames and IPs are going to be:

dataminer1-genericworkers.d4science.org 146.48.122.39
dataminer2-genericworkers.d4science.org 146.48.122.86
dataminer3-genericworkers.d4science.org 146.48.122.151

Actions #9

Updated by Gianpaolo Coro over 8 years ago

These are services that should run in the production environment, thus the scopes should be the same as the other DataMiners.

Actions #10

Updated by Andrea Dell'Amico over 8 years ago

  • % Done changed from 0 to 80

Provisioning is running. When ready, I'll update the haproxy configuration.

Actions #11

Updated by Andrea Dell'Amico over 8 years ago

  • Status changed from In Progress to Feedback
  • % Done changed from 80 to 100

The three nodes are online. They are behind dataminer-genericworkers.d4science.org for both http and https.

Actions #12

Updated by Gianpaolo Coro over 8 years ago

Nothing responds to the indicated proxy (503 Service Unavailable), e.g.

http://dataminer-genericworkers.d4science.org/wps/WebProcessingService?Request=GetCapabilities&Service=WPS&gcube-token=26209520-6e2d-4850-9716-0b2bd37fd017

the single dataminers respond.

Actions #13

Updated by Andrea Dell'Amico over 8 years ago

Fixed. It works now.

Actions #14

Updated by Gianpaolo Coro over 8 years ago

I have some issues at testing, due to some IS client library problem.

Actions #15

Updated by Gianpaolo Coro over 8 years ago

  • % Done changed from 100 to 80
Actions #16

Updated by Andrea Dell'Amico over 8 years ago

The library has been replaced on all the dataminer servers.

Actions #17

Updated by Gianpaolo Coro over 8 years ago

  • Related to Incident #5634: Wrong IC client library versions on Nexus added
Actions #18

Updated by Gianpaolo Coro over 8 years ago

  • Status changed from Feedback to Closed
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)