D4Science Infrastructure

@andrea.dellamico@isti.cnr.it @tommaso.piccioli@isti.cnr.it Who is responsible for creating the machines for the production environment and run the installation scripts?

One load balancer and two dataminer servers also in production. Other dataminer servers can be added at a later time

Required specs for the load balancer:

• Hostname: dataminer-lb1-p-d4s.d4science.org (with dataminer.d4science.org as alias)
• Ubuntu 14.04 LTS
• 1 GB of RAM
• 2 virtual CPUs
• 10 GB disk

Required specs for the dataminer VMs, from the wiki page https://wiki.gcube-system.org/index.php/DataMiner_Installation:

• Hostname: dataminer[1:2]-p-d4s.d4science.org
• Ubuntu 12.04.5 LTS
• 6 GB of RAM
• 10 virtual CPUs
• 10 GB of HD space

dataminer1-p-d4s.d4science.org IP: 146.48.122.251
dataminer2-p-d4s.d4science.org IP: 146.48.123.64
dataminer-lb1-p-d4s.d4science.org IP: 146.48.123.71

Test on all the algorithms were successful using 4 workers behind the scenes. I communicated the main dataminer link (load balancer) to FishBase people.
I have two questions on the installation @andrea.dellamico@isti.cnr.it :
1 - is the Dataminer configuration downloaded on-the fly during the installation? In other words: will it always be updated?
2 - is it possible to copy the encryption keys (https://goo.gl/cSXOBq) also in the PARALLEL_PROCESSING folder?

Gianpaolo Coro wrote:

Test on all the algorithms were successful using 4 workers behind the scenes. I communicated the main dataminer link (load balancer) to FishBase people.
I have two questions on the installation @andrea.dellamico@isti.cnr.it :
1 - is the Dataminer configuration downloaded on-the fly during the installation? In other words: will it always be updated?

Nope. It was not requested, so the download of the svn files is done once. If the matter is to run a svn update at regular intervals it can be done easily. Otherwise, please give details.

2 - is it possible to copy the encryption keys (https://goo.gl/cSXOBq) also in the PARALLEL_PROCESSING folder?

Yes, I'll do.

Thank you Andrea,
the cfg and the PARALLEL_PROCESSING folders would be better downloaded from this SVN location in the next installations:

https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration

Gianpaolo Coro wrote:

Thank you Andrea,
the cfg and the PARALLEL_PROCESSING folders would be better downloaded from this SVN location in the next installations:

Do we need to perform a svn update regularly or not?

https://svn.d4science.research-infrastructures.eu/gcube/trunk/data-analysis/DataMinerConfiguration

I was using this (public path, to avoid authentication and problems with the untrusted TLS certificate):

http://svn.research-infrastructures.eu/public/d4science/gcube/trunk/data-analysis/DataMinerConfiguration/{cfg,PARALLEL_PROCESSING}

I don't think there's a difference between the private and the public link.

So I misunderstood. We do not need an automatic periodic update. Those folders need to be installed only at the beginning and should be updated only when a new release is available.
Furthermore, your link is good, it is the free-access read-only version of my link. If your script downloads that folder at each installation there is no problem.

Gianpaolo Coro wrote:

If your script downloads that folder at each installation there is no problem.

Yes. I only changed the svn link and you changed the wiki accordingly, so that it now points to the public access svn repository.

Perhaps the best would be having the production keys installed on the prod. servers and the development key on the dev. servers. From a technical point of view there is no issue if you copy all the keys on the services, since the scopes are assigned at infrastructure level and the belonging infrastructure depends on the GHN configuration.

Gianpaolo Coro wrote:

Perhaps the best would be having the production keys installed on the prod. servers and the development key on the dev. servers.

It's what I'm (the playbook is) doing.