D4science

Mar 14, 2016

05:41 PM D4Science Infrastructure Task #2509 (Closed): Add Ganglia monitoring to the cassandra cluster

Done.
No special cassandra support, because some reconfiguration is needed.
Andrea Dell'Amico

04:57 PM D4Science Infrastructure Task #2509 (In Progress): Add Ganglia monitoring to the cassandra cluster

The ganglia basic configuration is ready to be deployed. Andrea Dell'Amico

04:59 PM D4Science Infrastructure Task #2507 (Closed): Move the transect app from dlib18.isti.cnr.it to data.d4science.org

OK, I'm going to shutdown dlib18.isti.cnr.it.
Andrea Dell'Amico

01:07 PM D4Science Infrastructure Task #2507: Move the transect app from dlib18.isti.cnr.it to data.d4science.org

Ah OK, perfect.
Andrea Dell'Amico

12:30 PM D4Science Infrastructure Task #2507: Move the transect app from dlib18.isti.cnr.it to data.d4science.org

So we can install the transect app on data.d4science.org too, and change the dlib18.isti.cnr.it hostname to become an... Andrea Dell'Amico

04:45 PM D4Science Infrastructure Task #2583 (Closed): The cassandra services are not enabled at boot time

cassandra enabled at boot on all the servers.
Andrea Dell'Amico

04:40 PM D4Science Infrastructure Task #2583 (Closed): The cassandra services are not enabled at boot time

Both in dev and in production.
Andrea Dell'Amico

04:34 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

cassandra2-d has been restarted.
Let me know if it's all OK on the cluster.
Andrea Dell'Amico

04:11 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

The firewall rules are active on cassandra1-d-d4s. cassandra2-d-d4s needs a reboot to activate the firewall, because ... Andrea Dell'Amico

03:52 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

Massimiliano Assante wrote:
> the d4science gateways and the Elastic Search Cluster (Not yet in production, but exis... Andrea Dell'Amico

03:36 PM D4Science Infrastructure Task #2508 (In Progress): No firewall on the cassandra hosts

Andrea Dell'Amico

03:35 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

The new rules wil permit traffic between the cluster hosts on the ports: 7000, 7001.
Port 9042 and 9160 will be open... Andrea Dell'Amico

Mar 10, 2016

08:47 PM D4Science Infrastructure Task #2275: Try foreman with the ansible plugin as an ansible console

Also to investigate:
http://code-complete.com/code/?p=40 https://github.com/ansible-semaphore/semaphore semaphore,... Andrea Dell'Amico

08:45 PM D4Science Infrastructure Task #2275: Try foreman with the ansible plugin as an ansible console

There's a simple role to install all the required stuff.
Andrea Dell'Amico

06:44 PM D4Science Infrastructure Task #2275 (In Progress): Try foreman with the ansible plugin as an ansible console

Andrea Dell'Amico

05:54 PM D4Science Infrastructure Task #2544: Update Dataminer provisioning scripts

Gianpaolo Coro wrote:
> Please, update dataminer provisioning scripts with the following information (also available... Andrea Dell'Amico

04:52 PM D4Science Infrastructure Task #2527 (Closed): Configure the ldap backend to automatically remove the transaction logs

The script is installed on ldap-liferay, ldap-liferay-d, issue.research-infrastructures.eu. It will be activated by t... Andrea Dell'Amico

04:16 PM D4Science Infrastructure Task #2527 (In Progress): Configure the ldap backend to automatically remove the transaction logs

Andrea Dell'Amico

12:07 AM D4Science Infrastructure Task #2527 (Closed): Configure the ldap backend to automatically remove the transaction logs

We only need to set a variable inside /var/lib/ldap/DB_CONFIG it seems. Or better, run `db_archive` with the `-d` opt... Andrea Dell'Amico

04:33 PM D4Science Infrastructure Task #2547 (Closed): The backup service must run a post-script on the ldap servers

The script, called `/usr/local/bin/ldap_logs_cleaner`, will remove the unused transaction logs.
Andrea Dell'Amico

12:09 AM D4Science Infrastructure Support #2513 (Closed): The ldap server logs keep growing

A solution strategy is described in the related task.
Andrea Dell'Amico

Mar 09, 2016

11:50 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

Those are similar to the ones installed on the social-isti cluster.
We know which ports are actually used, so my pro... Andrea Dell'Amico

02:04 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

Some info about the ports used by cassandra:
https://docs.datastax.com/en/cassandra/2.0/cassandra/security/secureF... Andrea Dell'Amico

12:39 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

Massimiliano Assante wrote:
> We had firewalls on the previous Production Cassandra Cluster (which was dismissed las... Andrea Dell'Amico

12:36 PM D4Science Infrastructure Task #2508: No firewall on the cassandra hosts

Massimiliano Assante wrote:
> which cassandra hosts are you referring about? dev or production?
Both of them.
Andrea Dell'Amico

12:16 PM D4Science Infrastructure Task #2508 (Closed): No firewall on the cassandra hosts

The cassandra hosts are open to the world.
Do we know which hosts can connect and on what ports?
Andrea Dell'Amico

04:15 PM D4Science Infrastructure Task #2514 (Closed): Free dlib8x and dlib10x and assign them to the ganeti cluster

We need some more resources for the 'corporate' services.
Andrea Dell'Amico

04:12 PM D4Science Infrastructure Task #471 (In Progress): Locate the hypervisors to dedicate to OpenStack

Let's use the machines from dlib14x to dlib17x, then.
Andrea Dell'Amico

03:34 PM D4Science Infrastructure Support #2513 (Closed): The ldap server logs keep growing

openldap never cleans up its data log. We need to put a procedure in place.
Andrea Dell'Amico

02:08 PM D4Science Infrastructure Task #2192: Gcc and boost installation

Any news about this one?
Andrea Dell'Amico

12:25 PM D4Science Infrastructure Task #2509 (Closed): Add Ganglia monitoring to the cassandra cluster

The cassandra cluster isn't monitored by ganglia.
cassandra can export a lot of metrics, but it should be configur... Andrea Dell'Amico

10:55 AM D4Science Infrastructure Task #2507 (Closed): Move the transect app from dlib18.isti.cnr.it to data.d4science.org

It's something that were discussed by email.
dlib18.isti.cnr.it is a very old Ubuntu (7.10!) with an IP address on... Andrea Dell'Amico

Mar 08, 2016

12:50 PM D4Science Infrastructure Task #2227: test replication tools for postgres

The pgpool main configuration file is now a template, and we have a set of tasks to configure the authentication. Andrea Dell'Amico

Mar 07, 2016

07:58 PM D4Science Infrastructure Task #2227: test replication tools for postgres

pgpool is actually running on `postgres1-d-d4s.d4science.org`. It's configured to run on replication mode and the loa... Andrea Dell'Amico

Mar 04, 2016

06:04 PM D4Science Infrastructure Task #2469 (Closed): Install org.gcube.dataanalysis.dataminer.1.0.2

Andrea Dell'Amico

05:24 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

Gianpaolo Coro wrote:
> Tests were successful on dataminer2-p.
> Please proceed with dataminer1-p
dataminer1-p d... Andrea Dell'Amico

05:02 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

dataminer2-p (not dataminer1-p) deployed.
Andrea Dell'Amico

04:31 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

Andrea Dell'Amico wrote:
> Gianpaolo Coro wrote:
> > All tests passed!
> > Please repeat the installation on datam... Andrea Dell'Amico

04:30 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

Gianpaolo Coro wrote:
> All tests passed!
> Please repeat the installation on dataminer1-d-d4s.d4science.org
Dep... Andrea Dell'Amico

04:30 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

Gianpaolo Coro wrote:
> The wps_config.xml file is already in the DataminerConfiguration SVN package you are dowload... Andrea Dell'Amico

04:06 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

dataminer2-d-d4s.d4science.org has been deployed again with an up to date wps_config.xml. Andrea Dell'Amico

03:54 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

Gianpaolo Coro wrote:
> The wps_config.xml file is not updated and does not correspond to the one on SVN. Could you ... Andrea Dell'Amico

01:07 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

dataminer2-d-d4s.d4science.org has been provisioned.
Andrea Dell'Amico

04:55 PM D4Science Infrastructure Task #2220 (In Progress): Configure the provisioned smartgears nodes with a web proxy in front of them

The solution is being tested on the dev smartexecutors right now. And it was already successfully tested on the EGI i... Andrea Dell'Amico

Mar 03, 2016

06:11 PM D4Science Infrastructure Task #102: AMQ in production: upgrade OS

Ok, you'll give me the green flag when ready.
Andrea Dell'Amico

Mar 02, 2016

06:00 PM D4Science Infrastructure Task #2469: Install org.gcube.dataanalysis.dataminer.1.0.2

The playbook is ready. Waiting for the 1.0.2-3.10.1 version to appear on the maven repository.
Andrea Dell'Amico

04:11 PM D4Science Infrastructure Task #2469 (In Progress): Install org.gcube.dataanalysis.dataminer.1.0.2

Andrea Dell'Amico

05:59 PM D4Science Infrastructure Task #102: AMQ in production: upgrade OS

Also the dev VM is old and not maintained. Should we start with that?
Andrea Dell'Amico

05:57 PM D4Science Infrastructure Task #176 (Paused): Configure DSS to support EUDAT-D4Science integration

Andrea Dell'Amico

05:45 PM D4Science Infrastructure Task #2227: test replication tools for postgres

pgpool II and postres are installed on both machines. I'm studying the pgpool configuration to make it usable.
Andrea Dell'Amico

04:09 PM D4Science Infrastructure Task #2227 (In Progress): test replication tools for postgres

Andrea Dell'Amico

05:31 PM D4Science Infrastructure Support #2463 (Closed): Need space on node6.t.openaire

I've doubled the disk space.
Andrea Dell'Amico

05:28 PM D4Science Infrastructure Support #2463 (In Progress): Need space on node6.t.openaire

Andrea Dell'Amico

Mar 01, 2016

02:45 PM D4Science Infrastructure Task #2227: test replication tools for postgres

I'm taking it. Andrea Dell'Amico

12:28 PM D4Science Infrastructure Task #2221: Experiment with a new monitoring/metrics tool: influxdata (influxdb)

I see that Kronograf is still very limited and the useful features will not be open sources. It seems that http://gra... Andrea Dell'Amico

Feb 29, 2016

02:56 PM D4Science Infrastructure Task #2174: Dataminer and Statistical Manager packages

Can I install it on all the servers? dataminer prod/dev, statistical prod/dev ?
Andrea Dell'Amico

02:38 PM D4Science Infrastructure Task #2341 (Closed): Nagios: remove checks on dl16.di.uoa.gr and dl21.di.uoa.gr

Done. Andrea Dell'Amico

02:36 PM D4Science Infrastructure Task #2341 (In Progress): Nagios: remove checks on dl16.di.uoa.gr and dl21.di.uoa.gr

Andrea Dell'Amico

Feb 18, 2016

09:10 PM D4Science Infrastructure Support #2299: Install DataMiner on the production machines for 3.10

I also imported into the pom.xml used by the playbook the dependencies that you sent me:
~~~
<dependency>
... Andrea Dell'Amico

08:51 PM D4Science Infrastructure Support #2299: Install DataMiner on the production machines for 3.10

This one happens on all the dataminer installation when tomcat starts. I told you on skype the other day about it. It... Andrea Dell'Amico

02:24 PM D4Science Infrastructure Support #2299 (Feedback): Install DataMiner on the production machines for 3.10

dataminer1-p-d4s's upgrade is finished. I shut down the container.
Andrea Dell'Amico

02:17 PM D4Science Infrastructure Support #2299 (In Progress): Install DataMiner on the production machines for 3.10

I'm starting the deploy of dataminer1-p
Andrea Dell'Amico

12:32 PM D4Science Infrastructure Support #2299: Install DataMiner on the production machines for 3.10

The script is there, a tag was missing and it was excluded from the upgrade tasks list.
Now it's fixed and the keys ... Andrea Dell'Amico

11:44 AM D4Science Infrastructure Support #2299: Install DataMiner on the production machines for 3.10

dataminer2-p-d4s.d4science.org is updated already, as I wrote yesterday. You only need to start the tomcat instance t... Andrea Dell'Amico

02:30 PM D4Science Infrastructure Incident #2287: registry-publisher-1.2.6-3.10.1.jar still contains the broken log4j.properties

Thanks. I removed the workaround from the playbook.
I cannot test it on the production dataminer, but I think we can... Andrea Dell'Amico

01:04 AM D4Science Infrastructure Incident #2287 (Closed): registry-publisher-1.2.6-3.10.1.jar still contains the broken log4j.properties

registry-publisher-1.2.6-3.10.1.jar as found in the smartgears 1.2.7-3.10.1 distribution still carry the log4j.proper... Andrea Dell'Amico

01:45 AM D4Science Infrastructure Task #2174: Dataminer and Statistical Manager packages

Those R packages and their dependencies are installed on all the dataminer (dev and prod) and all the statistical man... Andrea Dell'Amico

Feb 17, 2016

06:43 PM D4Science Infrastructure Task #2263 (Closed): nagios check for the jackrabbit services

Andrea Dell'Amico

06:42 PM D4Science Infrastructure Task #2263 (In Progress): nagios check for the jackrabbit services

The check is now active.
Andrea Dell'Amico

Feb 16, 2016

06:53 PM D4Science Infrastructure Task #2275 (Closed): Try foreman with the ansible plugin as an ansible console

It seems that is now possible to log the ansible activities using foreman: http://theforeman.org/plugins/foreman_ansi... Andrea Dell'Amico

05:41 PM D4Science Infrastructure Task #2174: Dataminer and Statistical Manager packages

The FLa4a R package needs the following debs:
~~~
- gsl-bin
- libgsl0-dev
~~~ Andrea Dell'Amico

11:04 AM D4Science Infrastructure Task #2263 (Closed): nagios check for the jackrabbit services

Can you provide a URL to verify if it's working?
Andrea Dell'Amico

Feb 15, 2016

08:38 PM D4Science Infrastructure Task #2261: Better check for data.d4science.org availability

@roberto.cirillo@isti.cnr.it @francesco.mangiacrapa@isti.cnr.it can you provide such URLs?
Andrea Dell'Amico

08:37 PM D4Science Infrastructure Task #2261 (Closed): Better check for data.d4science.org availability

Right now there's only a simple tcp check on port 80.
It would be better to have a URL for each of the applications ... Andrea Dell'Amico

08:28 PM D4Science Infrastructure Task #2259 (Closed): New VM for data.d4science.org (production)

Andrea Dell'Amico

08:27 PM D4Science Infrastructure Task #2259: New VM for data.d4science.org (production)

It runs on dlib28x, while the old run on dlib17x (can be useful if we need to resurrect it)
Andrea Dell'Amico

08:27 PM D4Science Infrastructure Task #2259 (In Progress): New VM for data.d4science.org (production)

The VM is alive and provisioned. Andrea Dell'Amico

08:26 PM D4Science Infrastructure Task #2259 (Closed): New VM for data.d4science.org (production)

A new VM is needed with the same hostname and IP of the old one, so that we can switch back easily if problems arise.... Andrea Dell'Amico

04:13 PM D4Science Infrastructure Task #2110: Please provide new VM with Elastic Search

Done.
Andrea Dell'Amico

03:44 PM D4Science Infrastructure Task #2110 (Feedback): Please provide new VM with Elastic Search

The service is up and running, listening on port 9200 (the default).
The configuration is provisioned and overwrites... Andrea Dell'Amico