D4science

Aug 11, 2016

11:45 AM D4Science Infrastructure Task #935 (Closed): monitor the d4science and openaire redmine instances

Andrea Dell'Amico

11:44 AM D4Science Infrastructure Task #936 (Closed): Add the d4science and openaire redmine instances to nagios

Done. Andrea Dell'Amico

Aug 09, 2016

10:24 AM D4Science Infrastructure Task #4842: Make the ldap configuration on rstudio(-dev).d4science.org point to ldap.d4science.org

A reference to a workaround for a problem that affects ubuntu 14.04: https://www.benjaminfleckenstein.name/en/pam-aut... Andrea Dell'Amico

Aug 08, 2016

11:51 AM D4Science Infrastructure Task #4852: Split the ganglia and nagios servers

In the meantime I added 2 GB of RAM and moved the ganglia data to a SAN exported volume.
Andrea Dell'Amico

11:27 AM D4Science Infrastructure Task #4852 (Rejected): Split the ganglia and nagios servers

The monitoring load is skyrocketing as we are adding more nagios checks and more services to ganglia.
Create two n... Andrea Dell'Amico

Aug 05, 2016

10:55 AM D4Science Infrastructure Task #4847 (Closed): Open CPU timeout is too small

Andrea Dell'Amico

Aug 04, 2016

05:46 PM D4Science Infrastructure Task #4847 (Feedback): Open CPU timeout is too small

Done. Let me know if it works.
Andrea Dell'Amico

05:35 PM D4Science Infrastructure Task #4847 (In Progress): Open CPU timeout is too small

I suppose you are referring to the `timelimit.post` options, because the `timelimit.get` option has a timeout of 60... Andrea Dell'Amico

Aug 02, 2016

05:36 PM D4Science Infrastructure Task #4842 (Closed): Make the ldap configuration on rstudio(-dev).d4science.org point to ldap.d4science.org

I changed /etc/ldap.conf but the services use nslcd. I tried to reconfigure it, but the connection to the ldap server... Andrea Dell'Amico

Aug 01, 2016

01:48 PM D4Science Infrastructure Task #4414: New server for the Broker Service

Server rebooted with the new resources.
Let me know what parts need to be contacted directly, because we still nee... Andrea Dell'Amico

Jul 29, 2016

04:18 PM D4Science Infrastructure Task #2261 (Closed): Better check for data.d4science.org availability

Done. Andrea Dell'Amico

03:39 PM D4Science Infrastructure Task #4815 (Closed): Enable dev2.d4science to reach postgreSQL instance on preprod1.d4science and viceversa

Andrea Dell'Amico

01:06 PM D4Science Infrastructure Task #4826: Create load balancing for data.d4science.org

As we said, there's no problem as long as the webapps have no state.
We can use the same VM that is acting as load... Andrea Dell'Amico

11:35 AM D4Science Infrastructure Task #4824 (Closed): Please delete user frippe12573 from LDAP

Done. Andrea Dell'Amico

Jul 28, 2016

06:31 PM D4Science Infrastructure Task #4806 (Closed): Nagios monitoring for the Couchbase cluster

We now have detailed checks for the two main buckets on ganglia too.
Andrea Dell'Amico

03:12 PM D4Science Infrastructure Task #2261 (In Progress): Better check for data.d4science.org availability

Added to nagios. You can see the host details here: http://monitoring.research-infrastructures.eu/cgi-bin/nagios3/sta... Andrea Dell'Amico

Jul 27, 2016

07:42 PM D4Science Infrastructure Task #4806 (Feedback): Nagios monitoring for the Couchbase cluster

@pieve.alessandro@gmail.com I need some feedback about the checks. I have activated a subset of the available ones. ... Andrea Dell'Amico

06:17 PM D4Science Infrastructure Task #4806: Nagios monitoring for the Couchbase cluster

There's also a ganglia plugin available: https://github.com/kozdincer/ganglia-couchbase
Andrea Dell'Amico

10:27 AM D4Science Infrastructure Task #4806 (In Progress): Nagios monitoring for the Couchbase cluster

Andrea Dell'Amico

03:55 PM D4Science Infrastructure Task #4815 (Feedback): Enable dev2.d4science to reach postgreSQL instance on preprod1.d4science and viceversa

On preprod1.d4science.org you now have a db named `infra_dev_db` accessible by the `infra_dev` user from dev2.d4scien... Andrea Dell'Amico

03:45 PM D4Science Infrastructure Task #4815 (In Progress): Enable dev2.d4science to reach postgreSQL instance on preprod1.d4science and viceversa

Andrea Dell'Amico

Jul 26, 2016

10:14 PM D4Science Infrastructure Incident #4805: SmartGears on dataminer4 fails to save status

There's an open ticket for the bug, #2176 Andrea Dell'Amico

07:20 PM D4Science Infrastructure Incident #4805: SmartGears on dataminer4 fails to save status

I don't think that the VM performances have a big impact on the corrupted state problem. Yesterday a dozen smartexecu... Andrea Dell'Amico

08:11 PM D4Science Infrastructure Task #4806: Nagios monitoring for the Couchbase cluster

@alessandro.pieve@isti.cnr.it can you provide us the credentials? Andrea Dell'Amico

08:10 PM D4Science Infrastructure Task #4806 (Closed): Nagios monitoring for the Couchbase cluster

To monitor the Couchbase cluster we need a username/password to access the db.
Possibly a username dedicated to th... Andrea Dell'Amico

07:46 PM D4Science Infrastructure Task #4722 (Closed): Enable Nagios check on SmartExecutor nodes

Done. Andrea Dell'Amico

07:39 PM D4Science Infrastructure Task #897 (Closed): Install and configure a couchdb cluster

Andrea Dell'Amico

07:39 PM D4Science Infrastructure Task #325 (Closed): Nagios monitoring for couchdb

Andrea Dell'Amico

07:07 PM D4Science Infrastructure Task #364 (Closed): Nagios: create a new hostgroup and assign all the CNR monitored hosts to it

All the monitored local hosts are now under the `CNR site` group. Andrea Dell'Amico

05:54 PM D4Science Infrastructure Task #3261 (Closed): Manage more ldap groups from the portal so that we can setup different roles on various services

We solved in a different way, see #3064 Andrea Dell'Amico

05:46 PM D4Science Infrastructure Task #4773 (Closed): The nagios check that verifies the support expiry day does not work anymore

Fixed by @tommaso.piccioli@isti.cnr.it Andrea Dell'Amico

05:42 PM D4Science Infrastructure Task #2720 (Closed): Explore rundeck as a better alternative to foreman

It does not work and the development is stuck.
Andrea Dell'Amico

05:41 PM D4Science Infrastructure Task #4791 (Closed): Increase the number of file descriptors

Andrea Dell'Amico

01:29 PM D4Science Infrastructure Task #4791 (Feedback): Increase the number of file descriptors

We now have two tasks.
One sets the root defaults and it will be executed on all our provisioned hosts.
The sec... Andrea Dell'Amico

12:27 PM D4Science Infrastructure Task #4791 (In Progress): Increase the number of file descriptors

I'm going to create two tasks:
* One that changes the defaults for the root user
* One that changes the limits fo... Andrea Dell'Amico

05:40 PM D4Science Infrastructure Task #4476 (Feedback): Please create new VM for production HA Proxy and deploy it

Andrea Dell'Amico

11:12 AM D4Science Infrastructure Task #4782: Provide ansible playbook for dataminer deployment

Gianpaolo Coro wrote:
> Hi Andrea, the "sobigdata" term makes me thrill... :-S
>
> The distribution of the datami... Andrea Dell'Amico

09:11 AM D4Science Infrastructure Task #4782: Provide ansible playbook for dataminer deployment

Gianpaolo Coro wrote:
> I have run complete tests, also on the geoprocessing algorithms. All the dataminers works. N... Andrea Dell'Amico

Jul 25, 2016

07:21 PM D4Science Infrastructure Task #4782: Provide ansible playbook for dataminer deployment

dataminer[1:2] has been upgraded (for real, this time).
Andrea Dell'Amico

12:31 PM D4Science Infrastructure Task #4476: Please create new VM for production HA Proxy and deploy it

Done, both for the preproduction and the production environments. Andrea Dell'Amico

12:09 PM D4Science Infrastructure Task #4476 (In Progress): Please create new VM for production HA Proxy and deploy it

We need to workaround a problem with the liferay clustering. We are exploiting the _weight_ option on the haproxy bac... Andrea Dell'Amico

12:07 PM D4Science Infrastructure Task #4703 (Closed): Create Tomcat startup script at boot

The startup script is working.
Andrea Dell'Amico

Jul 24, 2016

02:30 PM D4Science Infrastructure Task #4782: Provide ansible playbook for dataminer deployment

I've just added the following files:
~~~
52n-wps-algorithm-3.5.0.jar
52n-wps-commons-3.5.0.jar
52n-wps-database... Andrea Dell'Amico

Jul 22, 2016

04:57 PM D4Science Infrastructure Task #4723 (Closed): smartgears configuration: handle the case where a smartgears must run without a VO

It works.
Andrea Dell'Amico

04:56 PM D4Science Infrastructure Task #4782: Provide ansible playbook for dataminer deployment

A couple of notes:
~~~
dataminer1-p-d4s.d4science.org
dataminer2-p-d4s.d4science.org
~~~
will be upgraded ne... Andrea Dell'Amico

02:23 PM D4Science Infrastructure Task #4606 (Closed): Geonetwork dev 64 bit

Andrea Dell'Amico

02:22 PM D4Science Infrastructure Task #205 (In Progress): Automate the installation and configuration of the geonetwork servers

Work in progress. See also #4606 Andrea Dell'Amico

02:19 PM D4Science Infrastructure Task #183 (In Progress): tomcat running as root user under port 80

tomcat instances still running as root:
* wps.statistical.d4science.org
* the geonetworks (there's a dev installa... Andrea Dell'Amico

02:13 PM D4Science Infrastructure Task #4771 (Closed): Run smartgears on the rstudio-server instances as unprivileged user

Working correctly in production too.
Andrea Dell'Amico

01:12 PM D4Science Infrastructure Task #4771: Run smartgears on the rstudio-server instances as unprivileged user

I provisioned the production instance too, in the same way. I also changed the home directory permissions for the exi... Andrea Dell'Amico

Jul 21, 2016

07:37 PM D4Science Infrastructure Task #2174: Dataminer and Statistical Manager packages

All added to the playbook lists.
Andrea Dell'Amico

07:27 PM D4Science Infrastructure Task #4771 (Feedback): Run smartgears on the rstudio-server instances as unprivileged user

It seems I got it working. I heavily modified the script that adds the users so that it uses sudo.
It also creates a... Andrea Dell'Amico

12:04 PM D4Science Infrastructure Task #4771 (In Progress): Run smartgears on the rstudio-server instances as unprivileged user

Starting with rstudio-dev Andrea Dell'Amico

12:02 PM D4Science Infrastructure Task #4771: Run smartgears on the rstudio-server instances as unprivileged user

We need to:
* Give sudo privileges to the commands esecuted by the `Ruseradd.sh` script
* Add firewall rules to i... Andrea Dell'Amico

12:00 PM D4Science Infrastructure Task #4771 (Closed): Run smartgears on the rstudio-server instances as unprivileged user

We need to:
* Give sudo privileges to the commands esecuted by the `Ruseradd.sh` script
* Andrea Dell'Amico

01:39 PM D4Science Infrastructure Task #4773 (Closed): The nagios check that verifies the support expiry day does not work anymore

It seems that uses a deprecated API version Andrea Dell'Amico

Jul 20, 2016

03:23 PM D4Science Infrastructure Task #2946 (Closed): Install ganglia and the local nagios checks on thredds-p-d4s.d4science.org

Done. Andrea Dell'Amico

02:26 PM D4Science Infrastructure Task #2946 (In Progress): Install ganglia and the local nagios checks on thredds-p-d4s.d4science.org

Ganglia is active.
Andrea Dell'Amico

03:23 PM D4Science Infrastructure Task #4196 (Closed): Restore nagios check on thredds VM

Done. Andrea Dell'Amico

02:24 PM D4Science Infrastructure Task #4196 (In Progress): Restore nagios check on thredds VM

Andrea Dell'Amico

Jul 19, 2016

07:12 PM D4Science Infrastructure Task #4765 (Closed): Renew the wiki certificates

The Terena TLS certificate is missing some host aliases, let's ask for a Letsencrypt certificate that lists all the h... Andrea Dell'Amico

05:42 PM D4Science Infrastructure Task #3064 (Closed): Investigate the installation of an identity manager

It's working. I just reconfigured redmine to use the new ldap `redmine_admins` group as the administrators group.
Andrea Dell'Amico

04:07 PM D4Science Infrastructure Task #3064: Investigate the installation of an identity manager

It's working, I'm installing it on ldap.d4science.org too. Andrea Dell'Amico

12:05 PM D4Science Infrastructure Task #3064: Investigate the installation of an identity manager

I'm experimenting with fusiondirectory, reachable here: https://ldap-d.d4science.org/fusiondirectory/
The admin us... Andrea Dell'Amico

Jul 18, 2016

01:40 PM D4Science Infrastructure Task #4733: Remove node74.d4science.org from smart-executor ganglia cluster

This will happen automatically if the host will be provisioned by an ansible playbook.
Andrea Dell'Amico

01:38 PM D4Science Infrastructure Task #4742 (Closed): Registry services: Increase the maximum number of file descriptor to 32000

The limits were raised in 2013, but the pam configuration for the `su` command was not fixed. So the new limits were ... Andrea Dell'Amico

Jul 15, 2016

07:13 PM D4Science Infrastructure Task #3064: Investigate the installation of an identity manager

midPoint is way too much and way too complex for our needs. Better try a plain ldap manager:
https://www.fusiondir... Andrea Dell'Amico

03:55 PM D4Science Infrastructure Task #3064 (In Progress): Investigate the installation of an identity manager

idm.d4science.org. IN A 146.48.122.94 Andrea Dell'Amico

03:46 PM D4Science Infrastructure Task #469 (Closed): Move some VMs to the new hypervisor to set some servers free and usable to experiment with OpenStack

Andrea Dell'Amico

03:45 PM D4Science Infrastructure Task #2514 (Closed): Free dlib8x and dlib10x and assign them to the ganeti cluster

We are not sure that using old and low capacity servers to host corporate services is a good idea. Closing for now.
Andrea Dell'Amico

03:44 PM D4Science Infrastructure Task #472 (Closed): Move the VMs from the 'OpenStack servers' to the new hypervisor

Andrea Dell'Amico

03:44 PM D4Science Infrastructure Task #471 (Closed): Locate the hypervisors to dedicate to OpenStack

We are going to start with two new servers. Andrea Dell'Amico

01:39 PM D4Science Infrastructure Task #4736 (Closed): smartgears container.xml: the context specific scope configuration needs to be generic

Done. Modified all the use cases accordingly.
Andrea Dell'Amico

01:04 PM D4Science Infrastructure Task #4736 (In Progress): smartgears container.xml: the context specific scope configuration needs to be generic

Andrea Dell'Amico

01:04 PM D4Science Infrastructure Task #4736 (Closed): smartgears container.xml: the context specific scope configuration needs to be generic

The template automation was created for the smart executor, but it's needed by other applications.
So it needs to ... Andrea Dell'Amico

Jul 14, 2016

01:16 PM D4Science Infrastructure Task #4723 (Feedback): smartgears configuration: handle the case where a smartgears must run without a VO

Andrea Dell'Amico

12:22 PM D4Science Infrastructure Task #4723 (In Progress): smartgears configuration: handle the case where a smartgears must run without a VO

Andrea Dell'Amico

12:22 PM D4Science Infrastructure Task #4723 (Closed): smartgears configuration: handle the case where a smartgears must run without a VO

Add a conditional in the container.xml template to handle the case.
Andrea Dell'Amico

12:11 PM D4Science Infrastructure Task #4476: Please create new VM for production HA Proxy and deploy it

Massimiliano Assante wrote:
> Please remember that all these domains below, including some others yet to be created ... Andrea Dell'Amico

12:03 PM D4Science Infrastructure Task #4476 (Feedback): Please create new VM for production HA Proxy and deploy it

sobigdata.research-infrastructures.eu added as CNAME, x509 certificate updated with the new alias.
I increased the... Andrea Dell'Amico

11:58 AM D4Science Infrastructure Task #4606 (Feedback): Geonetwork dev 64 bit

Smartgears and postgresql 9.4 installed. The `geonetwork` db has the postgis extensions active.
Andrea Dell'Amico

Jul 13, 2016

07:47 PM D4Science Infrastructure Task #4606: Geonetwork dev 64 bit

The db user will be `geonetwork`.
I need the name of the application context for the virtualhost configuration.
Andrea Dell'Amico

07:18 PM D4Science Infrastructure Task #4648 (Closed): Add Ganglia monitoring to the couchbase cluster

You can see the couchbase nodes here: http://monitoring.research-infrastructures.eu/ganglia/?c=D4science%20Couchdb%20... Andrea Dell'Amico

07:10 PM D4Science Infrastructure Task #4648 (In Progress): Add Ganglia monitoring to the couchbase cluster

Andrea Dell'Amico

05:14 PM D4Science Infrastructure Task #4648: Add Ganglia monitoring to the couchbase cluster

Can we add them to the actual `couchdb` ganglia cluster or we need a new one? Andrea Dell'Amico

06:06 PM D4Science Infrastructure Task #4717 (Closed): Modify the solr_over_tomcat ansible playbook to support multiple core

Done.
Andrea Dell'Amico

05:16 PM D4Science Infrastructure Task #4717 (In Progress): Modify the solr_over_tomcat ansible playbook to support multiple core

Andrea Dell'Amico

05:16 PM D4Science Infrastructure Task #4717 (Closed): Modify the solr_over_tomcat ansible playbook to support multiple core

We need to be able to statically configure multiple solr cores, starting from a pristine `collection1` data set.
Andrea Dell'Amico