D4Science Infrastructure

Yes. The redmine instance is http://redmine-d.d4science.org/
Do I need to change the ldap base DN from

ou=People,o=Liferay,ou=Organizations,dc=d4science,dc=org

to

ou=People,o=D4Science,ou=Organizations,dc=d4science,dc=org

?

yes -> ou=People,o=D4Science,ou=Organizations,dc=d4science,dc=org

OK, done. We need to wait some minutes to give time to the next synchronization to complete.

Last, it could be useful to have a LDAP group called something like redmineAdmins and assign it to all the people that we want as redmine administrators.

Sounds good.
In my opinion the manually created group can be removed. They was useful when the procedure was manual, but now is useless and create confusion.
The group should only come from VRE.

I'm not sure that removing that current group is the best solution, perhaps we could remove the ones of partner organizations (FAO, IRD etc) but some of the groups, e.g. redmineAdmins, infraScience sysadmins could be useful.

Also, we need to schedule this update on production quite soon, possibly before the TCom (Jan 25th).

I think the plan could be as follows (prerequisite: announcing a minor Redmine downtime of 2 hours):

1. backup of the current redMine prod. DB (and current LDAP DB) and connection to an empty new redMine DB
2. Delete of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
3. configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
4. ensure that everything works as expected and be prepared to roll-back if anything goes wrong

I can take care of #2 and #4 (portal side), probably @andrea.dellamico@isti.cnr.it could take care of #1 and @luca.frosini@isti.cnr.it of #3 + #4 (redmine side)?

Does the plan work for you?

Massimiliano Assante wrote:

I'm not sure that removing that current group is the best solution, perhaps we could remove the ones of partner organizations (FAO, IRD etc) but some of the groups, e.g. redmineAdmins, infraScience sysadmins could be useful.

Can you create them with your script on ldap? so that we can exploit ldap to set the redmine administrators.

Also, we need to schedule this update on production quite soon, possibly before the TCom (Jan 25th).

I think the plan could be as follows (prerequisite: announcing a minor Redmine downtime of 2 hours):

1. backup of the current redMine prod. DB (and current LDAP DB) and connection to an empty new redMine DB

Why? OK for the backup, there are scripts already in place - there's also a nightly backup btw, as in all the 'new' postgresql installations. But I do not understand the need to use an empty one.

1. Delete of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
2. configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
3. ensure that everything works as expected and be prepared to roll-back if anything goes wrong

I can take care of #2 and #4 (portal side), probably @andrea.dellamico@isti.cnr.it could take care of #1 and @luca.frosini@isti.cnr.it of #3 + #4 (redmine side)?

Does the plan work for you?

Andrea Dell'Amico wrote:

Massimiliano Assante wrote:

I'm not sure that removing that current group is the best solution, perhaps we could remove the ones of partner organizations (FAO, IRD etc) but some of the groups, e.g. redmineAdmins, infraScience sysadmins could be useful.

Can you create them with your script on ldap? so that we can exploit ldap to set the redmine administrators.

I agree. Without creating a a VRE we should manage this from LDAP. Otherwise we will always have problems.
We can start add this group with a script (maybe ansible one).

Also, we need to schedule this update on production quite soon, possibly before the TCom (Jan 25th).

I think the plan could be as follows (prerequisite: announcing a minor Redmine downtime of 2 hours):

1. backup of the current redMine prod. DB (and current LDAP DB) and connection to an empty new redMine DB

Why? OK for the backup, there are scripts already in place - there's also a nightly backup btw, as in all the 'new' postgresql installations. But I do not understand the need to use an empty one.

1. Delete of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
2. configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
3. ensure that everything works as expected and be prepared to roll-back if anything goes wrong

No problem for me to do 3 and 4 (redmine side)

I can take care of #2 and #4 (portal side), probably @andrea.dellamico@isti.cnr.it could take care of #1 and @luca.frosini@isti.cnr.it of #3 + #4 (redmine side)?

Does the plan work for you?

My LDAP script reflects the VREs into LDAP Groups, unless we create a VRE for redmine admins is not possible to do it. And I don't want to hardcode anything in there.

As for:

Why? OK for the backup, there are scripts already in place - there's also a nightly backup btw, as in all the 'new' postgresql installations. But I do not understand the need to use an empty one.

Andrea you're right, of course there is no need to use an empty DB for Redmine, I meant the LDAP DB should be an empty one not the Redmine. Sorry. I am rewriting the plan:

I think the plan could be as follows (prerequisite: announcing a minor Redmine downtime of 2 hours):

1. backup of the current LDAP DB (Ldap-liferay)
2. drop of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
3. configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
4. ensure that everything works as expected and be prepared to roll-back if anything goes wrong

It seems we have an agreement on this:

Andrea takes #1
Massi takes #2 and #4 (portal side)
Luca takes #3 and #4 (redmine side)

Step #1 is prerequisite to #2 and #3

Massimiliano Assante wrote:

My LDAP script reflects the VREs into LDAP Groups, unless we create a VRE for redmine admins is not possible to do it. And I don't want to hardcode anything in there.

OK, I'll try to work out something that adds groups.

Andrea you're right, of course there is no need to use an empty DB for Redmine, I meant the LDAP DB should be an empty one not the Redmine. Sorry. I am rewriting the plan:

OK, now it's clear.

I think the plan could be as follows (prerequisite: announcing a minor Redmine downtime of 2 hours):

1. backup of the current LDAP DB (Ldap-liferay)
2. drop of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
3. configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
4. ensure that everything works as expected and be prepared to roll-back if anything goes wrong

It seems we have an agreement on this:

Andrea takes #1
Massi takes #2 and #4 (portal side)
Luca takes #3 and #4 (redmine side)

Step #1 is prerequisite to #2 and #3

OK. The ldap server already has daily backups, so it will be a matter of running it manually once just before the drop.

Can you list those users? I suspect that they also exist with the same email address in the old ldap server and they are configured to authenticate with that one.
The sync procedure skips those users, see:

...
-- Updating user 'mister.blue@d4science.org' (Blue Mister)...
-- Updating user 'mister.brown@d4science.org' (Brown Mister)...
-- Updating user 'mister.orange@d4science.org' (Tim Roth)...
-- Updating user 'mister.pink@d4science.org' (Jesse Pinkman)...
-- Updating user 'mister.white@d4science.org' (Walter White)...
-- Skipping user 'p.koltsida@di.uoa.gr': it already exists on a different auth_source
-- Skipping user 'paolo.fabriani@eng.it': it already exists on a different auth_source
-- Skipping user 'pasquale.pagano@isti.cnr.it': it already exists on a different auth_source
-- Skipping user 'roberto.cirillo@isti.cnr.it': it already exists on a different auth_source
-- Updating user 'statistical.manager@gmail.com' (statistical manager)...
-- Skipping user 'tommaso.piccioli@isti.cnr.it': it already exists on a different auth_source
-- Skipping user 'valentina.marioli@isti.cnr.it': it already exists on a different auth_source

Yes that could be the reason, in fact on Biodiversity Lab only my user is exported (and My user is configured to authenticate against the liferay-ldap) while, lucio, gxanpaolo and roberto (the 3 missing users) are configured to authenticate against the old ldap.

You can test changing the configuration of one of them. At the next synch it should appear in the group.

We'll have to handle this right after the migration.

I did try that, and Indeed I confirm the one you suspected is the actual reason.

How do you think we should proceed right after the migration? How do we know who are the users needing to auth against the liferay-ldap and who don't. I suppose we should just make all of them auth against the liferay-ldap.

Massimiliano Assante wrote:

I did try that, and Indeed I confirm the one you suspected is the actual reason.

How do you think we should proceed right after the migration? How do we know who are the users needing to auth against the liferay-ldap and who don't. I suppose we should just make all of them auth against the liferay-ldap.

After the first synchronization run we will have a complete list of duplicate entries. We can choose to migrate them all and then manage the new problems (forgotten passwords, whatever).
The change can be done updating the DB tables directly, it's a matter of minutes.

ok then the plan is updated as follows:

Announcement a minor Redmine downtime of 1 hour tomorrow at 17.30:

1) backup of the current LDAP DB (Ldap-liferay)
2) drop of current (Ldap-liferay) LDAP Structure and deployment of the ldap-export artifact written by me, on all the gateways
3) configuring (as explained in this ticket) the redMine prod. to support the export of LDAP groups and their "population"
4) ensure that everything works as expected and be prepared to roll-back if anything goes wrong
5) migrate all the users authenticating against the old ldap to the new one via updating the DB tables

Andrea takes #1 and #5
Massi takes #2 and #4 (portal side)
Luca takes #3 and #4 (redmine side)

Step #1 is prerequisite to #2, #3 #4
Step #2 and #3 are prerequisite of 5

The portal users were exported correctly on LDAP Liferay Production

#1, #2 and #4 complete (portal side)

The last step to do is associating the automatically created group from VRE to the relative redmine project if any, and removing the association beetween the manually created group to projects. Only with this step we complete the migration and we realize if everything is working.
I will take care of this tomorrow and I will also made a post on BlueBridge to advise everyone. Moreover to invite any member to contact me if they have any problems.

After one week we can try to start to remove manually created groups relative to external partners (keeping our internal one). Then we will disconnect the old LDAP and we will delete the uneeded old users presents only on old LDAP and not registered in any portal.