Project

General

Profile

Actions

Task #564

closed

clean mongodb cluster on dev environment

Added by Roberto Cirillo over 9 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
High
Category:
System Application
Target version:
Start date:
Sep 08, 2015
Due date:
% Done:

100%

Estimated time:
Infrastructure:
Development

Description

The clean is done running a full resync on secondaries member one at a time,and after that, step-down the primary and resync to newly secondary.

Actions #1

Updated by Roberto Cirillo over 9 years ago

The new secondary member (mongo2-d-d4s.d4science.org) had completed the full resync to the primary member (node41.d.d4science.research-infrastructures.eu).
At this time the primary has 306 GB of disk space used, while the new secondary has 282 GB of disk space used.
Now I'm going to sync the secondary member node42.d.d4science.research-infrastructures.eu.

Actions #2

Updated by Roberto Cirillo over 9 years ago

  • % Done changed from 30 to 70
Actions #3

Updated by Roberto Cirillo over 9 years ago

  • Assignee changed from Roberto Cirillo to _InfraScience Systems Engineer

The new members:
mongo2-d-d4s.d4science.org, mongo3-d-d4s.d4science.org are not able to receive connections from storage client library on port 27017. Could it be a iptables rules problem? Could anyone check it, please?

Actions #4

Updated by Andrea Dell'Amico over 9 years ago

Port 27017 is open to the nemis network only:

10816  649K ACCEPT     tcp  --  *      *       146.48.122.0/23      0.0.0.0/0            state NEW tcp dpt:27017

What other networks have to be allowed?

You can check the firewall tables by running the command

iptables -nvL

as root user, btw.

Actions #5

Updated by Tommaso Piccioli over 9 years ago

On mongo2-d-d4s.d4science.org and mongo3-d-d4s.d4science.org (and mongo1-d-d4s.d4science.org in the future) we can have the same iptables rules than on node67.p.d4science.research-infrastructures.eu

Actions #6

Updated by Roberto Cirillo over 9 years ago

I've tried from my machine: 146.48.85.73 and it doesn't work . The networks should be allowed for the development cluster are all the networks allowed for production cluster. (e.g. node58.p.d4science.research-infrastructures.eu). Could you compare the rules?

Actions #7

Updated by Tommaso Piccioli over 9 years ago

On the dev mongo cluster now we ave the same iptables rules of the prod mongo cluster

Actions #8

Updated by Roberto Cirillo over 9 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 70 to 100

it works now. The clean is done.

Actions #9

Updated by Tommaso Piccioli over 9 years ago

  • Status changed from Closed to In Progress
  • Assignee changed from _InfraScience Systems Engineer to Roberto Cirillo
  • Priority changed from Urgent to High
  • % Done changed from 100 to 90

New host mongo1-d-d4s.d4science.org, ready to be configured to substitute node40.d.d4science.research-infrastructures.eu

Actions #10

Updated by Roberto Cirillo over 9 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 90 to 100

The new dev cluster is complete.
For every member the OS has been upgraded and the log rotation has been enabled

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)