D4Science Infrastructure

[Thu Oct 25 06:04:16 2018] Out of memory: Kill process 19985 (paster) score 848 or sacrifice child
[Thu Oct 25 06:04:16 2018] Killed process 19985 (paster) total-vm:11607888kB, anon-rss:10814468kB, file-rss:24kB
[Thu Oct 25 08:05:48 2018] Out of memory: Kill process 2685 (paster) score 862 or sacrifice child
[Thu Oct 25 08:05:48 2018] Killed process 2685 (paster) total-vm:11788116kB, anon-rss:11030996kB, file-rss:0kB
[Thu Oct 25 08:05:48 2018] Out of memory: Kill process 7679 (java) score 58 or sacrifice child
[Thu Oct 25 08:05:48 2018] Killed process 7679 (java) total-vm:5765304kB, anon-rss:293396kB, file-rss:0kB
[Thu Oct 25 11:04:17 2018] Out of memory: Kill process 13434 (paster) score 860 or sacrifice child
[Thu Oct 25 11:04:17 2018] Killed process 13434 (paster) total-vm:11761740kB, anon-rss:10907200kB, file-rss:0kB
[Thu Oct 25 14:05:18 2018] Out of memory: Kill process 28508 (paster) score 860 or sacrifice child
[Thu Oct 25 14:05:18 2018] Killed process 28508 (paster) total-vm:11761740kB, anon-rss:10879624kB, file-rss:0kB
[Thu Oct 25 14:05:18 2018] Out of memory: Kill process 13605 (java) score 58 or sacrifice child
[Thu Oct 25 14:05:18 2018] Killed process 13605 (java) total-vm:5747676kB, anon-rss:500564kB, file-rss:0kB
[Thu Oct 25 16:05:21 2018] Out of memory: Kill process 11440 (paster) score 861 or sacrifice child
[Thu Oct 25 16:05:21 2018] Killed process 11440 (paster) total-vm:11773776kB, anon-rss:10844772kB, file-rss:0kB
[Thu Oct 25 19:03:47 2018] Out of memory: Kill process 21705 (paster) score 845 or sacrifice child
[Thu Oct 25 19:03:47 2018] Killed process 21705 (paster) total-vm:11563088kB, anon-rss:10490400kB, file-rss:248kB
[Thu Oct 25 19:03:48 2018] Out of memory: Kill process 11853 (java) score 57 or sacrifice child
[Thu Oct 25 19:03:48 2018] Killed process 11853 (java) total-vm:5733132kB, anon-rss:529832kB, file-rss:0kB
[Mon Oct 29 18:05:45 2018] Out of memory: Kill process 4464 (paster) score 861 or sacrifice child
[Mon Oct 29 18:05:45 2018] Killed process 4464 (paster) total-vm:11779196kB, anon-rss:10417568kB, file-rss:44kB
[Mon Oct 29 21:04:36 2018] Out of memory: Kill process 3599 (paster) score 862 or sacrifice child
[Mon Oct 29 21:04:36 2018] Killed process 3599 (paster) total-vm:11781080kB, anon-rss:10974100kB, file-rss:216kB
[Tue Oct 30 00:04:55 2018] Out of memory: Kill process 18361 (paster) score 862 or sacrifice child
[Tue Oct 30 00:04:55 2018] Killed process 18361 (paster) total-vm:11780936kB, anon-rss:10915728kB, file-rss:0kB
[Tue Oct 30 03:04:13 2018] Out of memory: Kill process 1084 (paster) score 853 or sacrifice child
[Tue Oct 30 03:04:13 2018] Killed process 1084 (paster) total-vm:11667784kB, anon-rss:10937940kB, file-rss:52kB
[Tue Oct 30 06:04:15 2018] Out of memory: Kill process 16159 (paster) score 852 or sacrifice child
[Tue Oct 30 06:04:15 2018] Killed process 16159 (paster) total-vm:11661136kB, anon-rss:10987832kB, file-rss:0kB
[Tue Oct 30 09:05:01 2018] Out of memory: Kill process 31313 (paster) score 859 or sacrifice child
[Tue Oct 30 09:05:01 2018] Killed process 31313 (paster) total-vm:11742284kB, anon-rss:11021840kB, file-rss:0kB

This is a known problem due to harvesting processes (managed through the paster environment) performed by CKAN.
The only feasible solution at the moment is to increase the physical memory on that machine. @andrea.dellamico@isti.cnr.it have you another idea?
In the future, we will be able to update the CKAN version (I hope) that will fix the issue.

There's no other way right now, and it's the reason why the two main CKAN instances have so much RAM.

Keep in mind that the old ckan-d4s must be reconfigured in the same way.

@tommaso.piccioli@isti.cnr.it @francesco.mangiacrapa@isti.cnr.it was the RAM amount increased? if not, let's do that.

Andrea Dell'Amico wrote:

@tommaso.piccioli@isti.cnr.it @francesco.mangiacrapa@isti.cnr.it was the RAM amount increased? if not, let's do that.

Last call, I'm closing this ticket next week if nobody answers or takes action.