Project

General

Profile

Actions

Support #11596

closed

CKAN BB1 Harvesting of GN iMarine failed: due to (DatabaseError) SSL SYSCALL error: EOF detect

Added by Francesco Mangiacrapa over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Urgent
Assignee:
_InfraScience Systems Engineer
Category:
System Application
Start date:
Apr 10, 2018
Due date:
% Done:

100%

Estimated time:
Infrastructure:
Production

Description

Harvesting of iMarine metadata from GN failed due to database error: sqlalchemy.exc.DatabaseError: (DatabaseError) SSL SYSCALL error: EOF detected
and then the error was sqlalchemy.exc.OperationalError: (OperationalError) FATAL: the database system is in recovery mode

Searching it on Google I found the following post:
https://devcenter.heroku.com/articles/postgres-logs-errors#pgerror-ssl-syscall-error-eof-detected

I don't know if the connection disconnected or it was due to an exception db-side

Could you check the db status? I will restart harvesting after that one

2018-04-10 05:15:42,770 INFO  [ckanext.geonetwork.harvesters.geonetwork] 2 AUTHOR found as OrganisationName in ResponsibleParty
2018-04-10 05:15:42,770 INFO  [ckanext.geonetwork.harvesters.geonetwork] 0 Point of Contact/s found as Contact-Info in ResponsibleParty
2018-04-10 05:15:42,770 INFO  [ckanext.geonetwork.harvesters.geonetwork] groups are: [{'name': 'biota'}]
2018-04-10 05:15:42,770 INFO  [ckanext.geonetwork.harvesters.geonetwork] 0 Feature Catalogue description added

Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 191, in command
    fetch_callback(consumer, method, header, body)
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 407, in fetch_callback
    fetch_and_import_stages(harvester, obj)
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/queue.py", line 424, in fetch_and_import_stages
    success_import = harvester.import_stage(obj)
  File "/usr/lib/ckan/default/src/ckanext-spatial/ckanext/spatial/harvesters/base.py", line 626, in import_stage
    package_id = p.toolkit.get_action('package_create')(context, package_dict)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/__init__.py", line 431, in wrapped
    result = _action(context, data_dict, **kw)
  File "/usr/lib/ckan/default/src/ckan/ckan/logic/action/create.py", line 190, in package_create
    model.Session.flush()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 149, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1907, in flush
    self._flush(objects)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2025, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 57, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1989, in _flush
    flush_context.execute()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 371, in execute
    rec.execute(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 524, in execute
    uow
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 67, in save_obj
    states_to_insert, states_to_update)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 776, in _finalize_insert_update_commands
    mapper.dispatch.after_insert(mapper, connection, state)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/event/attr.py", line 257, in __call__
    fn(*args, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/orm/events.py", line 532, in wrap
    return fn(*arg, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/vdm/sqlalchemy/base.py", line 635, in after_insert
    self.make_revision(instance, mapper, connection)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/vdm/sqlalchemy/base.py", line 599, in make_revision
    connection.execute(ins)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 727, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 824, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 954, in _execute_context
    context)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1116, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 947, in _execute_context
    context)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 435, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.DatabaseError: (DatabaseError) SSL SYSCALL error: EOF detected
 'INSERT INTO package_extra_revision (id, package_id, key, value, state, revision_id, continuity_id, expired_timestamp) VALUES (%(id)s, %(package_id)s, %(key)s, %(value)s, %(state)s, %(revision_id)s, %(continuity_id)s, %(expired_timestamp)s)' {'continuity_id': u'b960f3e2-334d-471a-8978-20d0bbe74334', 'package_id': u'59a61ece-3632-4455-a3c5-ff3fbe84c587', 'value': 'info@i-marine.eu', 'state': 'active', 'key': u'contact-email', 'expired_timestamp': datetime.datetime(9999, 12, 31, 0, 0), 'revision_id': u'4e918ac3-917d-4ff3-8088-329f302e5348', 'id': u'b960f3e2-334d-471a-8978-20d0bbe74334'}
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 139, in command
    self._load_config()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 212, in _load_config
    super(Harvester, self)._load_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 161, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 129, in load_all
    unload_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
    unload(*reversed(_PLUGINS))
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 210, in unload
    plugins_update()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 121, in plugins_update
    environment.update_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 266, in update_config
    model.init_model(engine)
  File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 157, in init_model
    version_table = Table('migrate_version', meta.metadata, autoload=True)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 352, in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 425, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 448, in _autoload
    self, include_columns, exclude_columns
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1670, in run_callable
    with self.contextual_connect() as conn:
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1737, in contextual_connect
    self.pool.connect(),
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 332, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 630, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in checkout
    rec = pool._do_get()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 949, in _do_get
    return self._create_connection()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 278, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 404, in __init__
    self.connection = self.__connect()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 530, in __connect
    connection = self.__pool._creator()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 95, in connect
    connection_invalidated=invalidated
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 89, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 376, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 179, in connect
    connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  the database system is in recovery mode
FATAL:  the database system is in recovery mode
 None None
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 139, in command
    self._load_config()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 212, in _load_config
    super(Harvester, self)._load_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 161, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 129, in load_all
    unload_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
    unload(*reversed(_PLUGINS))
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 210, in unload
    plugins_update()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 121, in plugins_update
    environment.update_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 266, in update_config
    model.init_model(engine)
  File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 157, in init_model
    version_table = Table('migrate_version', meta.metadata, autoload=True)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 352, in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 425, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 448, in _autoload
    self, include_columns, exclude_columns
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1670, in run_callable
    with self.contextual_connect() as conn:
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1737, in contextual_connect
    self.pool.connect(),
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 332, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 630, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in checkout
    rec = pool._do_get()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 949, in _do_get
    return self._create_connection()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 278, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 404, in __init__
    self.connection = self.__connect()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 530, in __connect
    connection = self.__pool._creator()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 95, in connect
    connection_invalidated=invalidated
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 89, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 376, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 179, in connect
    connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  the database system is in recovery mode
FATAL:  the database system is in recovery mode
 None None
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 139, in command
    self._load_config()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 212, in _load_config
    super(Harvester, self)._load_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 161, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 129, in load_all
    unload_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
    unload(*reversed(_PLUGINS))
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 210, in unload
    plugins_update()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 121, in plugins_update
    environment.update_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 266, in update_config
    model.init_model(engine)
  File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 157, in init_model
    version_table = Table('migrate_version', meta.metadata, autoload=True)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 352, in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 425, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 448, in _autoload
    self, include_columns, exclude_columns
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1670, in run_callable
    with self.contextual_connect() as conn:
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1737, in contextual_connect
    self.pool.connect(),
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 332, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 630, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in checkout
    rec = pool._do_get()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 949, in _do_get
    return self._create_connection()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 278, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 404, in __init__
    self.connection = self.__connect()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 530, in __connect
    connection = self.__pool._creator()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 95, in connect
    connection_invalidated=invalidated
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 89, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 376, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 179, in connect
    connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  the database system is in recovery mode
FATAL:  the database system is in recovery mode
 None None
Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/paster", line 11, in <module>
    sys.exit(run())
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 102, in run
    invoke(command, command_name, options, args[1:])
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 141, in invoke
    exit_code = runner.run(args)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/paste/script/command.py", line 236, in run
    result = self.command()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 139, in command
    self._load_config()
  File "/usr/lib/ckan/default/src/ckanext-harvest/ckanext/harvest/commands/harvester.py", line 212, in _load_config
    super(Harvester, self)._load_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/lib/cli.py", line 161, in _load_config
    load_environment(conf.global_conf, conf.local_conf)
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 97, in load_environment
    p.load_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 129, in load_all
    unload_all()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 182, in unload_all
    unload(*reversed(_PLUGINS))
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 210, in unload
    plugins_update()
  File "/usr/lib/ckan/default/src/ckan/ckan/plugins/core.py", line 121, in plugins_update
    environment.update_config()
  File "/usr/lib/ckan/default/src/ckan/ckan/config/environment.py", line 266, in update_config
    model.init_model(engine)
  File "/usr/lib/ckan/default/src/ckan/ckan/model/__init__.py", line 157, in init_model
    version_table = Table('migrate_version', meta.metadata, autoload=True)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 352, in __new__
    table._init(name, metadata, *args, **kw)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 425, in _init
    self._autoload(metadata, autoload_with, include_columns)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 448, in _autoload
    self, include_columns, exclude_columns
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1670, in run_callable
    with self.contextual_connect() as conn:
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1737, in contextual_connect
    self.pool.connect(),
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 332, in connect
    return _ConnectionFairy._checkout(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 630, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in checkout
    rec = pool._do_get()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 949, in _do_get
    return self._create_connection()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 278, in _create_connection
    return _ConnectionRecord(self)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 404, in __init__
    self.connection = self.__connect()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 530, in __connect
    connection = self.__pool._creator()
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 95, in connect
    connection_invalidated=invalidated
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 89, in connect
    return dialect.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 376, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 179, in connect
    connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  the database system is in recovery mode
FATAL:  the database system is in recovery mode
 None None

Actions #1

Updated by Andrea Dell'Amico over 7 years ago

  • Status changed from New to In Progress

The database is working, but last night it exhausted all the memory so some processes were killed by the OOM. We should increase the RAM of that VM, let us know when we can reboot it. A downtime of a couple of minutes is required.

Actions #2

Updated by Francesco Mangiacrapa over 7 years ago

Andrea Dell'Amico wrote:

The database is working, but last night it exhausted all the memory so some processes were killed by the OOM. We should increase the RAM of that VM, let us know when we can reboot it. A downtime of a couple of minutes is required.

Yes, please perform it. Thanks.

Actions #3

Updated by Andrea Dell'Amico over 7 years ago

postgres now is running with 8 GB of ram. For a better job we should also increase the shared memory, now.

Actions #4

Updated by Andrea Dell'Amico over 7 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

The shared memory has been increased to 3GB.

Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)