D4Science Infrastructure

As usual, it was nice to talk with VLIZ technicians: they have said that empty families and genus are normal in WoRMS, because, for example, they do not harvest FishBase completely. They are not going to look into the issues we have highlighted, thus we should solve them by ourselves. I think that the DWCA process on SPD should be revised to avoid failure in the case of empty taxonomic branches. Perhaps, avoiding a consistency check for the DWCA could work for the issues listed in the errors file.

We are struggling to produce the DWCA for some crucial species (ca. 14400) but there are some issues:

1 - there are some species (e.g. WoRMS:183256) that are no more on the WoRMS system but, although the SPD service reports an exception internally, it continues to stay in the "running" state indefinitely;
2 - after some requests, the SPD service leaves all the requests in a "pending" state. I guess this is related to point 1 somehow.

I can see if I can find a workaround at client side, but I guess this issue affects the SPD work.

It seems something happens when the service meets a taxonomic name which is either "quarantined" or deleted. Further, at a certain point the service does not accept jobs anymore and I see this exception in the ghn.log:

16:08:42.968 [spd-job-thread-9] WARN  AbstractLocalReader: the queue is empty
java.lang.InterruptedException: null
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017) ~[na:1.7.0_80]
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095) ~[na:1.7.0_80]
        at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389) ~[na:1.7.0_80]
        at org.gcube.data.spd.plugin.fwk.readers.LocalReader.hasNext(LocalReader.java:21) ~[spd-plugin-framework-3.1.0-4.3.0-144690.jar:na]
        at org.gcube.data.spd.executor.jobs.dwca.MapDwCA.createTaxaTxt(MapDwCA.java:135) [MapDwCA.class:na]
        at org.gcube.data.spd.executor.jobs.dwca.MapDwCA.createDwCA(MapDwCA.java:45) [MapDwCA.class:na]
        at org.gcube.data.spd.executor.jobs.dwca.DWCAJobByIds.execute(DWCAJobByIds.java:93) [DWCAJobByIds.class:na]
        at org.gcube.data.spd.executor.jobs.SpeciesJob.run(SpeciesJob.java:43) [SpeciesJob.class:na]
        at org.gcube.common.authorization.library.AuthorizedTasks$2.run(AuthorizedTasks.java:75) [common-authorization-2.0.2-4.3.0-144378.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_80]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_80]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]

I have run a web crawler that excludes deleted and quarantined names from the submitted IDs, but we still get job pending issues.

I also see many accounting issues in some threads:

java.lang.NullPointerException: null
        at org.gcube.data.spd.executor.jobs.SpeciesJob.generateAccounting(SpeciesJob.java:57) [SpeciesJob.class:na]
        at org.gcube.data.spd.executor.jobs.SpeciesJob.run(SpeciesJob.java:48) [SpeciesJob.class:na]
        at org.gcube.common.authorization.library.AuthorizedTasks$2.run(AuthorizedTasks.java:75) [common-authorization-2.0.2-4.3.0-144378.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_80]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_80]

Shall we continue to investigate this issue? We will need to periodically update our resources and we cannot lose weeks each time. Valentina, could you please make a plan (perhaps together with Lucio) to investigate the issue?