Project

General

Profile

Actions
Copy link

Feature #23449

open

The GRSF Management Panel must be made available also in the "GRSF_Pre" VRE

Added by Francesco Mangiacrapa almost 3 years ago. Updated almost 3 years ago.

Status:
Feedback
Priority:
Normal
Assignee:
Francesco Mangiacrapa
Target version:
GRSF
Start date:
Jun 21, 2022
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)

Description

The GRSF Management Panel already available in the " GRSF_Admin " VRE must be deployed and made available also in the " GRSF_Pre " VRE.

Files

Download all files
Error_on_displaying_Manage_Panel_in_GRSF_Pre.png (14.7 KB) Error_on_displaying_Manage_Panel_in_GRSF_Pre.png Francesco Mangiacrapa, Jun 08, 2022 03:51 PM
404_Error_On_Accessing_To_GRSF_Service_Updater_Of_DEV.png (37.5 KB) 404_Error_On_Accessing_To_GRSF_Service_Updater_Of_DEV.png Francesco Mangiacrapa, Jun 16, 2022 11:04 AM
clipboard-202206201248-lf0hn.png (11 KB) clipboard-202206201248-lf0hn.png Yannis Marketakis, Jun 20, 2022 11:48 AM
clipboard-202206201249-ypuio.png (13.2 KB) clipboard-202206201249-ypuio.png Yannis Marketakis, Jun 20, 2022 11:49 AM

Subtasks 2 (0 open — 2 closed)

Bug #23549: GRSF_Manage_Widget: serialization issue on the GRSFRecordAlreadyManagedStatusException java objectClosedFrancesco MangiacrapaJun 21, 2022

Actions
Bug #23561: GRSF_Manage_Widget: the Merging Request throws a null pointer exceptionClosedFrancesco MangiacrapaJun 22, 2022

Actions

Related issues

Related to StocksAndFisheriesKB - Task #23486: Remove records from GRSF PREClosedYannis MarketakisJun 09, 2022Jun 09, 2022

Actions
Actions
Copy link
 #1

Updated by Francesco Mangiacrapa almost 3 years ago

Hi @marketak@ics.forth.gr,

as a first step, kindly provide the service endpoint that will be used to contact the KB on the merging requests that will be performed (by the Management Panel) from the GRSF_Pre environment.

At the moment, we have the following "Service Endpoint" registered in the GRSF_ADMIN VRE:

      <AccessPoint>
         <Description>The base url of the GRSF-Updater service</Description>
         <Interface>
            <Endpoint EntryName="GRSF Update service">http://athena.ics.forth.gr:9000/grsf-services-updater</Endpoint>
         </Interface>
      </AccessPoint>
Actions
Copy link
 #2

Updated by Yannis Marketakis almost 3 years ago

  • Status changed from New to In Progress

Hi @francesco.mangiacrapa@isti.cnr.it

It's better if we use https://isl.ics.forth.gr/grsf/grsf-services-updater/ which is more permanent

Actions
Copy link
 #3

Updated by Francesco Mangiacrapa almost 3 years ago

Yannis Marketakis wrote in #note-2:

Hi @francesco.mangiacrapa@isti.cnr.it

It's better if we use https://isl.ics.forth.gr/grsf/grsf-services-updater/ which is more permanent

OK, should it be used for both (GRSF_Pre and GRSF_Admin)? Or just in GRSF_Pre (available at https://i-marine.d4science.org/group/grsf_pre/data-catalogue)?

Actions
Copy link
 #4

Updated by Yannis Marketakis almost 3 years ago

It could be used for both VREs.

Technically, https://isl.ics.forth.gr/grsf/grsf-services-updater/ is a permanent URL and is not subject to changes. So it's preferred to use it everywhere

Francesco Mangiacrapa wrote in #note-3:

OK, should it be used for both (GRSF_Pre and GRSF_Admin)? Or just in GRSF_Pre (available at https://i-marine.d4science.org/group/grsf_pre/data-catalogue)?

Actions
Copy link
 #5

Updated by Luca Frosini almost 3 years ago

Yannis Marketakis wrote in #note-4:

It could be used for both VREs.

Technically, https://isl.ics.forth.gr/grsf/grsf-services-updater/ is a permanent URL and is not subject to changes. So it's preferred to use it everywhere

Francesco Mangiacrapa wrote in #note-3:

OK, should it be used for both (GRSF_Pre and GRSF_Admin)? Or just in GRSF_Pre (available at https://i-marine.d4science.org/group/grsf_pre/data-catalogue)?

@marketak@ics.forth.gr can you please confirm again that the endpoint is fine for GRSF_Pre too. Are you able to differentiate the context of the request by looking at the token?
I'm asking because I understand that GRSF_Pre is another KB with respect to GRSF/GRSF_Admin.

Actions
Copy link
 #6

Updated by Yannis Marketakis almost 3 years ago

@luca.frosini@isti.cnr.it thanks for insisting on this.
You are probably right. Let's use another service for GRSF PRE. Below I'm listing the URLs of the services and the VREs they refer to

Actions
Copy link
 #7

Updated by Francesco Mangiacrapa almost 3 years ago

Hi all,

I configured the GRSF_Pre VRE in order to provide the "Manage GRSF Item" facility, see at https://blue-cloud.d4science.org/group/grsf_pre/data-catalogue.

Unfortunately, by testing it, I noticed an issue (see the screenshot)... and in order to fix it, @marketak@ics.forth.gr a new publishing of the records in the GRSF_Pre VRE is kindly required.

If you remember, the latest republishing of the GRSF records, started from the GRSF_Admin VRE. In this context, a revisited grsf-service has been deployed with a new mapping on the GRSF fields. This work has been done by @luca.frosini@isti.cnr.it and @aureliano.gentile@fao.org, they can confirm...

E.g. the field "Stock Name" has been renamed to "GRSF Stock Name", and so on..., so in the GRSF_Admin we have records with the field "GRSF Stock Name" (e.g. "GRSF Stock Name" = "Metapenaeus monoceros - EEZ IDN", see https://data.d4science.org/ctlg/GRSF_Admin/b67597e5-c9b2-385e-a80b-13d3826f8e81)

The records published in the GRSF_Pre are still in the "old format" with the "old fields names", so this causes a mismatching between "the expected fields names" and "the registered fields names" in the GRSF_Pre VRE.
(The expected fields names are stored into two Generic Resources ("GRSF Stock" and "GRSF Fishery") registered in the Information System).

E.g. In the GRSF_Pre VRE, the field name is still "Stock Name"(e.g. "Stock Name" = "Kajikia audax - 77 - 87", see https://data.d4science.org/ctlg/GRSF_Pre/b9a5ac80-a001-3c20-99ec-6ce387365823)

I hope I have clearly explained the problem.

A purge and republishing of the records registered in the GRSF_Pre VRE is the first step required, and then (I hope) we will be able to test the "Manage GRSF Panel" in the GRSF_Pre.

Actions
Copy link
 #8

Updated by Aureliano Gentile almost 3 years ago

For what I am concerned we are using the GRSF PRE for testing the new records "FAO SDG 14.4.1 Questionnaire", therefore all records can be deleted anytime and we will be loading sdg records via the new input form after your purge.

Actions
Copy link
 #9

Updated by Yannis Marketakis almost 3 years ago

Dear @francesco.mangiacrapa@isti.cnr.it your explanation is perfectly clear.
It is not a problem we will re-publish everything in GRSF PRE again. In fact for testing the management panel we will publish legacy records and GRSF Stocks and publish GRSF Fisheries (that are too many) later on.

@aureliano.gentile@fao.org yes you are right. GRSF PRE is still a staging environment (for GRSF purposes) therefore we can move as suggested.

I'm starting deletions right away. I'll let you as soon as, everything has been removed (#23486).

Actions
Copy link
 #10

Updated by Francesco Mangiacrapa almost 3 years ago

  • Related to Task #23486: Remove records from GRSF PRE added
Actions
Copy link
 #11

Updated by Francesco Mangiacrapa almost 3 years ago

The GRSF_Pre Catalogue is empty, up & running at https://blue-cloud.d4science.org/group/grsf_pre/data-catalogue

The configurations required to show the "GRSF Management Panel" in the GRSF_Pre should be OK.

Now, two actions are required:

  1. @luca.frosini@isti.cnr.it please check if the (CKAN) groups registered in the GRSF_Pre Catalogue are updated. If not, please update them.

  2. (After the green light by Luca), @marketak@ics.forth.gr you may republish few records in GRSF_Pre, then we could test the Management Panel.

Actions
Copy link
 #12

Updated by Aureliano Gentile almost 3 years ago

In this occasion, @luca.frosini@isti.cnr.it please consider whether to add methods as group see ticket https://support.d4science.org/issues/23409 (just in case ;-)

Actions
Copy link
 #13

Updated by Luca Frosini almost 3 years ago

I have deleted the old groups and created the new ones

Actions
Copy link
 #14

Updated by Francesco Mangiacrapa almost 3 years ago

Luca Frosini wrote in #note-13:

I have deleted the old groups and created the new ones

Thanks @luca.frosini@isti.cnr.it.

@marketak@ics.forth.gr kindly (when you want) republish a few records in GRSF_Pre, so we can proceed with testing the "GRSF Management Panel". Thanks.

Actions
Copy link
 #15

Updated by Yannis Marketakis almost 3 years ago

Thanks @francesco.mangiacrapa@isti.cnr.it and @luca.frosini@isti.cnr.it .

I've just started

Actions
Copy link
 #16

Updated by Yannis Marketakis almost 3 years ago

@francesco.mangiacrapa@isti.cnr.it

I've published some records.
Can you confirm that you are using https://dev.isl.ics.forth.gr/grsf/grsf-services-updater/ in GRSF PRE?
I've just tried to update the record and no request has arrived in https://dev.isl.ics.forth.gr/grsf/grsf-services-updater/

Actions
Copy link
 #17

Updated by Francesco Mangiacrapa almost 3 years ago

Hi @marketak@ics.forth.gr,

yes, I can confirm that the request to https://dev.isl.ics.forth.gr/grsf/grsf-services-updater/service/updater/post
has been performed with the JSON object:

{"old_status":"approved","annotation_message":"","new_status":"approved","knowledge_base_id":"03967d11-f2bd-3d8e-a9ef-af211100da93","similar_grsf_records":[],"grsf_type_old":"Assessment Unit","short_name_new":"Megrim - Bay of Biscay South and Portuguese Waters East - updated","catalog_id":"e0da71e6-2d83-45d6-8994-99dd3170144d","sdg_flag":false,"grsf_type_new":"Assessment Unit","traceability_flag":false,"administrator_name":"Yannis Marketakis","short_name_old":"Megrim - Bay of Biscay South and Portuguese Waters East","connections":[]}

but I got an error: "There was a problem while performing this operation at knowledge base side"

See the following logs:

2022-06-16 08:35:11,350 INFO  o.g.d.g.s.m.GRSFUpdaterServiceClient [ajp-bio-8009-exec-37,updateKB:146][PORTAL] 1708842150 [ajp-bio-8009-exec-37] INFO  org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFUpdaterServiceClient - Update request looks like {"old_status":"approved","annotation_message":"","new_status":"approved","knowledge_base_id":"03967d11-f2bd-3d8e-a9ef-af211100da93","similar_grsf_records":[],"grsf_type_old":"Assessment Unit","short_name_new":"Megrim - Bay of Biscay South and Portuguese Waters East - updated","catalog_id":"e0da71e6-2d83-45d6-8994-99dd3170144d","sdg_flag":false,"grsf_type_new":"Assessment Unit","traceability_flag":false,"administrator_name":"Yannis Marketakis","short_name_old":"Megrim - Bay of Biscay South and Portuguese Waters East","connections":[]}
2022-06-16 08:35:11,350 INFO  o.g.d.g.s.m.GRSFUpdaterServiceClient [ajp-bio-8009-exec-37,updateKB:148][PORTAL] 1708842150 [ajp-bio-8009-exec-37] INFO  org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFUpdaterServiceClient - Sending request to https://dev.isl.ics.forth.gr/grsf/grsf-services-updater/service/updater/post
2022-06-16 08:35:11,555 ERROR o.g.d.g.s.m.GRSFUpdaterServiceClient [ajp-bio-8009-exec-37,updateKB:165]org.json.simple.parser.ParseException: null
    at org.json.simple.parser.Yylex.yylex(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.json.simple.parser.JSONParser.nextToken(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.json.simple.parser.JSONParser.parse(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.json.simple.parser.JSONParser.parse(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.json.simple.parser.JSONParser.parse(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFUpdaterServiceClient.updateKB(GRSFUpdaterServiceClient.java:163) ~[grsf-manage-widget-1.6.0.jar:1.6.0]
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.Utils.updateRecord(Utils.java:201) [grsf-manage-widget-1.6.0.jar:1.6.0]
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFNotificationService.notifyProductUpdate(GRSFNotificationService.java:400) [grsf-manage-widget-1.6.0.jar:1.6.0]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_242]
    at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.1.jar:na]
    at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.1.jar:na]
    at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.1.jar:na]
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-
....
....
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.62]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242]
[PORTAL] 1708842355 [ajp-bio-8009-exec-37] ERROR org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFUpdaterServiceClient - Failed to parse response from knowledge base
org.json.simple.parser.ParseException: null
    at org.json.simple.parser.Yylex.yylex(Unknown Source) ~[json-simple-1.1.jar:na]
    at org.json.simple.parser.JSONParser.nextToken(Unknown Source) ~[json-simple-1.1.jar:na]
....
....
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_242]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242]
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.62]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242]
2022-06-16 08:35:11,557 ERROR o.g.d.g.s.m.Utils [ajp-bio-8009-exec-37,updateRecord:242]java.lang.Exception: There was a problem while performing this operation at knowledge base side
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFUpdaterServiceClient.updateKB(GRSFUpdaterServiceClient.java:169) ~[grsf-manage-widget-1.6.0.jar:1.6.0]
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.Utils.updateRecord(Utils.java:201) ~[grsf-manage-widget-1.6.0.jar:1.6.0]
    at org.gcube.datacatalogue.grsf_manage_widget.server.manage.GRSFNotificationService.notifyProductUpdate(GRSFNotificationService.java:....
....

Checking by the browser, I noticed that accessing to https://dev.isl.ics.forth.gr/grsf/grsf-services-updater/, I'm getting a 404 Error (see the screenshot). Maybe the URL is not accessible on Internet or the service is down. Could you check? Thanks.

Actions
Copy link
 #18

Updated by Yannis Marketakis almost 3 years ago

My fault,

I deployed the service in a different location (https://dev.isl.ics.forth.gr/grsf-services-updater/).
I'll change that.

Apologies for the inconvenience.

Actions
Copy link
 #19

Updated by Yannis Marketakis almost 3 years ago

I've tested the management panel and it seems that most of the actions work as expected (update of short name, update of flags, status updates, connections, annotation messages).

However, when merging two records, an error message is thrown (shown below).
It is remarkable that the record itself is updated (I see that its new status is to_be_merged) and I do not encounter any errors in the GRSF KB.

Furthermore, If I try to edit the merged record afterwards, I am unable to open the management panel because of the following error message (i.e. try opening the management panel for the record https://data.d4science.org/ctlg/GRSF_Pre/662eb566-2b51-36e4-9ba4-900d06fc8f03)

@francesco.mangiacrapa@isti.cnr.it can you please inform me about the error messages that appear @ server side when open the management panel?

Actions
Copy link
 #20

Updated by Francesco Mangiacrapa almost 3 years ago

Hi @marketak@ics.forth.gr,

try opening the management panel for the record https://data.d4science.org/ctlg/GRSF_Pre/662eb566-2b51-36e4-9ba4-900d06fc8f03, I see the following error on the server side:

org.gcube.datacatalogue.grsf_manage_widget.shared.ex.GRSFRecordAlreadyManagedStatusException: The record is locked due to a merge request in progress!

Unfortunately, the error did not reach the client (so the management panel did not display it) due to a serialization issue on the GRSFRecordAlreadyManagedStatusException java object. I will fix it.

Regarding the first issue: "...when merging two records, an error message is thrown...". Could you retry with another record and then notify me? Pasting here the Record URL. Thanks.

Actions
Copy link
 #21

Updated by Yannis Marketakis almost 3 years ago

Thanks @francesco.mangiacrapa@isti.cnr.it

Since you want to inspect the logs, I guess you could try to merge https://data.d4science.org/ctlg/GRSF_Pre/517f9fe8-1c51-3fd0-a14e-df6f2ba76583 with 5b2b0e53-1804-3faf-957d-ff23b8c17844. This way you can see live the logs

The steps to do so are:

Actions
Copy link
 #22

Updated by Francesco Mangiacrapa almost 3 years ago

Yannis Marketakis wrote in #note-21:

Thanks @francesco.mangiacrapa@isti.cnr.it

Since you want to inspect the logs, I guess you could try to merge https://data.d4science.org/ctlg/GRSF_Pre/517f9fe8-1c51-3fd0-a14e-df6f2ba76583 with 5b2b0e53-1804-3faf-957d-ff23b8c17844.
...
...

Many thanks @marketak@ics.forth.gr. I just replicated the issue reported by you in the comment #note-19. It is a bug (see #23561). I will fix it asap.

Actions
Copy link
 #24

Updated by Roberto Cirillo almost 3 years ago

This status is blocking the release closure. I'm going to detach from release ticket.

Actions
Copy link
 #26

Updated by Francesco Mangiacrapa almost 3 years ago

  • Status changed from In Progress to Feedback

Hi @marketak@ics.forth.gr,

today I'm back to work (I was on holiday) and I noticed that last week you performed some tests on the "GRSF Manage Panel" in the GRSF_Pre VRE. Does it work as expected?

Actions
Copy link
 #27

Updated by Yannis Marketakis almost 3 years ago

Hi @francesco.mangiacrapa@isti.cnr.it
I hope you had a nice and relaxing vacation period.

I tried testing the management panel, however, I encountered a HTTP 500 error (which occurred when the record was about to be updated in the catalogue). I opened a detailed ticket about this (#23679)

Actions
Copy link

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)