Support #17934
closedProvide a GRSF enviroment on preproduction to test the publishing of GRSF records by grsf-services
100%
Description
The GRSF-PRE catalogue provided for testing is https://ckan-grsf.pre.d4science.org (accessible only to a list of public IP, the public access is denied). The catalogue is reachable by preproduction gateway at https://pre.d4science.org/group/dorne/data-catalogue (the VRE is /pred4s/preprod/Dorne
).
We have already generated a user (the grsf.publisher
) and a token (ddfda8a9-8594-4344-MASKED-TOKEN
) that have to be used for testing by grsf-services
of preproduction available at http://smart-grsf-d-d4s.d4science.org.
The FORTH (by @marketak@ics.forth.gr) has notified the following error:
2019-11-06 10:24:39 DEBUG GrsfPublisher:84 - publishing URL: https://smart-grsf-d-d4s.d4science.org/grsf-publisher-ws/rest/firms/stock/publish-product?gcube-token=MASKED-TOKEN 2019-11-06 10:24:39 DEBUG GrsfPublisher:88 - publishing content: {"stock_name":"Mackerel icefish - South Georgia","license_id":"CC-BY-SA-4.0","version":1.0,"d...."} 2019-11-06 10:24:39 ERROR GrsfPublisher:99 - Method failed: HTTP/1.1 500 Internal Server Error; Response body: {"id":null,"knowledge_base_id":null,"product_url":null,"error":"Error while performing GET. Request url was: https://ckan-grsf.pre.d4science.org//api/3/action/organization_show?id=firms&include_datasets=false&include_users=true CkanResponse{error=Ckan error of type: Not Found Error message:Not found Other fields:{}, success=false, help=https://ckan-grsf.pre.d4science.org/api/3/action/help_show?name=organization_show} CkanClient{catalogURL=https://ckan-grsf.pre.d4science.org, ckanToken=*****MASKED_TOKEN*******}"} 2019-11-06 10:24:39 ERROR GrsfPublisher:111 - Fatal transport error: java.lang.Exception: Method failed: HTTP/1.1 500 Internal Server Error; Response body: {"id":null,"knowledge_base_id":null,"product_url":null,"error":"Error while performing GET. Request url was: https://ckan-grsf.pre.d4science.org//api/3/action/organization_show?id=firms&include_datasets=false&include_users=true CkanResponse{error=Ckan error of type: Not Found Error message:Not found Other fields:{}, success=false, help=https://ckan-grsf.pre.d4science.org/api/3/action/help_show?name=organization_show} CkanClient{catalogURL=https://ckan-grsf.pre.d4science.org, ckanToken=*****MASKED_TOKEN*******}"}
The reported issue is on the CKAN-side. I believe that is due to the missing organizations (at the moment) on the GRSF-PRE catalogue, the forth is trying to publish into FIRMS
organization (that does not exist).
I'm going to create the list of GRSF organizations needed for publishing (like the GRF-ADMIN organizations, see at http://ckan-grsf-admin.d4science.org/api/action/organization_list)
Files
Updated by Francesco Mangiacrapa over 5 years ago
- Status changed from In Progress to Feedback
- % Done changed from 50 to 100
I just created the GRSF-ADMIN organizations also in the GRSF-PRE (see at https://ckan-grsf.pre.d4science.org/api/action/organization_list) and added the user grsf.publisher
to each one.
Now, on the CKAN side, the configurations should be OK.
@marketak@ics.forth.gr, could you retry publishing?
Updated by Yannis Marketakis over 5 years ago
- Status changed from Feedback to Closed
Thanks @francesco.mangiacrapa@isti.cnr.it .
I've started publishing without errors
Updated by Yannis Marketakis over 5 years ago
- File DorneVRE-capture.jpg DorneVRE-capture.jpg added
- Status changed from Closed to In Progress
I noticed that although the records are published without any issues, I cannot browse them in the catalogue, since they seem to be private to me.
Updated by Francesco Mangiacrapa over 5 years ago
Hi @marketak@ics.forth.gr,
now you (with your personal account) should be able to access the items published on that catalogue, in the organizations:
FIRMS FishSource GRSF Admin RAM
let me know...
Updated by Yannis Marketakis over 5 years ago
- Status changed from In Progress to Closed
I confirm I do.
Many thanks!
Updated by Yannis Marketakis about 5 years ago
- File grsf-publish.log grsf-publish.log added
It seems that we're facing some issues while publishing GRSF records in https://pre.d4science.org/group/dorne/data-catalogue
Although legacy records (records from FIRMS, RAM, FishSource) are published without any issues, for GRSF records, we're facing an Access Denied error.
The detailed log from the GRSF publishing client is attached.
Updated by Francesco Mangiacrapa about 5 years ago
- Status changed from Closed to In Progress
- % Done changed from 100 to 80
Hi Yannis,
could you kindly attach a JSON that you are trying to publish? I suppose that the organisation ID (as field in the JSON) which you are passing to Catalogue does not exist or you are not authorized to publish into it. Should be the organisation ID of "GRSF Admin", right?
Moreover, I just noticed an error returned by gCat on preprod (see at http://gcat.pre.d4science.org/gcat), @roberto.cirillo@isti.cnr.it or @luca.frosini@isti.cnr.it could you check it asap? Many thanks.
Updated by Roberto Cirillo about 5 years ago
I see several calls coming from Dorne VRE:
rejecting call to gcat in invalid context /pred4s/preprod/Dorne, allowed context are [/pred4s/preprod, /pred4s/preprod/preVRE]
gcat is running only in preVRE. @francesco.mangiacrapa@isti.cnr.it if you want I could add gcat to Dorne VRE as well.
Updated by Yannis Marketakis about 5 years ago
@francesco.mangiacrapa@isti.cnr.it please find attached an indicative json file.
I am not sure what you mean with Organization ID. I never used this field.
Updated by Francesco Mangiacrapa about 5 years ago
Yannis Marketakis wrote:
@francesco.mangiacrapa@isti.cnr.it please find attached an indicative json file.
I am not sure what you mean with Organization ID. I never used this field.
Hi Yannis,
you're right, you do not use the field "Organization ID" but the rest interface to push the items in the proper organization directly. That field is used into another context, sorry for this.
In this period, I'm out of office, thanks to @pasquale.pagano@isti.cnr.it, he just added the user "grsf_publisher" to "GRSF PRE" organisation. Now you should be authorized to publish in such organization. Let me know...
Updated by Yannis Marketakis about 5 years ago
- Status changed from In Progress to Closed
- % Done changed from 80 to 100
Thanks @francesco.mangiacrapa@isti.cnr.it and @pasquale.pagano@isti.cnr.it .
I confirm it works.