Bug #24162
closed
Publishing of SDG records not allowed in GRSF Admin VRE
100%
Description
I tried publishing some new records under FAO SDG 14.4.1 Questionnaire in GRSF Admin, but the service fails with the following error:
Method failed: HTTP/1.1 500 ; Response body: {"id":null,"knowledge_base_id":null,"product_url":null,"error":"Error while performing GET. Request url was: https://ckan-grsf-admin.d4science.org//api/3/action/organization_show?id=sdg&include_datasets=false&include_users=true CkanResponse{error=Ckan error of type: Not Found Error message:Not found Other fields:{}, success=false, help=https://ckan-grsf-admin2.d4science.org/api/3/action/help_show?name=organization_show} CkanClient{catalogURL=https://ckan-grsf-admin.d4science.org, ckanToken=**MASKED_TOKEN****}"}
According to the comment in #24035#note-8 SDG publishing should be supported in GRSF Admin. Could you please check?
Files
Updated by Luca Frosini over 2 years ago
- Status changed from In Progress to Paused
Updated by Luca Frosini over 2 years ago
- Status changed from Paused to Feedback
- % Done changed from 0 to 100
@marketak@ics.forth.gr sorry for the inconvenience. Can you please retry now?
Updated by Yannis Marketakis over 2 years ago
Thanks @luca.frosini@isti.cnr.it.
Now the records are published normally, although SDG records seem to be private and I cannot see them, but I suspect this is another issue (related to the catalogue configuration) so I will open a different ticket for this.
Updated by Luca Frosini over 2 years ago
Yannis Marketakis wrote in #note-5:
Thanks @luca.frosini@isti.cnr.it.
Now the records are published normally, although SDG records seem to be private and I cannot see them, but I suspect this is another issue (related to the catalogue configuration) so I will open a different ticket for this.
This is really really strange.
@francesco.mangiacrapa@isti.cnr.it can you help me to understand the Ckan issue
Updated by Luca Frosini over 2 years ago
@marketak@ics.forth.gr maybe I have found the issue and if I'm right even the group association could work.
Can you please try again? Sorry for the inconvenience.
Updated by Luca Frosini over 2 years ago
I have seen that the Record has been created successfully now, but the issue on the group is not solved.
It is a very complicated issue to solve with the actual code which uses many naming conventions instead of configuration.
In the code there where the assumption that the Organization Display Name is used as id (with lowercase characters).
The organization name is also used to associate the record with the group.
This convention property works when the sources of the record were ["FIRMS", "FishSource", "RAM"].
SDG breaks this paradigm because the required Organization Display Name is FAO SDG 14.4.1 Questionnaire.
The code needs deep refactoring to support such a case.
Updated by Pasquale Pagano over 2 years ago
Could we use FAO_SDG_14_4_1_Questionnaire ?
Not the best but this could solve the issue if I correctly understood it.
Updated by Yannis Marketakis over 2 years ago
Currently we face the following when publishing/updating records using https://smart-grsf-d4s.d4science.org/grsf-publisher-ws/rest/grsf/stock/publish-product?
Method failed: HTTP/1.1 502 Bad Gateway; Response body: <!DOCTYPE html>
Error
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
An error occurred.
Sorry, the page you are looking for is currently unavailable.
Please try again later.
If you are the system administrator of this resource then you should check
the error log for details.
Faithfully yours, nginx.
Updated by Luca Frosini over 2 years ago
Yannis Marketakis wrote in #note-10:
Currently we face the following when publishing/updating records using https://smart-grsf-d4s.d4science.org/grsf-publisher-ws/rest/grsf/stock/publish-product?
Method failed: HTTP/1.1 502 Bad Gateway; Response body: <!DOCTYPE html>
Error
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
An error occurred.
Sorry, the page you are looking for is currently unavailable.
Please try again later.
If you are the system administrator of this resource then you should check
the error log for details.
Faithfully yours, nginx.
@marketak@ics.forth.gr the host is changed https://smart-grsf-admin.d4science.org. Sorry if I did not advise you
Updated by Yannis Marketakis over 2 years ago
@luca.frosini@isti.cnr.it Has it changed for GRSF VRE only or for GRSF Admin as well ?
Updated by Yannis Marketakis over 2 years ago
Moreover, publishing or updating using the new host raises the following:
Method failed: HTTP/1.1 400 ; Response body: Error (400) : this resource cannot process this request because it is malformed
Stacktrace:
org.gcube.smartgears.handlers.application.request.RequestException: GRSFPublisher cannot be called in scope /d4science.research-infrastructures.eu/FARM/GRSF
at org.gcube.smartgears.handlers.application.request.RequestError.toException(RequestError.java:110)
at org.gcube.smartgears.handlers.application.request.RequestError.fire(RequestError.java:94)
at org.gcube.smartgears.handlers.application.request.RequestValidator.validateScopeCall(RequestValidator.java:106)
at org.gcube.smartgears.handlers.application.request.RequestValidator.handleRequest(RequestValidator.java:68)
at org.gcube.smartgears.handlers.application.RequestHandler.onEvent(RequestHandler.java:58)
at org.gcube.smartgears.handlers.application.RequestHandler.onEvent(RequestHandler.java:21)
at org.gcube.smartgears.handlers.Pipeline.forward(Pipeline.java:65)
at org.gcube.smartgears.handlers.Pipeline.forward(Pipeline.java:72)
at org.gcube.smartgears.managers.RequestManager.doFilter(RequestManager.java:89)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:679)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Updated by Luca Frosini over 2 years ago
Yannis Marketakis wrote in #note-13:
Moreover, publishing or updating using the new host raises the following:
Method failed: HTTP/1.1 400 ; Response body: Error (400) : this resource cannot process this request because it is malformed
Stacktrace:
org.gcube.smartgears.handlers.application.request.RequestException: GRSFPublisher cannot be called in scope /d4science.research-infrastructures.eu/FARM/GRSF
at org.gcube.smartgears.handlers.application.request.RequestError.toException(RequestError.java:110)
at org.gcube.smartgears.handlers.application.request.RequestError.fire(RequestError.java:94)
at org.gcube.smartgears.handlers.application.request.RequestValidator.validateScopeCall(RequestValidator.java:106)
at org.gcube.smartgears.handlers.application.request.RequestValidator.handleRequest(RequestValidator.java:68)
at org.gcube.smartgears.handlers.application.RequestHandler.onEvent(RequestHandler.java:58)
at org.gcube.smartgears.handlers.application.RequestHandler.onEvent(RequestHandler.java:21)
at org.gcube.smartgears.handlers.Pipeline.forward(Pipeline.java:65)
at org.gcube.smartgears.handlers.Pipeline.forward(Pipeline.java:72)
at org.gcube.smartgears.managers.RequestManager.doFilter(RequestManager.java:89)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:660)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:679)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:806)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
You are using the scope of GRSf and not GRSF Admin.
In such a case, you must use the following host https://smart-grsf-d4s.d4science.org
Updated by Luca Frosini over 2 years ago
To summarize, you must use the following hosts for the corresponding context:
PreProduction infrastructure
| GRSF Publisher Service URI | Scope | Catalogue URL |
|---|---|---|
| https://smart-grsfpre.pre.d4science.org | /pred4s/preprod/GRSF_Pre | https://pre.d4science.org/group/grsf_pre |
Production infrastructure
| GRSF Publisher Service URI | Scope | Catalogue URL |
|---|---|---|
| https://smart-grsfpre.d4science.org | /d4science.research-infrastructures.eu/FARM/GRSF_Pre | https://blue-cloud.d4science.org/group/grsf_pre/grsf_pre |
| https://smart-grsf-admin.d4science.org | /d4science.research-infrastructures.eu/FARM/GRSF_Admin | https://i-marine.d4science.org/group/grsf_admin |
| https://smart-grsf-d4s.d4science.org | /d4science.research-infrastructures.eu/FARM/GRSF | https://i-marine.d4science.org/group/grsf |
Updated by Yannis Marketakis over 2 years ago
Luca Frosini wrote in #note-15:
To summarize, you must use the following hosts for the corresponding context:
PreProduction infrastructure
GRSF Publisher Service URI Scope Catalogue URL https://smart-grsfpre.pre.d4science.org /pred4s/preprod/GRSF_Pre https://pre.d4science.org/group/grsf_pre Production infrastructure
GRSF Publisher Service URI Scope Catalogue URL https://smart-grsfpre.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF_Pre https://blue-cloud.d4science.org/group/grsf_pre/grsf_pre https://smart-grsf-admin.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF_Admin https://i-marine.d4science.org/group/grsf_admin https://smart-grsf-d4s.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF https://i-marine.d4science.org/group/grsf
These are the ones we already use. However, when we use the service to publish in GRSF VRE we face the HTTP 502 reported in #24162#note-10
Moreover, I noticed that https://smart-grsf-d4s.d4science.org/grsf-publisher-ws/rest seems to be down.
Updated by Luca Frosini over 2 years ago
Yannis Marketakis wrote in #note-16:
Luca Frosini wrote in #note-15:
To summarize, you must use the following hosts for the corresponding context:
PreProduction infrastructure
GRSF Publisher Service URI Scope Catalogue URL https://smart-grsfpre.pre.d4science.org /pred4s/preprod/GRSF_Pre https://pre.d4science.org/group/grsf_pre Production infrastructure
GRSF Publisher Service URI Scope Catalogue URL https://smart-grsfpre.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF_Pre https://blue-cloud.d4science.org/group/grsf_pre/grsf_pre https://smart-grsf-admin.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF_Admin https://i-marine.d4science.org/group/grsf_admin https://smart-grsf-d4s.d4science.org /d4science.research-infrastructures.eu/FARM/GRSF https://i-marine.d4science.org/group/grsf These are the ones we already use. However, when we use the service to publish in GRSF VRE we face the HTTP 502 reported in #24162#note-10
Moreover, I noticed that https://smart-grsf-d4s.d4science.org/grsf-publisher-ws/rest seems to be down.
@andrea.dellamico@isti.cnr.it @roberto.cirillo@isti.cnr.it can you please check?
I'm able to login to the host via ssh.
Looking at the access log seems that the last served request was on 2022-11-22.
2022-11-22 16:52:16,243 [catalina-exec-1] INFO RequestAccounting: REQUEST SERVED ON GRSFPublisher:Data-Catalogue(POST /rest/grsf/stock/publish-product) CALLED FROM grsf.publisher@139.91.183.96 IN SCOPE /d4science.research-infrastructures.eu/FARM/GRSF SUCCEDED(CODE 201) IN 5000 millis
Updated by Andrea Dell'Amico over 2 years ago
I'm going to check the service and why no alert was emitted.
Updated by Luca Frosini over 2 years ago
Andrea Dell'Amico wrote in #note-19:
The service is back online.
Thanks a lot.
@marketak@ics.forth.gr can you please reply?
Updated by Yannis Marketakis over 2 years ago
- Status changed from Feedback to Closed
I've just tested it and it works as expected.
Thanks @luca.frosini@isti.cnr.it and @andrea.dellamico@isti.cnr.it
