Project

General

Profile

GRSF ยป

30-31 May 2022 Discussing GRSF developments

Participants:

CNR (Massimiliano Assante, Luca Frosini, Francesco Mangiacrapa, Pasquale Pagano)
FAO (Aureliano Gentile)
FORTH (Yannis Marketakis)

Meeting Notes

Blue-Cloud All-Hands meeting was an occasion for meeting in person CNR, FORTH and IRD colleagues to lively discuss ongoing developments and future prospects. See hereafter the detailed agenda of the side events and the main decisions/recommendations.

Day 1: 9am to 11.30 am (CNR headquarters); Day 2: 2pm to 4 pm (CNR Headquarters)

1.Status of the collaboration among CNR, FAO and FORTH
2.Status of the GRSF VREs and applications
3.What is further needed to make more efficient / faster the refresh and publishing process?
4.CKan features

4.a FAO SDG 14.4.1 Questionnaire (new source of data, import interface, data exposure with confidential aspects)
4.b Traceability Units - user scenario
4.c Standards on areas (any integration among CKan Git, GRSF KB)
4.d Management Panel
4.e Public user Interface improvements (options to explore, new features in the catalogue, etc..)

5.GRSF in the coming years - Building a community for the next 5 years under iMarine / FIRMS partnership

1. Status of the collaboration among CNR, FAO and FORTH : the group highlighted the importance of gathering after more than two years of remote working, with the need of re-engagement by lively exchange on the technical aspects of the collaboration. Mr Pagano highlighted that he is expecting an update of the CNR-FAO MoU currently in progress with Mr Ellenbroek. In general it is preferred to have fewer VREs with more focused work on selected products with the possibilities to dedicate more effort to such concrete activities. The in-kind collaboration needs clear justifications and the time dedicated is anyhow documented in the effort sheets of CNR.

Recent enforcement in security protocols, EU data policies (GDPR), handling of registries etc. require more and more effort to maintain VREs with high levels of interoperability among different systems.
In terms of reducing VREs, the RO (AG) indicated that the VME VRE can be soon dismissed. An analysis of the other VREs under FAO will be made to verify if any other one can be dismissed or reconfigured, merged, etc.

FORTH confirmed the regular maintenance of the GRSF Knowledge Base, the competency queries and the APIs. Bug fixes and minor improvements are included together with periodic data harvests to refresh the KB (with reference to the FAO-FORTH SLA), while substantial modifications of the GRSF KB fall under dedicated LoAs.

Lastly, Mr Pagano suggested the preparation of a Paper on the GRSF VREs powering a distributed system, among multiple data providers sharing information under global, regional and national standards, with focus on today's GRSF features offered including transparent and reproducible analysis.

2. Status of the GRSF VREs and applications : Mr Frosini recalled that due to the recent requirements for confidential data in the public GRSF VRE, the CKan application has now distinct behavior in the two environments (Public, Admin), the related endpoints will be communicated to FORTH for the new publishing protocol.

As soon as Mr Luca Frosini will communicate to Mr Yannis Marketakis the new endpoints, FORTH will publish everything again in GRSF public VRE.

3. Improving publishing process : Ckan showed an intrinsic limitation in receiving data for publishing, the application would be upgraded by CNR taking into account all the ad hoc improvements applied to the current instance. In each data harvest FORTH is always improving the overall logic. The addition of the UUIDs in the source databases is also a way to facilitate the process (e.g. for validation purposes).

4.a FAO SDG 14.4.1 Questionnaire: The GRSF data are expected to be soon enriched with this new source for which FAO is the custodian agency. FORTH is developing the Excel based import interface, while the Ckan is expected to be configured accordingly (legacy records and new "Organization"). Recalling that confidentiality in Ckan is handled at provider level, not at record level, the GRSF will allow access to SDG data only to authorized users in the public VRE, while the Admin VRE will expose all data likewise the other sources (FIRMS, FishSource, RAMLDB). The recent changes for the Groups in Ckan is also easing the handling of the new data.

CNR will improve GRSF Publisher to support the new organization (FAO SDG14.4.1 Questionnaire), and deploy that first on GRSF_PRE VRE to test it.

Afterward FORTH will update the GRSF API Import facilities accordingly to construct and publish the records.

Finally, the foreseen (estimated but not limited to) overall number of records to be added under the new organization should be 1K-3K new records.

4.b Traceability Units: A new type of record was created in the KB in support of the traceability use case run by the FishSource team. Traceability Units are identified by dedicated semantic identifier and UUIDs, these are generated in the KB when a stock and a fishery records are connected and flagged for traceability. The CNR team is waiting for the finalization of the requirements (see also GRSF Wiki at https://support.d4science.org/projects/stocksandfisherieskb/wiki/22-03-23-GRSF_traceability_unit), then the development will be carried out in the GRSF Pre for testing purposes.

CNR teams need to investigate deeper the implication of adding this new type. We need to support a new URL e.g. /traceability-unit, create configurations, etc. The "Domain" metadata will be probably hidden.

4.c Standards on areas (any integration among Ckan, Git, GRSF KB) : this matter was not discussed. However the work is proceeding within the GRSF KB and the national standards are progressively added in the source databases, thus reducing the overall number of unknown area codes in the catalogue.

4.d Management Panel : the management panel will be finalized and made it ready for testing in the GRSF Pre environment. Obsolete tickets were closed and the thread will be updated according to new feedback. CNR will investigate how to add the panel to GRSF_Pre. FORTH has to construct the Traceability Unit when the management panel will send the traceability flag for connected record. This is actually done in bulk when refreshing the knowledge base so they have to investigate on how to implement this new on-the-fly feature.

4.e Public user Interface improvements : Mr Pagano suggested a dash board in the VREs powered by Data Studio with key indicators and some configurations available to end users. Such dashboard would work with SparQL connectors and FORTH is invited to explore its feasibility. This feature will be realized if there is a working SparQL connector in the marketplace, or alternative solutions such as producing intermediate content which can be read by Data Studio.

5. GRSF in the coming years - Building a community for the next 5 years under iMarine / FIRMS partnership : the overall governance of the GRSF was recalled highlighting the progress made to make available the GRSF to those stakeholders involved in stock status determination for their use and relevant analysis. Moreover, the development of the Traceability Unit will be functional to leverage the community involved in fishery traceability matters. In this regard, the need of a powerful application is key for the engagement of such GRSF communities.

Actions

Tickets will be created according to the decisions taken, in brief:
CNR to finalize the behavior of the GRSF public VRE and provide instructions to FORTH for the new records publishing protocol.
GRSF Pre VRE is configured to test:

  • SDG 14.4.1 records and data
  • Traceability Units records
  • Management Panel

CNR is looking forward to receiving specification from FAO for the Traceability Unit for starting the implementation in the Ckan catalogue.
FAO to draft elements of the dashboard for possible implementation.

Resources

Add picture from clipboard (Maximum size: 8.91 MB)