7 October 2020 - Finalizing the new data harvest workflow¶
- Table of contents
- 7 October 2020 - Finalizing the new data harvest workflow
Meeting Notes¶
Participants:
FORTH (Yannis Marketakis)
FAO (Bracken van Niekerk, Anton Ellenbroek, Aureliano Gentile)
Main topics¶
This meeting follows the new data harvest and release in July 2020. A review with several chats took place during the summer to assess the new harvesting logic up. In this meeting a few steps were discussed to ensure a proper refresh of the GRSF knowledge base while avoiding any misalignment and errors.
This meeting discussed:
- Records with missing species
- Obsolete records
- Publishing workflow
- Management Panel
Records with missing species¶
It was decided to skip this record, they will stay in the KB but not accessible through the catalogue. The FIRMS Secretariat will consider the possibility to add species references to those records (e.g. FAO regional reviews).
Obsolete records¶
It was decided to provide a list of obsolete records (e.g. caused by server malfunctions) so the GRSF team can review before the next step of the publishing workflow.
Publishing workflow¶
The following step were discussed and approved:
- Data harvest from the Source databases
- Transformation of harvested resources.
- Construction of GRSF KB using the transformed records. During this process, a set of obsolete records will be identified. They refer to "legacy" records that exist in the previous version of the GRSF KB, but are not amongst the new harvested resources. Obsolete "Legacy" records will be provided to data source providers for examination.
- Construction of GRSF records using the previous version of GRSF KB. During this process, a set of obsolete records will be identified. Obsolete GRSF records refer to records that were publicly available in the previous version of the GRSF KB (i.e. the ones with status approved). Obsolete "GRSF" records will be provided to data source providers for examination.
- Publish all records in GRSF Pre
- Publish all records in GRSF Admin and the public ones in GRSF Public (after reviewing them in GRSF Pre).
Management Panel¶
It was recalled the importance to have the management panel up and running. Some tests are also needed since it was not working for more than a year.
This feature is critical for the GRSF partners to manage their own records.
Resources¶
- GRFS Pilot release (public) https://i-marine.d4science.org/web/grsf/data-catalogue
- GRSF API
- GRSF Competency queries: https://i-marine.d4science.org/group/grsf_admin/grsf-competency-queries
- "GRSF Admin": https://i-marine.d4science.org/group/grsf_admin
- "GRSF VRE": https://i-marine.d4science.org/group/grsf/data-catalogue
- API to return GRSF information for a specific stock https://support.d4science.org/projects/stocksandfisherieskb/wiki/GRSF_API#API-for-FIRMS