Project

General

Profile

GRSF »

11 December 2017 - Reviewing the pending issues between FishSource and FORTH to build the GRSF knowledge base

Meeting Notes
Topics: GRSF Admin VRE, GRSF Knowledge base
FishSource (Braddock Spears, Susanna Segurado, Susanna Brian, Merul Patel)
FORTH (Nikos Minadakis)
FAO (Anton Ellenbroek, Giulia Gorelli, Aureliano Gentile)

Notes

Flagged issues:

1) The management entities in most of the cases have null under the acronym and the country tag

This is a data cleanup issue, and I have created a Redmine on our internal system for review: 13914
Some management entities have just the code of SFP instead of the name (eg SFP-5013)
Will change countries to ISO 3
Aureliano notes that maintaining country management entity over time could be a challenge
Will finish by the end of the month
Regarding RFMOs and similar organizations: SFP to maintain current country code for Secretariat location. FORTH will change to “INT” as 3 letter code in the shared database

2) The get SFP gears API is not ready yet (but so far we are using the excel mapping file, so it is not a top priority)

This has been implemented, and v2 of the API now supports the new :sfp_gear field. Brad/Susanna: authors now need to be told that if a ISSCFG code does not apply to a gear, then we must create a new SFP_CODE entry. We can discuss in our weekly call.
Confirmed FAO gears in FishSource are the new ones.

3) Some management entities have just the code of SFP instead of the name (eg SFP-5013)

Data cleanup - SFP should finish data cleaning entries by end of week. Noted “SFP-xxx” is a legacy placeholder and that all such references will be replaced

4) some 3Alpha species code tags are empty

Data cleanup, and new SFP redmine created: 13915
Were able to add in 3 species with ASFIS Code
Then using WORMS as identifier, any other type (n=1) would be unknown
SFP will add field to species indicating the source of the name
Changing API to pull 3 digit alpha name.
Will add attribute by end of year.

5) The reference year tag has not been defined yet

Reference years are exported by two endpoints:

Fisheries https://www.fishsource.org/apipie/v2/fisheries.html. This exposes reference years for 6 catch related variables associated with the given fishery
Data https://www.fishsource.org/apipie/v2/data.html. This exposes reference years for up to 127 (we only usually have a few) variables related to abundance and exploitation associated with the top level of the profile tree, ie, either a Stock, Assessment Unit or a Resource

If desired we could add a ‘reference year’ attribute to the fisheries endpoint, which would report the latest year for which we have any data related to the fishery. We are referring to the definition of the reference year here: https://support.d4science.org/projects/stocksandfisherieskb/wiki/GRSF_Definitions
Alternatively, we could amend the Data endpoint so it returns all data associated at all levels of our profile trees. Note, this will include levels of our trees that you have not processed programmatically to date, ie, nested Assessment Units and Flag Profiles

6) The reporting years (or citation_date) are null in most of the times and the format should be only the YEAR and not the ISO date

  • SFP’s API currently reports reporting year - using two proxies:

Date in Data endpoint
LatestCitationDate in the Stocks endpoint

  • Date format has already been changed to year only
  • SFP proposes to add additional metadata records to the DataFile model. Each time an editor uploads a new DataFile, they will have to specify at least one metadata record. The attributes of each record are:

The node within the profile tree for which a new report has led to a change in the DataFile
The type of the report: Stock, Assessment, Catch, Management, Erratum
A CSL for the corresponding reference - from which we can extract the year

  • Reference year for the relevant file (e.g. reporting year—that is the year of the stock assessment) will take longer for SFP to add to all records.
  • However the “reporting year or assessment year” would have one value for a set of time series
  • Notes that additional sections join the reporting year with it + the data source (e.g. Scientific advice, etc.)
  • Interim fix: pull latest date from data file and display that as the year of the reference report
  • Proposed solution- adding a field in our reference manager flagging a reference is an assessment report, this would affect new updates and profiles
  • Could also review assessment reports and upload year from Excel file upload though this will take time
  • For score year, easiest to add on fishery endpoint.
  • Note we may have different dates from different time series. Notes, that we have the year of corresponding data points captured in the data sheets.
  • Have an array of data points. For biomass- send over ratio. For GRSF definitions, year is the last year for the fishery status.

7) The reporting years (or citation_date) for fisheries are empty

Will add latest year from the data point.
Reference year + speed up population of reporting years.

Follow-up actions

  • SFP to check information in GRSF for FS and FIRMS (e.g. 23 records).
  • SFP to confirm process for pulling reference year and report year.
  • SFP to add field to species table indicating source of name.
  • SFP to confirm solution for reference and data point dates.
  • FORTH to reflect modifications in the GRSF knowledge base.

Resources

Add picture from clipboard (Maximum size: 8.91 MB)