Project

General

Profile

Actions

Bug #12906

closed

Update "problematic" records in the GRSF admin catalog

Added by Yannis Marketakis over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Target version:
Start date:
Nov 20, 2018
Due date:
% Done:

100%

Estimated time:

Description

Because of the related issues, there are several records that have problems. Once all these problems are resolved, we will update the records in the catalog.
For the time being it is evident that we should update the following:

  • FIRMS stock records
  • GRSF stock records

Files

StocksFromFirms.txt (4.53 KB) StocksFromFirms.txt Yannis Marketakis, Nov 22, 2018 12:29 PM
StocksFromFishSource.txt (19.1 KB) StocksFromFishSource.txt Yannis Marketakis, Nov 22, 2018 12:29 PM
StocksFromFirms.xlsx (12.6 KB) StocksFromFirms.xlsx Yannis Marketakis, Nov 22, 2018 02:23 PM
StocksFromFishSource.xlsx (21.1 KB) StocksFromFishSource.xlsx Yannis Marketakis, Nov 22, 2018 02:23 PM
StocksFromFirms_RevAG.xlsx (13.2 KB) StocksFromFirms_RevAG.xlsx firms assessment units list (subset) Aureliano Gentile, Nov 23, 2018 05:29 PM
StocksFromFirms_RevAG.xlsx (13.2 KB) StocksFromFirms_RevAG.xlsx Aureliano Gentile, Nov 26, 2018 12:52 PM

Related issues

Blocked by StocksAndFisheriesKB - Bug #12424: Stock status FAO categories not properly handledClosedYannis MarketakisSep 07, 2018

Actions
Blocked by StocksAndFisheriesKB - Bug #12419: Fishing Pressure time series is missing in several recordsClosedNikos MinadakisSep 07, 2018

Actions
Blocked by StocksAndFisheriesKB - Bug #12261: Multiple time series in GRSF from RAM dataClosedYannis MarketakisJul 27, 2018

Actions
Blocked by StocksAndFisheriesKB - Bug #12420: Narrative text is messed up when in multiple xml nodesClosedYannis MarketakisSep 07, 2018

Actions
Blocks StocksAndFisheriesKB - Task #12857: Bulk approval of (valid) GRSF recordsClosedYannis MarketakisNov 11, 2018Nov 30, 2018

Actions
Actions #1

Updated by Yannis Marketakis over 6 years ago

  • Blocked by Bug #12424: Stock status FAO categories not properly handled added
Actions #2

Updated by Yannis Marketakis over 6 years ago

  • Blocks Bug #12419: Fishing Pressure time series is missing in several records added
Actions #3

Updated by Yannis Marketakis over 6 years ago

  • Blocks deleted (Bug #12419: Fishing Pressure time series is missing in several records)
Actions #4

Updated by Yannis Marketakis over 6 years ago

  • Blocked by Bug #12419: Fishing Pressure time series is missing in several records added
Actions #5

Updated by Yannis Marketakis over 6 years ago

  • Blocked by Bug #12261: Multiple time series in GRSF from RAM data added
Actions #6

Updated by Yannis Marketakis over 6 years ago

  • Blocked by Bug #12420: Narrative text is messed up when in multiple xml nodes added
Actions #7

Updated by Aureliano Gentile over 6 years ago

It might be good to republish the entire data set to avoid any undetected issues.
In example today I was preparing a ppt and I used the GRSF while I noticed that for whatever reason this record has been flagged as marine resource instead of assessment unit and I do not know why http://data.d4science.org/ctlg/GRSF_Admin/ee1d7967-6da2-3d8a-88d5-c6adcf9576e6 (maybe because there is no stock status values? but there is some text for that and stockstatusentry...)

Actions #8

Updated by Yannis Marketakis over 6 years ago

Practically, if we update a record it will have the same effect with republishing it.
As regards your comment about http://data.d4science.org/ctlg/GRSF_Admin/ee1d7967-6da2-3d8a-88d5-c6adcf9576e6, we will investigate it and if necessary open a new ticket for that.

Actions #9

Updated by Yannis Marketakis over 6 years ago

@aureliano.gentile@fao.org As regards the record http://data.d4science.org/ctlg/GRSF_Admin/ee1d7967-6da2-3d8a-88d5-c6adcf9576e6 that appeared having type marine resources, I checked it and this happens because there are no time-series attached with this record.

As described in https://support.d4science.org/projects/stocksandfisherieskb/wiki/GRSF_database_overview a stock is an Assessment unit whenever a stock assessment is conducted. Let me know if this includes information about the state and trend of a record (as happens with this case), so that I will update the records accordingly.

Actions #10

Updated by Aureliano Gentile over 6 years ago

Thanks, I confirm the overall requirement https://support.d4science.org/projects/stocksandfisherieskb/wiki/GRSF_database_overview#GRFS-Information-domains :

  • Types for stock: 1) "Assessment unit", whenever a stock assessment is conducted; 2) "Marine Resource", when there is no stock assessment available.

And to that we need criteria for the implementation although it is not always straight forward to design such logic. There are cases where even if no assessment has been ever conducted there are other evidence that it is an assessment unit. Dentex dentex Northern Tunisia https://bluebridge.d4science.org/group/grsf_admin/data-catalogue?path=/dataset/ee1d7967-6da2-3d8a-88d5-c6adcf9576e6 is an example and it needs to be treated as an Assessment Unit. This is deducted from the status trend text also in absence of time series of stock status indicators:

In the North (GSA 12), the yield by recruit value is below the optimal level; the stock seems to be under-exploited. The exploitation profile in the eastern region (GSA 13) is in optimal conditions. [Data Owner: GFCM, DB Source: FIRMS] [Ref. Year: 2006]

So, maybe this is something which so far needs to be treated case by case. In the future we may have better rules or data.
In conclusion, thanks for the investigation, and the record type needs to be changed. Also, we can do it while approving and reviewing records.

Actions #11

Updated by Yannis Marketakis over 6 years ago

Thank you for the answer. So I will update the type of the record. Would you like to do it for all the records that face a similar situation (i.e. there are no time-series but they do have a status trend indicator) ?

Actions #12

Updated by Aureliano Gentile over 6 years ago

One clarification: if the record has status trend indicators then it is an Assessment Unit by default. These status indicators can be multiples across time hence the notion of time series. So, the presence of time series of abundance/mortality (Abundance Level (FIRMS Standard) ; Abundance Level ; Fishing Pressure (FIRMS Standard) ; Fishing Pressure) activate the Type "assessment unit".
The presence of the "State and Trend" could be added as criteria (to facilitate our life), then some of those can be reverted to marine resource if not correct. But please note that "State and Trend" is a narrative text, not an "indicator" per se. We had a discussion with CNR on this section which is not readable through the interface (text too long within the cell).

In conclusion, before apply the changes, would it be possible to understand how many are with no S&T indicators but do have "State and Trend" so I can review them and see if it make sense to change as assessment unit or not? With thanks

Actions #13

Updated by Yannis Marketakis over 6 years ago

Attached you will find the list of stocks records that have state and trend info and do not have time-series info.
There are two lists one for stocks from FIRMS and one for the ones from FishSource. There are no such records from RAM.

Actions #14

Updated by Aureliano Gentile over 6 years ago

Thanks, I'll review. Actually with the title next to the URL it would be much easier for me to spot which are assessment units. (But please ignore if too long or complex)
I.e. http://data.d4science.org/ctlg/GRSF_Admin/378a26e6-a871-3dcb-bdb1-3de95873d89b is Smalleye smoothhound - Coastal areas of Trinidad and Tobago and indeed it is an assessment unit.

Finally, I do not think the list should include the legacy records, but only the GRSF records.

Actions #15

Updated by Yannis Marketakis over 6 years ago

Sure. Attached you will find the corresponding titles

Actions #16

Updated by Yannis Marketakis over 6 years ago

  • Blocks Task #12857: Bulk approval of (valid) GRSF records added
Actions #17

Updated by Aureliano Gentile over 6 years ago

I updated the list of FIRMS, please find attached excel file.
The presence of Text in the "State and trend of Marine Resource" is unfortunately not enough, need to be read and evaluated. But with this review, can it be kept also for the next data harvests?

Regarding FishSource:

I checked the API from FishSource https://www.fishsource.org/apipie/v3/stocks/index.html and confirmed that there is Type (Indicates whether the root of the Fishery Profile Tree is a Stock, AssessmentUnit or Unascertained), where type "unascertained" corresponds to a "marine resource".
So, there should not need to check the FishSource records as this is clearly indicated.

Actions #18

Updated by Yannis Marketakis over 6 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

I would prefer an automatic way of identifying them (so that we are sure that future updates are consistent). I understand that this is not feasible because you have to read the narratives to identify them.

What about http://data.d4science.org/ctlg/GRSF_Admin/41cdffcb-fff2-3aaa-ab38-1d23a2b651c4 ? Its type is missing from the list.

Actions #19

Updated by Aureliano Gentile over 6 years ago

"Sharks - Global" (http://data.d4science.org/ctlg/GRSF_Admin/41cdffcb-fff2-3aaa-ab38-1d23a2b651c4) is marine resource. While the above one "Common cuttlefish - Côte d'Ivoire" is assessment unit. Apologies for the oversight, I upload the excel file again.

In terms of criteria, when no assessment is produced and there is a "State and Trend", we could assign an assessment unit if the area is a for specific location (e.g. one or multiple EEZs, specific fao area codes..), it is a marine resource if it is at regional or global level. In fact see the Marine resources I have flagged: Marine resources Southern Ocean, Tuna and tuna-like species - Global, Sharks - Global. But this may not work in some other cases so, I suspect it would be better for manual revision. And if you can keep this knowledge for the next data harvest.

Actions #20

Updated by Yannis Marketakis over 6 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 10 to 100

The following records have been updated:

  • 701 legacy Stock records (from FIRMS)
  • 2928 GRSF Stock records
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)