Project

General

Profile

Actions

Task #11786

closed

GRSF-services-core updates

Added by Yannis Marketakis almost 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Target version:
-
Start date:
May 17, 2018
Due date:
% Done:

100%

Estimated time:

Description

The following updates are required in the grsf-services-core component:

  • apply the new types for Stocks: (Assessment Unit and Marine Resource) and Fisheries: (Fishing Unit and Other Fishery) ( DONE )

  • create a new field called similarities_indicator that will contain the values with similar records / without similar records. ( DONE )

  • create a new field called connections_indicator that will contain the values connected / not connected

  • Store the creation date (at least the year) of a legacy record and the series name (for the moment only for FIRMS records)

  • Construct the citation string and submit it while publishing a record

Actions #1

Updated by Yannis Marketakis almost 7 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Yannis Marketakis almost 7 years ago

  • % Done changed from 0 to 80
  • create a new field called connections_indicator that will contain the values connected / not connected --> DONE

  • Store the creation date (at least the year) of a legacy record and the series name (for the moment only for FIRMS records) --> DONE

  • Construct the citation string and submit it while publishing a record --> DONE

Actions #3

Updated by Yannis Marketakis almost 7 years ago

The aforementioned updates have been carried out. There are still the following pending issues, which I would like to check with CNR colleagues.

  • We tried to improve the readability of species and water areas fields, by adding commas or dashes. These fields appear as tags and FAO colleagues mentioned that it is important to be readable. However, I noticed that for these cases we submit a string that contains the required separators which are removed when creating the tag. (i.e. check the species or the assessment area fields at https://ckan-grsf-admin2.d4science.org/dataset/397db3fe-e27d-3d63-86c1-2eda3658db12 and the corresponding tags). Could these extra characters (i.e. commas and colons) be preserved in the tag?

  • The citation string of a record is now submitted during publishing in the field named citation. The information that will be added from the catalog is the access date. When publishing it has the value [Date accessed and/or downloaded]. Is this OK? Do you want to change it to something else (so that you can find it and replace it with the actual access date)? Below you will find the citation of an indicative FIRMS record.

© FAO 2017. Striped marlin - Eastern Pacific. FIRMS Data Collection.
In: Fisheries and Resources Monitoring System (FIRMS) [online].
Updated: 2017-04-03. Accessed: [Date accessed and/or downloaded].
http://firms.fao.org/firms/resource/10079/en

@luca.frosini@isti.cnr.it could you take a look to the above questions ?

Actions #4

Updated by Francesco Mangiacrapa almost 7 years ago

Hi Yannis,

I can reply to..

Yannis Marketakis wrote:

  • We tried to improve the readability of species and water areas fields, by adding commas or dashes. These fields appear as tags and FAO colleagues mentioned that it is important to be readable. However, I noticed that for these cases we submit a string that contains the required separators which are removed when creating the tag. (i.e. check the species or the assessment area fields at https://ckan-grsf-admin2.d4science.org/dataset/397db3fe-e27d-3d63-86c1-2eda3658db12 and the corresponding tags). Could these extra characters (i.e. commas and colons) be preserved in the tag?

Unfortunately, we can't preserve them. We have an engine "limitation" (that is CKAN-side). As you can see at: http://docs.ckan.org/en/2.6/api/index.html#ckan.logic.action.create.tag_create

    name (string) – the name for the new tag, a string between 2 and 100 characters long containing only alphanumeric characters and -, _ and ., e.g. 'Jazz'

It works like a (Persistent Uniform Resource Locator) PURL that points to an intermediate resolution service (the Catalogue-Resolver). Instead of pointing directly to the location of the item (so the CKAN URL) We are using the resolution servicw (that is VRE-dependent because each item belongs to a VRE (where it was created)). The resolution service returns the URL to the client as a standard HTTP/s redirect.

Actions #6

Updated by Luca Frosini almost 7 years ago

Hi @marketak@ics.forth.gr

it is ok from my side. I already added the support for 'citation' field in the code.

  • The citation string of a record is now submitted during publishing in the field named citation. The information that will be added from the catalog is the access date. When publishing it has the value [Date accessed and/or downloaded]. Is this OK? Do you want to change it to something else (so that you can find it and replace it with the actual access date)? Below you will find the citation of an indicative FIRMS record.

[...]

@luca.frosini@isti.cnr.it could you take a look to the above questions ?

@aureliano.gentile@fao.org is it ok the text composed by @marketak@ics.forth.gr

@francesco.mangiacrapa@isti.cnr.it will take care of adding the [Date accessed and/or downloaded] on client side.

Actions #7

Updated by Aureliano Gentile almost 7 years ago

the above example of SOURCE RECORD CITATION is fine with me.

Actions #8

Updated by Yannis Marketakis almost 7 years ago

Thank you all for your answers.
The only pending issue, therefore, is the identification of the catalogue prefix.
When publishing a record, the publishing client is aware of the scope (the VRE), therefore I assume that I'll be able to retrieve the catalog prefix.

@francesco.mangiacrapa@isti.cnr.it and @luca.frosini@isti.cnr.it I assume that I'll be able to retrieve it using an ic-client, right ?

Actions #9

Updated by Francesco Mangiacrapa almost 7 years ago

Yannis Marketakis wrote:

@francesco.mangiacrapa@isti.cnr.it and @luca.frosini@isti.cnr.it I assume that I'll be able to retrieve it using an ic-client, right ?

Yes, Yannis.

In order to build right prefix, you can retrieve the RuntimeResource "HTTP-URI-Resolver" to get the hostname of the URI-Resolver.

Using the scope of ROOT (/d4science.research-infrastructures.eu) or VRE (i.e /d4science.research-infrastructures.eu/FARM/GRSF_Admin), it is unique in the infrastructure:

<Resource version="0.4.x">

   <ID>9fe02de0-bc9e-11e2-85ae-b807e8ac0391</ID>

   <Type>RuntimeResource</Type>

   <Scopes>

     <Scope>/d4science.research-infrastructures.eu/FARM/GRSF_Admin</Scope>

      <Scope>/d4science.research-infrastructures.eu</Scope>

....

      <Category>Service</Category>

      <Name>HTTP-URI-Resolver</Name>

....
     <HostedOn>data.d4science.org</HostedOn>


in the resource you can read the property "HostedOn" that is "data.d4science.org". Then you must add the suffix "/ctlg"+"/VRE_NAME".

Recap, you can build the right prefix in this way:

"https://data.d4science.org" + "/ctlg" + "/VRE_NAME"

i.e.
This is a prefix for the GRSF_ADMIN catalogue: http://data.d4science.org/ctlg/GRF_ADMIN
This is a PURL for the BlueBridgeProject catalogue: http://data.d4science.org/ctlg/BlueBridgeProject/fao-aquatic-species-distribution-map-of-bathyraja-spinicauda

and so on..

These links are generated first via HTTP Post.
You can see how at: https://wiki.gcube-system.org/gcube/URI_Resolver#CATALOGUE_Resolver. Unfortunately the wiki page is old and does not contain valid links :-( I will update the page asap.

I hope this clarifies

Actions #10

Updated by Yannis Marketakis almost 7 years ago

  • Status changed from In Progress to Closed

Perfect Thanks

Actions #11

Updated by Yannis Marketakis almost 7 years ago

  • % Done changed from 80 to 100
Actions #12

Updated by Luca Frosini almost 7 years ago

  • Project changed from 8 to StocksAndFisheriesKB
  • Category deleted (Default)
  • Target version deleted (Unsprintable )
Actions

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 8.91 MB)