Support #511
closed
Support #510: Data Transfer on the THREDDS instance
Publishing raster data via StatMan
Added by Gianpaolo Coro over 9 years ago.
Updated over 9 years ago.
Category:
High-Throughput-Computing
Infrastructure:
Development
Description
A process to publish raster data via StatMan is required. This will publish maps on GeoExplorer via GeoNetwork and Thredds along with metadata.
Files
|
DivaInterface.png (226 KB)
DivaInterface.png |
Diva web service interface |
Gianpaolo Coro, Sep 07, 2015 03:56 PM
|
|
|
ExperimentConfiguration.png (129 KB)
ExperimentConfiguration.png |
StatMan experiment configuration |
Gianpaolo Coro, Sep 07, 2015 03:56 PM
|
|
|
Result.png (86.2 KB)
Result.png |
Experiment result |
Gianpaolo Coro, Sep 07, 2015 03:57 PM
|
|
|
DIVAtempGeoExplorer.png (366 KB)
DIVAtempGeoExplorer.png |
Visualisation on GeoExplorer |
Gianpaolo Coro, Sep 07, 2015 03:57 PM
|
|
|
StatManUpload.png (95.2 KB)
StatManUpload.png |
Upload tools of StatMan |
Gianpaolo Coro, Sep 07, 2015 04:10 PM
|
|
- % Done changed from 0 to 20
A publication process core for GIS raster data has been implemented, but still requires data transfer installation to be fixed.
Data transfer, in fact, seems to not support direct transfer from a local file. In particular, providing a local file URI from the client causes the service to search for the file on its file system. On the other hand, providing a http link works if this link is "simple": it works with dropbox links but does not work with URI resolver generated links (the service requires a "certificate"). Finally, directly providing smp links does not work because the service states that this protocol is not supported.
I have used the GHN on the following machine for these tests: dewn04.madgik.di.uoa.gr
I think a new installation should be done on the Thredds service, with updated libraries, also enabling the smp protocol.
Gianpaolo Coro wrote:
A publication process core for GIS raster data has been implemented, but still requires data transfer installation to be fixed.
Data transfer, in fact, seems to not support direct transfer from a local file. In particular, providing a local file URI from the client causes the service to search for the file on its file system. On the other hand, providing a http link works if this link is "simple": it works with dropbox links but does not work with URI resolver generated links (the service requires a "certificate").
It should work on production where all servers acting as clients have their own certificate. Is this wrong? I suggest to add Roberto and Andrea as watchers.
Finally, directly providing smp links does not work because the service states that this protocol is not supported.
I have used the GHN on the following machine for these tests: dewn04.madgik.di.uoa.gr
I think a new installation should be done on the Thredds service, with updated libraries, also enabling the smp protocol.
After investigating with Roberto, we found the issue was maily due to a transformation of smp URIs made by the class:
org.apache.commons.vfs2.provider.SmpFileObject in vfs-provider-smp-1.1.0-3.8.0.jar
This class, in fact, corrupts current smp links because it was suited for the old format. I will modify it and release a new version.
This issue is now blocked by ticket #549
- % Done changed from 20 to 40
The blocking issue has been solved, now it is possible to upload files on thredds and produce layers through GeoNetwork. I'm going to build a new algorithm on top of these features.
An algorithm has been implemented that allows publishing raster data on the geospatial network of D4Science.
The algorithm publishes raster data either as WMS/WCS maps (in the case of NetCDf-CF files) or as generic datasets (with attached metadata).
The user is asked to indicate a file, the name of the layer, an abstract and other parameters.
An example is reported by means of the workflow described by the attached images:
- a uniform temperature dataset is manually generated via the DIVA interpolation web service of SeaDataNet (http://gher-diva.phys.ulg.ac.be/web-vis/diva.html#),
- the file is uploaded on StatMan,
- the experiment is configured and executed,
- the output reports the name of the generated layer and other information,
- the layer is finally visualised on GeoExplorer.
This algorithm is currently under testing and will be officially released in October.
In order to build a more complete suite for managing raster files, also the DIVA web service is going to be integrated with StatMan.
- Status changed from New to Closed
- % Done changed from 80 to 100
Also available in: Atom
PDF
Updated by 
Updated by 
