D4Science Infrastructure

What do you mean by you need access to the following endpoint?

If I understand correct, each endpoint is valid for a certain scope (or set of scopes). Our portlet shall operate in the preproduction scope and we need access to that endpoint.
I didn't know the name of the preproduction scope so I could not check in the Infrastructure Monitor myself.

Dear Denis, differently from the development environment, in the production environment DataMiner is a cluster of machines with a proxy on top.
Thus, we can provide you with the main endpoint of the cluster, i.e. the proxy, which will act as it was one machine from your point of view. In other words, you will not have to change your code to interact with it.

The cluster proxy depends on the VRE you are using. Our services discover the proxy address dynamically, by querying the Information System and asking for a Service Endpoint named DataMiner in the DataAnalysis category. You can consult the catalogue of the Service Enpoints in the Information System monitoring here (after selecting the scope-VRE you are interested in):

https://services.d4science.org/infrastructure-monitor

By browsing your scope (e.g. /d4science.research-infrastructures.eu/gCubeApps/ScalableDataMining) you can see the address of the proxy. Alternatively, you can write code to discover the address dynamically from the Information System, following this guide:

https://wiki.gcube-system.org/gcube/Ic-client

Remind that, once you identify the scope, the token you use should be the one of the VRE.

So Gianpaolo, the DataMiner is accessible to newly created scopes-VREs automatically (out of the box)? Or one should ask for DataMiner specifically, as part of the VRE creation ticket?

During the creation phase of a VRE (or after), Dataminer should be explicitly requested along with algorithms. Depending on the computational resources you require, we select a cluster to assign to your VRE.

Not all the algorithms hosted by Dataminer can be assigned to a VRE, because of policy constraints. Basically, VREs are our main mechanism to manage access to the resources (databases, processes, data etc.) and their policies.

Currently, we have a prototyping environment (RPrototypeLab) that allows using processes before these are moved to their community-specific VREs. Algorithms can be removed or added at any time there. This should allow you to test also your portlet. The address of the Dataminer cluster proxy is dataminer-prototypes.d4science.org. Obviously, you require an RPrototypeLab token, thus you should subscribe there.

So far we had

1. the dev DataMiner as found at https://next.d4science.org/infrastructure-monitor# and is valid for /gcube/devsec/devVRE (among other scopes)

2. the actual clustered DataMiner as found at https://services.d4science.org/infrastructure-monitor which is valid for any VRE requesting it

Where the RPrototypeLab fits? Preproduction? It will replace the (1) above?

As @gianpaolo.coro@isti.cnr.it pointed, we will ask for permission on a per VRE basis. Nevertheless I am very interested on the clarification I asked :)

I am leaving the closing to the assignee as I am not sure regarding the final status.

Hi Denis, yes, the RPrototypeLab substitutes the dev. environment for you and acts like a pre-production environment for algorithms developers.

The dev. environment is now really used only by infrastructure services developers. Note that the RPrototypeLab can be accessed also by your colleagues who may want to see your prototypes.

This should increase the availability of the e-Infrastructure resources in all the developments sectors.

Please bear with me and let me get it straight. In a brand new portlet we are developing and in a servlet we are developing, we are quering IS for DataMiner. This endpoint gives us the dataminer ip address, which we use etc.
Now we should change this approach? If so, could you please pass me documentation links?