D1.2 Key Performance Indicators report [M6]¶
Authors: Eleni Koulocheri
Editor: Natalia Manola
Last Update: 06/03/2019
The Key Performance Indicators report (M6-ongoing, R-wiki) is on a wiki page with up to date with the KPIs collected during the project life-time activities.
Disclaimer¶
This document contains description of the OpenAIRE-Connect project findings, work and products. Certain parts of it might be under partner Intellectual Property Right (IPR) rules so, prior to using its content please contact the consortium head for approval.
In case you believe that this document harms in any way IPR held by you as a person or as a repre-sentative of an entity, please do notify us immediately.
The authors of this document have taken any available measure in order for its content to be ac-curate, consistent and lawful. However, neither the project consortium as a whole nor the individual partners that implicitly or explicitly participated in the creation and publication of this docu-ment hold any sort of responsibility that might occur as a result of using its content.
This publication has been produced with the assistance of the European Union. The content of this publication is the sole responsibility of OpenAIRE-Connect consortium and can in no way be taken to reflect the views of the European Union.
Key Performance Indicators (KPIs)¶
To follow the progress of the project activities on a regular basis and better understand potential difficulties, delays and/or deviations to the original plan, the OpenAIRE-Connect partners have tried to quantify, and thereby measure, a number of quality parameters related to the project. These quality parameters, key performance indicators from now on, have been classified into three groups:
- Scientific Activities (SA)
- Management Activities (MA)
- Dissemination Activities (DA) The Scientific Activities (SA) Key Performance Indicators are also listed and described in depth in Part A of Grant Agreement.
All three groups of Key Performance Indicators are listed, along with the success description and success target for each of them. The success description and success target have been authored by the same working team that created the list of KPI s of Part A.
Measurable objectives require Key Performance Indicators (KPIs) to ensure project management effectively and efficiently monitor the project evolution and progress towards such objectives. OpenAIRE-Connect has identified a number of KPIs related to each objective to ensure the highest impact, as well as the quality and success of the expected outputs.
KPIs represent an essential toolkit for monitoring and reviewing the overall progress of the project, and of the individual research, development and demonstration (RDD) activities performed in frame of its implementation. To effectively manage the monitoring of the KPIs, specific issues have been created in the Issue Tracking System to trace the evolution of the KPIs during the project’s lifetime. These issues have been assigned to selected WP Leaders that are in charge of the regular update of those issues (by modifying the value of “achievement” in suitable time).
The following table briefly presents the main indicators for the quality assurance in the OpenAIRE-Connect project. The Scientific Activities (SA) Key Performance Indicators are also listed and described in depth in Part A of Grant Agreement
Procedure for Collecting Data¶
All KPIs and been transferred in a google doc in order to be measured collaboratively and each timestamp reflects the numbers from the beginning of the project.
All Workpackage Leaders were asked to fill the numbers to those KPIs that are involved in by end of M6, M12, M18, M24, M26, M30 (estimations) by noting their WP. If a KPI measurement occured from more than one WP, then the total number was calculated cumulatively. Numbers in each time stamp, reflect the values from the beginning of the project.
Key Performance Indicators¶
Below there is the list with all three groups of Key Performance Indicators, along with the success description and success target for each of them. The success description and success target have been authored by the same working team that created the list of KPI s of Part A.
Scientific Activities (SA) Indicators -- Success Measures -- Success Targets¶
SAI1 - Number of artefacts ingested into OpenAIRE by researchers of research communities in the proposal:
An overtime increase of methods and packages collected ingested or deposited by researchers in OpenAIRE -- Improve of 25% the claim ratio of OpenAIRE
SAI2 - Number of artefacts ingested into OpenAIRE via content providers in the proposal:
An overtime increase of methods and packages collected from content providers in OpenAIRE -- At least 10,000 datasets, methods, and packages interlinked with other objects in OpenAIRE (at least 1,000 per community)
SAI3 - Number of researchers registering to the dashboard for research communities in the proposal:
An overtime increase of enduser registrations from the communities -- End of the project: 10% of researchers contacted by community is registered to OpenAIRE
SAI4 - Number of researchers actively using the dashboard from research communities in the proposal (at least one deposition or one search):
An overtime increase of usage actions (monitoring tools WP7)
SAI5 - Number of publications/methods/data etc. touching upon different research communities:
Identifying multidisciplinary artefacts, shared across (claimed by) different communities -- Hard to estimate
SAI6 - Number of institutional repositories subscribed to Catch-All Broker Service beyond partners:
An overtime increase of repositories registering to the service -- 150 (around 20% of today’s OpenAIRE registered repositories)
SAI7 - Number of data archives subscribed to Catch-All Broker Service beyond partners:
An overtime increase of data archives registering to the service -- 25
SAI8 - Number of publishers subscribed to Catch-All Broker Service beyond partners:
An overtime increase of publishers registering to the service -- 10
SAI9 - Number of Research Communities requiring a Dashboard “as a Service” beyond partners
-- 2
SAI10 - Technical uptime ratio (production deployment)
-- 99,98% (OpenAIRE infrastructure ratio 2015)
SAI11 - Score of the service assessments at the end of the project
-- 8 out of 10 points (or equivalent percentage for different evaluation schemes)
SAI12 - Number of researchers contacted in F2F meetings (workshops)
All stakeholder groups in work plan are invited and engaged outreach workshops -- 30-40 each
SAI13 - Number of repository and data archives managers contacted in F2F meetings (workshops)
All stakeholder groups in work plan are invited and engaged outreach workshops -- 20-30 each
SAI14 - Number of participants in webinars for content providers
All stakeholder groups in work plan are invited and engaged at these events --30-40 each
SAI15 - Number of participants in webinars contacting communities
All stakeholder groups in work plan are invited and engaged at these events -- 40-50 each
SAI16 - Number of participants subscribing the learning courses
All stakeholder groups in work plan are invited and engaged at these events -- 15-30 each
Management Activities (MA) Indicators -- Success Measures -- Success Targets¶
MAI1 - Number of Project physical meetings
All related partners are invited and engaged for F2F collaboration --4-5 in the project life
MAI2 - Number of Project remote meetings
All related partners are invited and engaged so as to discuss and solve open issues -- 15 in the project life
MAI3 - Number of researchers registering to the dashboard for research communities in the proposal
An overtime increase of enduser registrations from the communities -- End of the project: 10% of researchers contacted by community is registered to OpenAIRE
MAI4 - Number of successfully completed Tasks
Following the Description of Work -- 30
MAI5 - Number of Risks/Deviations tackled
As less as possible number of risks/deviations -- <10
MAI6 - Number of delayed Deliverables
As low as possible number of delayed deliverables -- <5 projects with a delay <1 month
MAI7 - Number of data archives subscribed to Catch-All Broker Service Number of revisions of the project Deliverables
Low number of revisions prove in a deliverable leads to high quality. -- Max 2 revisions in 20% of deliverables
Dissemination Activities (DA) Indicators -- Success Measures -- Success Targets¶
DAI1 - Number of external participants at OpenAIRE-Connect events
Project impact beyond consortium -- End of project: 500
DAI2 - Number of projects and organisations involved in OpenAIRE-Connect activities
Possibilities potential synergies and collaborations with external bodies/projects -- End of project: 5 projects, 8 organisations
DAI3 - Promotional and dissemination material produced
--End of project: 3 press releases, 5 leaflets, 5 fact sheets
DAI4 - Impact of project’s website (number of hits, unique visitors, length of stay, most popular pages, peaks, etc. through web statistic module) website (number of hits, unique visitors, length of stay, most popular pages, peaks, etc. through web statistic module)
-- End of project: 1000 hits
DAI5 - Twitter engagement metrics (number of followers, number of tweets, number of retweets, number of @connects: both @replies and @mentions, URL mentions, brand mentions, click through rates, etc.)
-- 8000 followers, 100 tweets
DAI6 - Facebook engagement metrics (number of followers/fans/friends, number of posts, number of friends’ posts, number of comments on posts, number of “likes” on posts, number of “shares”, etc.)
-- End of project: 150 friends, 20 posts, 50 likes, 30 comments on posts, 10 shares.
DAI7 - LinkedIn engagement metrics (number of connections and groups)
-- End of project: 80 connections and groups
DAI8 - Number of social media campaigns
Social media campaigns that promote a key project event. -- 4
Results as reflected in M6, M12, M18, M24, M26, M30 (estimation)¶
| KPI ID | M6 | M12 | M18 | M24 | M26 | M30 (estimation) |
|---|---|---|---|---|---|---|
| SAI1 | 0 | 0 | 28 | 43 | 98 | |
| SAI2 | 0 | 15,734 | N/A | 741,745 | 24,618 | |
| SAI3 | 0 | 0 | N/A Estimation: 20. | N/A | 72 | |
| SAI4 | 0 | 0 | 4 users made at least one claim | 6 users made at least one claim | 12 | |
| SAI5 | 0 | 0 | 1 | 12 | 14 | |
| SAI6 | 0 | 0 | N/A | N/A | 158 | |
| SAI7 | 0 | 0 | 0 | 0 | 0 | |
| SAI8 | 0 | 0 | 0 | 0 | 0 | |
| SAI9 | 0 | 2 | N/A | 5. Requests from RIs: RDA, CLARIN, Instruct-ERIC, DARIAH-EU Requests from RCs: AgInfra | 7 | |
| SAI10 | 99.99% | 99.92% | 99.95% | 99.93% | 99.94% | |
| SAI11 | 0 | 0 | 0 | 0 | 0 | |
| SAI12 | 0 | 50 | 200 | 600 | 1020 | |
| SAI13 | 0 | 50 | 180 | 350 | 545 | |
| SAI14 | 0 | 0 | 108 | 261 | 414 | |
| SAI15 | 0 | 0 | 0 | 30 | 60 | |
| SAI16 | 0 | 0 | 0 | 0 | 0 | |
| MAI1 | 1 | 1 | 3 | 3 | 3 | 5 |
| MAI2 | 4 | 8 | 15 | 19 | 19 | |
| MAI3 | 0 | 0 | N/A. Estim.: 20 | N/A | 72 | |
| MAI4 | 0 | 1 | 2 | 2 | 2 | |
| MAI5 | 1 | 1 | 1 | 1 | 1 | |
| MAI6 | 6 | 4 | 5 | 0 | 0 | 0 |
| MAI7 | 0 | 0 | 8 | 8 | 8 | |
| DAI1 | 0 | 90 | 360 | 710 | 750 | |
| DAI2 | 1 | 3 | 6 | 11 | 15 | |
| DAI3 | 3 | 3 | 5 leaflets | 5 leaflets | 6 leaflets, 2 factsheets | |
| DAI4 | 6106 hits | 7048 hits | N/A | N/A | 16718 hits in https://www.openaire.eu/connect | |
| DAI5 | 7023 followers, 20 tweets | 8030 followers,30 tweets | N/A | N/A | 10.5K followers, 80 tweets | |
| DAI6 | 5 | 1400 friends (from OpenAIRE Facebook), 3 posts, 4 shares, 20 likes | 1500 friends (from OpenAIRE Facebook), 5 posts, 6 shares, 50 likes | 1719 friends (from OpenAIRE Facebook), 9 posts, 8 shares, 105 likes | ||
| DAI7 | 1 | 1 | N/A | N/A | 72 connections | |
| DAI8 | 1 | 2 | 4 | 9 | 11 |
Future Plans¶
The next version of D1.2 will cover the entire project life, 30 months along with success evaluation rating.
