The piloting and evaluation of the KRAKEN final release has been conducted in September 2022 by involving a sample of 35 participants who were asked to assess the usability and subjective quality of the platform, as well as to expressed their main preferences and concerns regarding KRAKEN future adoption.
In this second evaluation round we involved 30 persons representing key target user groups in the Healthcare domain (e.g., digital health researchers, blockchain and big data experts, project managers, legal experts) and 5 students at the Technical University of Graz assessing the KRAKEN solution for the Education domain.
Some main highlights from the study are the following:
- 90% of users in the health pilot showed interest in using KRAKEN for providing and/or consuming data products.
- 83% of users in the health pilot and 80% of users in the education pilot appreciated the possibility of sharing analytics data products through the platform.
- 70% of users in the health pilot would be motivated to share data by monetary compensation including cryptocurrencies.
- Most participants in both the pilots positively judged the information on data privacy and protection available in the KRAKEN platform.
About the methodology, the study protocol involved two evaluation phases:
- First, participants were invited to individually use the KRAKEN final release and perform relevant tasks with it (e.g., access the KRAKEN marketplace by using the SSI mobile app, registration of a new account, browsing of data products available, creation and publication of a data product, including analytics data products, etc.) to rate usability of the digital solution.
- The following week, participants were invited to individual interviews to answer some key questions regarding their preferences, and possible concerns regarding their adoption of data sharing platforms for sharing personal data as data providers or consumers.
Overall, we found a marginal acceptance of the prototype release in terms of usability, as measured by means of the SUS (System Usability Scale): the average score (101 points scale) was 52.95 for the health domain and 50 for the education domain. When considering only participants with some knowledge or good expertise with data sharing systems, the SUS average score is 60.33 for the health pilot and 60.83 for the education pilot. This result would support the deployment of the KRAKEN final release and marketplace in a B2B (Business2Business) scenario, whilst for deployment in a B2C (Business2Consumer) scenario a step-by-step tutorial supporting the first interaction session is strongly recommended.
Results from the interviews with participants showed a good level of trust and interest of users regarding future adoption of the KRAKEN platform for sharing data, strong interest for using its privacy preserving analytics component, as well as for being empowered and supported by the platform’s features in keeping control over the personal data shared (e.g., by deciding which types of parties can access the data, for how long, etc.).
More details on the evaluation study results are presented in D5.8 KRAKEN marketplace testing and final validation report, to be submitted by October 2022.
It will also be disseminated by means of a scientific publication in the coming months.