Final release of the Kraken marketplace - Review of components and technologies
The KRAKEN Marketplace will act as a decentralized exchange for data, securely connecting providers and consumers of high-quality datasets across three different data sharing modalities (Secure Multiparty Computation (SMPC), direct download of encrypted batch datasets and data streams transfer over the decentralized Streamr Network).
With the Marketplace approaching the release of its final implementation, the timing feels right to provide an overview of the main developments and achievements over the course of the two and a half years since the KRAKEN project’s inception.
Three-layered Marketplace structure
We’ll start with an overview of the three-layered marketplace architecture. The core of the KRAKEN marketplace system has built upon, enhanced and integrated existing technologies developed in two projects; Streamr and MyHealthMyData (MHMD). It is also integrating with consortium partners’ technologies in the Self Sovereign Identity (SSI) and crypto pillars of the KRAKEN project.
The guiding principle of the design, and indeed of the project itself, has been to implement decentralisation throughout the overall KRAKEN platform. At the same time we want to provide the highest level of privacy protection for marketplace users and the data they exchange, and we want to achieve compliance with national and European privacy laws, especially the GDPR.
The result is a marketplace architecture consisting of three main functional areas:
- A permissioning layer, mostly implemented by the Lynkeus HyperLedger Fabric blockchain in conjunction with the SSI system for the identification, authentication and credentialing of both individual and organisational users.
- A data protection layer, which implements a variety of data security and privacy preserving modules including the Secure Multi Party Computation (SMPC) system used for both distributed data analytics and encryption keys sharing in the encrypted batch dataset exchange modality.
- A transaction management layer, mostly implemented through the Streamr marketplace technology, which provides both user-facing and back-end functionalities, such as UIs, payments execution and control, secure and decentralized transfers of end-to-end encrypted streaming data, and Data Product visualisation.
A desktop frontend and two mobile apps
The frontend user facing interfaces of the KRAKEN marketplace can be accessed either through the online desktop platform or a mobile app. A decision was made to implement a mobile app for the management of Data Products that was separate to the SSI mobile app. This causes some minor tradeoffs in the usability of the KRAKEN platform but is necessary in order to better guarantee the security of the user authentication process, leaving the SSI app as a standalone general-purpose identity management module.
The desktop version of the marketplace app allows individual and institutional users dealing with datasets on behalf of their employers, to publish, purchase, consume and manage Data Products that are shared using the three different data exchange modalities. The KRAKEN marketplace mobile app is mostly dedicated to individual users who will be able to use it to update permissions and availability of their own Data Products and see how well they are performing on the market.
Highly innovative development
One of the main accomplishments in the marketplace system achieved within the KRAKEN project is the integration of the SMPC network with the marketplace frontend and backend. To our knowledge it is the first real-world implementation that brings together data access permissioning, privacy preserving distributed computations, and token-based payment systems.
The integration allows encrypted datasets to be made available for distributed evaluation of basic statistical functions (analytics), without exposing the data itself to the nodes of the SMPC network or the data buyer. The privacy of the data producers and data sellers involved in the computation is therefore preserved in these types of marketplace transactions, whilst data sellers retain control over who can access their Data Products to conduct analytical computations, thanks to the integration with the marketplace permissioning layer.
A pay-per-computation system powered by the Streamr DATA token has also been developed, which includes the possibility of combining multiple Data Products with matching data variables in a single computation, whilst distributing payments to the individual Data Product owners.
Other key developments - institutional user management tools and Data Unions
There are two other key developments worthy of mentioning in this blog post. The first is the development and currently progressing integration of institutional users management tools. Thanks to this development a natural person logged in to the KRAKEN marketplace can operate directly for themself or on behalf of a company.
Using the institutional management tools a natural person that wants to act on behalf of a company or institution, such as a hospital, can be duly authorised by a legal representative of the company to act on their behalf through the assignment of an Attorney Verifiable Credential (VC). On registering with the marketplace the user may present this Attorney VC as evidence of their institutional affiliation and proof of their permission to operate in the marketplace on the company’s behalf. Two software tools have been developed in the project that support the generation of Attorney VCs and the verification of their veracity and status by the marketplace; the Depute Tool and the KRAKEN Company Identification Tool (KCIT).
The second, is an integration with the Streamr Data Unions Framework and Streamr Network. The KRAKEN team is in the process of producing a pilot Data Union demonstrator for the health pilot. In this Data Union demonstrator, individuals can opt-in to join other individual data subjects and offer data from locally stored health apps in an aggregated Data Product on the KRAKEN marketplace.
In the marketplace mobile app, individual users will be provided with a simple way to opt-in to join the Data Union and publish their mobile health data to a Data Product in the KRAKEN marketplace. Eligible buyers would be able to purchase access to the Data Union using the Streamr DATA token, and revenues generated in cryptocurrency would be shared between all members that have opted into the Data Union.
Final user testing and evaluation in pilots due in September 2022
The final KRAKEN Marketplace architecture established over the course of the KRAKEN project offers different levels of data and identity protection that users can pick from while enforcing all key tenets of the GDPR. It also supports an efficient user-friendly e-commerce experience focusing on data discoverability, ease of access and control.
The implementation of an innovative infrastructure integrating blockchain-based permissioning, token-based payments and distributed computations with SMPC, for the first time to our knowledge, realises synergistic dynamics amongst privacy constraints, data value and information sharing.
We now look forward to subsequently testing and evaluating this architecture with users in the education and health pilots due to take place in September 2022