SIP-19: Streamr Bounties and Temporary Storage Improvements Proposal

Context

The launch of the Log Store Network marks the inception of a visionary endeavour aimed at enabling decentralised custom and secure inter-system data protocols. These protocols are fundamentally powered by the Streamr Network and designed to decentralise the collection of bespoke data for immutable storage, and subsequent utility of data in destination systems that encompass blockchains. The ultimate goal is to transform signals and event data into tangible on-chain outcomes.

Usher Labs has recognised the potential for such data protocols to scale Streamr Temporary Storage. It's important to note that Permanent and Temporary Storage of time-series data serve distinct purposes. Permanent Storage establishes a complete historical record of events for use within Smart Contracts and other on-chain applications. On the other hand, Temporary Storage is essential for shared analytics, enabling networks and other public good software to aggregate their custom logs and analytical data into a shared database.

In light of these insights, Usher Labs proposes a research project aimed at developing a Reporter Network to power a data protocol with the capacity to integrate trustless and on-chain usage-based billing into Streamr's Temporary Storage technology. Community Operators will have the flexibility to participate in operating Streamr Storage Nodes, thereby earning rewards for providing storage capabilities. They can also choose to operate Reporter Nodes, responsible for reporting on the performance of Storage Nodes. Furthermore, the data protocol enables $DATA to be used as a currency for billing storage, and rewarding operators participating as Reporters or Storage Providers.

Solution

Storage:

Participating as a storage provider will involve a familiar experience for operators, as it aligns with the existing Streamr Broker Node with the Storage Plugin enabled. This means that the current Streamr Light Node and SDKs remain instantly compatible with third-party Storage Providers. To engage, Stream Owners simply add a third-party storage node to their Stream through the Streamr SDK. They can then stake digital assets into a new Smart Contract, specifically designed to manage storage balances and their reallocation to network operators. The integration requires a simple Broker Node Storage Filter Plugin to ensure that only appropriately compensated data is stored. Usher Labs will develop this plugin.

Reporting:

Usher Labs will create a dedicated node capable of subscribing to Streams similarly to the Storage Node. When an event occurs, the Reporter Node initiates a "resend" request to the Storage Nodes responsible for storing the data. This process behaves as verification that the event was stored. Proof of this event's "write to storage" is then forwarded to the Log Store. Periodically, "resend" requests are executed to verify the availability and duration of data storage specified by the Stream owner. Coupled with a Smart Contract to consume this pure reporting data from the Log Store, this process enables trustless and verifiable metered billing of data writes to storage and storage usage.

Every Reporter Node operates as a data relayer and receives a Bounty Reward for contributing relevant data streams to the Log Store for use within a Smart Contract. Each Reporter Node has its own Streamr stream featuring a specific identifier or metadata, allowing the Oracle function to identify the appropriate streams to query within the Log Store. An example identifier might be: 0x123.../streamr-temporary-storage_namespace_key123/owner-specific-namespace. The use of a namespace prefix such as streamr-temporary-storage_namespace_key123/ facilitates the parallel search and querying of common but independent data streams within the Log Store. The query outcome presents an aggregated dataset containing every message's metadata, ensuring it is verifiable.

Smart Contract:

Reporting Data stored through the Log Store is ingested into a Smart Contract on an hourly basis through oracle technology such as the Gelato Network’s Web3 Function, or Chainlink Functions. Consensus is reached based on reported events through the confirmation-based consensus paradigm. Every reported event should match identical content from another reporter. Summing up these confirmations to a threshold determines consensus. Storage Nodes receive $DATA tokens relevant to their stored data, and Reporter Nodes earn $DATA tokens if their events are confirmed. The participation of Reporter Nodes is capped, and they earn reputation, ensuring their place within the restricted network. Reputation is essential for data to be included in consensus decisions and for the allocation of rewards. The consensus model is built on Proof-of-Event, requiring no stake for node participation. Invalid participation results in ejection and blacklisting.

Coordination via Notion:

As this is a research project with a defined scope, a Notion Page/Database(s) serves as the user interface. It provides a platform for the community to discover vetted Storage Providers and Reporter Nodes. Additionally, it features a comprehensive guide on how to participate as a node operator and how to stake funds into the Smart Contract for temporary data storage.

Video & Diagram

To further illustrate the solution, and diagram and video walkthrough can be found here:

https://www.loom.com/share/640e8310bbdd430695dd36e47b89606e?sid=9a56f6b2-eb87-4d9c-8e7f-2dada91fdf1b

Execution & Deployment

The software developed for this research project will be completely open source.

Usher Labs will operate a Storage Node in Sydney, AU and Ohio, US to complement the default Streamr Germany Storage Node.

Node Operators that wish to participate can follow the documented guides.

Reporters will need to create a dedicated stream, ensuring Log Store funding for the stream, and then operate the open source reporting node.

Storage Providers will be guided through node operation, but will need to complete a form application to be listed on the Notion page so that Stream Owners are aware of vetted Operator Storage Node public keys.

Storage Pricing

The Streamr Germany Storage Node will remain free during this research project as to prevent rapid changes to user experiences.

This means that adding additional storage nodes is a means of maximising data replication for fault tolerance and data security.

The base storage fees charged to Stream Owners per additional storage node will be:

Write Request per GB	0.2500
Read Request per GB	0.0000
Data Storage (GB/month)	0.1000

These fees are very conservative, and are expected to increase as the project is adopted and assimilated into the Streamr ecosystem.

Funding

This proposal (SIP) enables the Streamr Community to vote on funding the research project.

Due to the immediate and small nature of the scope, being a proof-of-concept, the funding request is for $7,500 USD dollars.

Voting

This proposal (SIP) will allow $DATA token holders to vote on the most appropriate way forward for the delivery of these improvements to the Streamr Temporary Storage facility:

• Approve & Fund: Approval by $DATA token holders to commence the project.
• Postpone: The proposal requires further review and will be postponed for the next governance round.
• Reject: Reject the proposal.

The case AGAINST the proposal

• The Streamr Network may have an alternative vision for temporary storage.
• The proposed solution creates too much dependency on third-party technology adding a layer of security risk to Streamr Developers and Data Providers.

The case FOR the proposal

• Inevitable development: To scale the Streamr Network’s temporary storage offering without any re-evaluation or deviation of existing technologies, a data protocol is required.
• Monetisation: Streamr’s core storage node is currently a free offering because it requires a mechanism for managing usage and billing. This proposal outlines as a road to this feature.
• (Optional) $DATA Utility: The research project can facilitate all of it’s digital asset management using the existing $DATA token, enabling a new utility for the token.
• Community Engagement: Data protocols powered by the Streamr Network empower the Streamr ecosystem with new incentives for Community Node Operators to get involved through rewards.