Creation of DAO-Owned Catalyst Archival Node for Decentraland Core

by 0xe400a85a6169bd8be439bb0dc9eac81f19f26843 (howieDoin)

Should the following $19,800 grant in the Platform category be approved?

Abstract

We believe there is a need to provide a DAO-owned Decentraland catalyst node for the community to use for applications. This node would be deployed as an archival node to make additional, historical data available to the community. The DAO-owned node would not be used to support users in world but instead to power an ecosystem of apps; thereby replacing the load on the existing catalyst node network.

We will also be developing an open source code base that can be used in conjunction with a historical node to query common requests against the historical database, host the infrastructure, and make it available to the community for one year alongside our other DAO Data APIs.

Grant size

19,800 USD

Beneficiary address

0xe64581F067Cfdce58657E3c0F58175e638C30f2B

Email address

howie@atlascorp.io

Description

The catalyst node network currently supports users as they play in Decentraland. The node network saves information on scene deployments, user profiles, and other Decentraland content that is not directly on chain but requires storage. Any application that wants to use this data – say to display the profile pictures of the user on an app, or to understand what parcels are associated with a scene – would use the APIs available on the catalyst nodes. If the application ecosystem around Decentraland begins to scale, this may cause an undue burden or strain on the node network as this load is above and beyond what is required to support users in game.

Similar to the DAO Data Aggregation initiative we undertook at the end of last year, we believe there is a need to stand up a dedicated catalyst node which can be used by the community for applications and application development. We are requesting funds which will translate directly into infrastructure costs to support an Application node with historical data.

This idea was conceived of by @Zino of the Grant Support Squad, with Atlas CORP volunteering to support the deployment and backfilling of data required.

Budget Specification

This project will cost the DAO $19,800 and will take approximately 2-3 months to deploy. This breaks down into:

$7,400.00 - Infrastructure and Hosting Costs*:

• Budget for Digital Ocean API servers for one year

• Budget for Digital Ocean Volumes (<=5TB) for one year

• Reimbursement to @HPrivakos for any data transfer or costs incurred in assisting in the set up.

*This estimate is based on current pricing for each of the above platforms. $7,400 may not last a full year if pricing is altered by the provider.

$10,000.00 - Development Costs (estimated 3 month delivery):

• Configuration and deployment of historical catalyst node

• API query development resources to produce queries and prevent overuse

• DevOps and infrastructure development resources to automate as much as possible

• Technical Writing resources to provide user-facing API documentation

$2,400 - Ongoing Support Costs (6 months post deployment):

• Code updates when breaking changes occur due to external forces

• API Documentation updates

• API query user support in the Decentraland discord

Personnel

The development of this data platform would be done by the Atlas CORP (@atlascorp_dcl) team, who have extensive experience working with this data set:

• HowieDoin – Lead Analytics & Infrastructure innovation

• MorrisMustang – Lead DCL & Solidity innovation

• JosephAaron – Operations and task management

• StaleDegree – Senior Solidity/UI Development

• RyanNFT –Solidity/DCL Developer

• MrFlickIt – UX and Design

Roadmap and milestones

Currently the footprint of a node is about 2.25 TB which is expected to grow over time. This proposal covers 12 months of cloud infrastructure costs – a server with the appropriate horsepower as well as a volume to persist the 3-5 TB of [current and future] data required to host the catalyst node with historical data. We are currently working with the Foundation to transfer the current source of historical catalyst data which @HPrivikos stewards over to our catalyst node.

This node will then be made available to the community, which is encouraged to transfer queries to this new node. Instead any application that requires catalyst node data including player profile, scene definitions, and any new functionality to get comprehensive history should be served from this application-supporting catalyst node.

An open source API will be developed to perform queries against the historical data. These API queries will include information that has been asked of the Atlas CORP team but to date are not possible to answer without a historical catalyst node. These queries may include the ability to answer the following questions:

• When did a user first sign up for Decentraland?

• What is a user’s “user number”, from the ranked order all users signed up for Decentraland?

• What parcels did a scene cover in the past?

• What parcels did a scene cover over time?

Query documentation (e.g. Redoc) will be published and made available for users looking to operate or integrate with the API.

This proposal is summarized due to technical limitations. To view it complete and vote on it, visit the DCL DAO Governance dApp