Request for funds - Refactor Cowllector - 0xww

Revamping our Infrastructure

The multichain future we predicted is officially here. It’s become clear that Beefy will eventually be deployed in tens or hundreds of different L1s and L2s. We will have to support thousands of vaults running either single or multi strategies. Many of these vaults will be cross-chain and will have to execute work in multiple chains at the same time. For the past few months we have been revamping the Beefy infrastructure to better accommodate those scaling efforts. This has included everything from our treasuries, multi sigs, timelocks, automation, reporting, etc. One of the last pieces of infrastructure that we need to upgrade to be on par with the required quality of service is our harvesting bot, also known as the Cowllector.

Vault Harvesting

Beefy is a yield optimizer with many vaults running. Each vault implements different strategies to make money for its depositors. To make a vault execute its strategy, they have a public function that anyone can execute. This function is commonly called harvest(). People can either manually call the function, or create bots that call it when certain criteria is met. Whoever calls the harvest function gets a small payment for their effort. The payment is a % of the rewards earned in the period since the last harvest. This means that people cannot exploit the vaults by calling harvest frequently. They will only achieve burning gas by doing that.

Harvesting Ecosystem

There is a diverse ecosystem of users and bots keeping the Beefy vaults running. There are four main types of participants involved.

1. Manual calls: Large depositors sometimes call the harvest function manually before withdrawing from a vault. This helps ensure that they get up to the last second of yield. They are helping the other depositors by doing it, by compounding all the rewards. We cannot depend on them as a core strategy to keep our vaults running as they’re too sporadic.
2. Community bots: Some community members maintain harvesting bots. They mainly do this for self-interest. They calculate the balance between the cost of calling the harvest function and the rewards that they will get. Once they determine that they will make a profit by calling harvest, they do so. The one problem is that these bots don’t really harvest vaults unless it makes them money. The Beefy protocol might want to keep some vaults running even if they’re unprofitable to maintain. Those vaults might attract users that we can later redirect to more profitable vaults.
3. Automation networks: These are networks like Gelato or the Chainlink Keepers. These networks have workers that you can hire to perform certain tasks. These are a cost effective way to keep our vaults running but those networks are not in every chain that Beefy is present. Another drawback is that If we depend on them, it’s another piece of third party infrastructure that we need to be present before we can deploy in a chain.
4. The Cowllector: This is the main harvester bot that Beefy maintains. Its role within the harvesting ecosystem has evolved through time. Right now we’re focusing on providing a minimum level of service for all vaults in all chains. This is done even if a certain vault is harvested at a loss for the protocol. The frequency varies depending on vault apy, blockchain execution costs, vault TVL and popularity, etc.

Problem

Cowllector take so long (hours) on harvest all strat as long that beefy has been scaling. What was the problem? Probably the main problem here comes from trying to harvest all strats for all the chains in the same single process and in a synchronous way, making the process take longer than hours. Also there are many steps that can be refactored to solve in faster ways

Proposed Solution

Make system work in multi-process and asynchronous way, harvesting all the strats in differents process (one process per chain) and solving all possibles consults in asynchronous ways, faster and saving time (what this main refactor is focus in):

• In general terms, upgrade the reliability of the system, meaning that vaults get harvested as expected and the harvester bot is able to overcome basic issues that arise.
• Better reporting when the bot can't solve something on its own. For example if a vault is reverting on harvest, it should send a notification over Discord so that we can see it and handle it.

Features Added ( Changelog )

• Set multi-process behavior, one child-proccess per chain
• Create a local "gasLimits.json" where persist "gas limit" estimation (avoid erros on rpc when request for eth_estimateGas)
• Refactor "shouldHarvest" function to run in parallel for every strat, making a list of which strats we should continue to harvest (this save the mostly of the time)
• For Every strat to harvest set a static gasLimit and gasPrice to be harvest
• Because harvest rewards are pay in wrapped native token, before every new harvest, check if we have enough wrapped token to unwrap
• Checking in-code that harvest has enough balance to pay the transaction gas and run harvest function, avoiding sending bad transactions and saving the time of waiting that blockchain returns us the same error
• Implement a list of possible errors that when it happen cowllector move forward with the next strat to harvest
• retry many (5) times the harvest function before desist on do it, only in case that blockchain returns an unknown error
• every strat harvested, even success or fail, returns JSON with message of harvest operation, preparing an Array of JSON with harvests results
• Create a broadcast-API to spread message through different channels for posibles alarms like refund harvester-accounts and reports (in another repo: https://github.com/0xww/beefy-broadcast-api )

Features Being

• Automation (CI) periodically sync cowllector strats.json with beefy-API strats using GitHub Action
• Automation (CI) periodically update gasLimits.json using GitHub Action
• Send Harvest Report to Discord channel
• Add Error reporter (may be use sentry.io), for unknown or unexpected errors
• Improve gasPricing estimator, persisting and array of gas used when harvest function was called
• Fork beefy-broadcast-api into beefyfinance repositories and autodeploy it on heroku
• After harvest all strats of every chain, send report to beefy-broadcast-api (also we can persisted that report data in a S3 bucket in GCP or upload it to ipfs, I am open to discuss it )
• Maintaining is included

Fund Requested

$ 10.000