Estimating the amount of censorship in the Ethereum blockchain. Censorship monitoring system with machine learning under the hood.
To kick things off, and in order to understand the big picture, let's explain how and why censorship is implemented in the blockchain world. In order to prevent sanction listed individuals from obtaining funds, governments (in our case, the U.S. government represented by OFAC) prohibit transactions that in any way affect these people. Naturally, they are not able to directly prohibit or cancel such transactions, so they make lists of "prohibited" addresses and direct participants of the transaction execution process, guided by them, may not include such transactions in blocks for quite a long time. This is what blockchain censorship is all about.
The main goal of our project is to create a tool that allows to estimate the approximate amount of censorship in Ethereum, as well as to estimate the impact of one of the biggest players in this market, Lido Finance, on the amount of censorship.
Challenges
One of the challenges in solving this problem has been the lack of any universally accepted metrics for censorship in the blockchain world.
In our project, we came up with five main metrics: - Latency of transactions affecting wallets on the OFAC list - transaction latency caused directly by censorship ( because not all "bad" transactions are ultimately censored) - share of each Lido Finance validator in the volume of validated transactions (separately for OFAC compliant and for OFAC non-compliant) - ratio of these shares for each Lido validator - ratio of these shares by pools
The need to collect, process and store a huge amount of on-chain data, as well as data from BeaconChain and the OFAC website, was also an important obstacle. For this purpose, we launched an Ethereum node and built a system to collect and process data in real time. We use python as our main development language, and store the data in mongodb.
