Getting Started

1. Install block-crawler:

   python -m pip install block-crawler
   

2. Setup AWS credentials for Boto3

3. Setup the DynamoDB schema.

   Warning

   The following command will reset any previously loaded data!

   block-crawler dev reset-db
   

4. Get the current block height

   block-crawler evm block-number wss://some-archive-node-uri
   

5. Load the blockchain data to the current block height

   block-crawler nft --blockchain ethereum-mainnet \
   --evm-rpc-nodes  wss://some-archive-node-uri 1 \
   load --increment-data-version=True 1 16_622_533 16_622_533
   

   Note

   You may encounter errors while running. If so, resolve the issue and run the load command again with the starting block at the block after the last known loaded block based on the logs and increment-data-version set to False. increment-data-version should only be set to True on the first run of the loader.

6. Get the current block height again

   block-crawler evm block-number wss://some-archive-node-uri
   

7. Crawl the blocks from the end of the load to the current block height.

   block-crawler nft --blockchain ethereum-mainnet \
   --evm-rpc-nodes  wss://some-archive-node-uri 1 \
   crawl 16_622_533 16_701_373
   

   Note

   Depending on how many blocks you need to crawl, you may need to run the command multiple times. You don’t want to start tailing until you are within a few hundred blocks of the current block height.

8. Seed the database with the last block processed by crawl.

   block-crawler nft seed 16_701_373
   

9. Start the tailer to keep up to date with the latest blocks.

   block-crawler nft tail
   

More detailed information regarding these commands can be found in the User Guide.