Why can't I scrape data from etherscan

Question

I am new to web scraping. I have been trying to use bs4 to scrape this website with python: https://etherscan.io/token/0xb3999f658c0391d94a37f7ff328f3fec942bcadc#balances but it didn't work. After researching a bit, I switched to using python playwright, but it also didn't work. Now I am trying to scrape with selenium, but it still doesn't work. Here is my selenium code:

from selenium import webdriver
from selenium.webdriver.common.by import By
import os
os.environ['PATH'] += r"C:\chromedriver_win32\chromedriver.exe"
driver = webdriver.Chrome()
driver.get('https://etherscan.io/token/0xb3999f658c0391d94a37f7ff328f3fec942bcadc#balances')
driver.implicitly_wait(20)
driver.switch_to.frame(driver.find_element(By.ID, 'tokeholdersiframe'))
elements = driver.find_element(By.CLASS_NAME, 'align-middle text-nowrap')

for element in elements:
    print(element.text)

Is the problem from my code or the website? If it is from my code, how do I fix it?

The data I am trying to get from is this: enter image description here.which is in an iframe The result should be the text from the html codes. Ex: Binance 14 99,037,941.875131110474174858 9.9038% $55,716,765.34, etc.

Answer 1

There are two small problems in your code.

• You are incorrectly using By.CLASS_NAME - select only one class, not two. Since the element is unique, I recommend using:

  • By.XPATH driver.find_element(By.XPATH, """//*[@id="maintable"]/div[2]/table/tbody""")
  • By.CSS_SELECTOR driver.find_element(By.CSS_SELECTOR, "table tbody")

• The element is not iterable. To print the text, use print(element.text).

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()

driver.get('https://etherscan.io/token/0xb3999f658c0391d94a37f7ff328f3fec942bcadc#balances')

#driver.implicitly_wait(20)

driver.switch_to.frame(driver.find_element(By.ID, 'tokeholdersiframe'))

#element = driver.find_element(By.CSS_SELECTOR, "table tbody")
element = driver.find_element(By.XPATH, """//*[@id="maintable"]/div[2]/table/tbody""")

print(element.text)