Skip to content
W

Web Scraping with Selenium

This is a small project in which I have worked with Selenium and Chrome Drivers to automate a small task.

6

Created on 26th February 2023

•

W

Web Scraping with Selenium

This is a small project in which I have worked with Selenium and Chrome Drivers to automate a small task.

Carousel Gallery Item: 1
Carousel Gallery Item: 2
Carousel Gallery Item: 3

The problem Web Scraping with Selenium solves

Task to automate

  1. To extract normal transactions from the link: https://bscscan.com/txs
  2. Store them in

    json

    fromat

Installation

To install Selenium for Python

run

pip install selenium

in the terminal

Chrome Driver needs to be downloaded based on your browser version

  1. Open Chrome browser in the desktop

  2. Open Settings in Chrome

  3. Click on About Chrome and check your version
    image

  4. Visit https://chromedriver.chromium.org/downloads and download Chrome Driver for your version
    image

Reference links

  1. Selenium - Getting started: https://www.selenium.dev/documentation/webdriver/
  2. Chrome driver - Getting started: https://chromedriver.chromium.org/getting-started

Challenges I ran into

It was a bit difficult to figure out proper xpath of the element to scrape the right data as the data on website was in the form of html <table>, with some of <td> containing no data. In that case I had to carefuly observe the pattern in <td> tags containing data otherwise it lead to an error.

Technologies used

Selenium

Python

JSON

Jupyter Notebook

Chrome Drivers

Discussion

Builders also viewed

See more projects on Devfolio