Using Python to Automate Web Navigation

I use Selenium to log into and download files from websites.  Here's and example to get started.

Install Selenium Library

Assumptions

I'm assuming you have python installed and you have admin rights or are working inside a virtual environment (virtualenv) where you can install python libraries.

I also assume you can access the internet using the "pip" command.  If you're behind a proxy or firewall you might need to download Fiddler to tunnel your internet through the proxy.

Steps

1. Open a command prompt
2. Type: "pip install selenium"

Download the WebDriver for Your Browser

Assumptions

You're using Chrome.  Firefox may be able to skip this step as i don't think it needs a specific driver.

You can extract files from a zip file.  

Steps

1. Go to the download site
2. Download and extract the driver to a location you can access later with your program

Start Coding

Assumptions

I'm using the interactive python REPL that came with my installation of python called IDLE.  The code should work no matter what IDE or tool you're using.

I'll put code next to a description of each step.  For example:

1. Print hi to console: print("hi")

Steps

1. Import the selenium web driver module: from selenium import webdriver
2. Store a path to the web driver to a variable: chromedriver=r"C:\Test\chromedriver.exe"
3. Instantiate the driver: driver = webdriver.Chrome( chromedriver )
4. Open a web page: driver.get("http://stackoverflow.com")

That's it!  You can do a lot more with the selnium library, like log in to website, fill out forms, simulate button clicks, download files, etc.