Selenium's WebDriver & Python: Dealing with script timeouts

Selenium's WebDriver is a great way to interact with web sites in an automated way. It's primarily for testing, though I've also had occasion to use it for other purposes (with the permission of the relevant site owners).

In order to interact with sites that use a lot of Javascript, it's helpful to use Selenium's Firefox driver. It works great, but I did have one problem, where scripts were timing out. You can modify Firefox's timeouts via changing preferences, but Selenium's Firefox driver uses its own profile, so it ignores Firefox's usual means for changing preferences.

I searched the web for solutions and found various suggestions that didn't work. Perhaps many of them worked with pre-WebDriver versions of Selenium, I don't know.

In any case, the following worked for me to change the timeouts to 10 minutes:

from selenium import webdriver

profile = webdriver.firefox \

.firefox_profile.FirefoxProfile()

set_pref = profile.set_preference

set_pref('dom.max_script_run_time', 600)

set_pref('dom.max_chrome_script_run_time', 600)

driver = webdriver.Firefox(fx_profile=profile)

Note that the dom.max_chrome_script_run_time preference has nothing to do with Google Chrome, it's related to Firefox's internal Chrome URL's. In fact, for my purposes, I only needed to change dom.max_chrome_script_run_time.

Note: if you're not familiar with WebDriver, do help(webdriver.Firefox) for a description of the methods you have available for interacting with sites. Hint: the oddest thing is that to fill in a text field, you get an object representing the element with the text field, and then call myElement.send_keys("the text").

Update: Docs on the Python bindings may be found here.

 

Comments

[Mike Combs]

Thanks! I didn't know about the help command, and the webdriver syntax is so verbose compared to the IDE syntax (1 line to get the element, 1 line to set it), that I figured I must be doing something terribly wrong.

[anupam saini]

Thanks Gary ,

I needed to prevent Selenium firefox profile from downloading images and css on my sites page .
Your tip was really helpful

[Carrie]

Thank you so much for this post, I've been googling for a solution to this with Python & WebDrvier all afternoon. I was setting the first preference, but not the chrome one, and this fixed the problem. THANK YOU!

[Greg Meece]

Although it doesn't address what you're dealing with here directly, I've greatly enjoyed getting to know the Robot Framework for automated testing. It is written in Python, so that makes it very extendable. The Selenium2 Library also makes websites drop-dead simple to automate.