Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Some said that it is necessary to set waiting time, otherwise it may cause NoSuchElementException error because the page didn't finish loading.

However, I tried the following code (without any waiting lines) with a low-speed network (I limited the speed), and the login process still ran smoothly (it kept loading at first, and when I canceled the limit, the loading finished immediately and things went on...).

from selenium import webdriver
import json
# Get user info
with open('wjxlogin.json', encoding='utf-8') as fp_login:
    login = json.load(fp_login)
username = login['username']
password = login['password']
# First login
browser = webdriver.Firefox()
browser.get('https://www.wjx.cn/login.aspx')
browser.find_element_by_id('UserName').send_keys(username)
browser.find_element_by_id('Password').send_keys(password)
browser.find_element_by_id('RememberMe').click()
browser.find_element_by_id('LoginButton').click()
So, I wonder if there's an automatic waiting mode in current Selenium, which does not allow executing the next line until the last process is finished?
And is it still necessary to set a waiting time (in my code for example)?
The default timeout in Selenium is set to  0. What this means is that Selenium will throw NoSuchElementExpception after the page finishes loading and the particular element is not present in the DOM. The default page load timeout is pretty high (I think its 600 seconds) - this is why what you have tried did not impact the test execution while network was bad.
However - changing the page load timeout will not result in NoSuchElementException instead a different exception will be thrown. If you want to experiment with the settings:
driver.set_page_load_timeout(3)
Will probably get you some failures with limited speed network.
When it comes to the waiting - as you saw yourself you did not need it. It is required only in some specific scenarios - i.e. dynamic content gets updated, user interaction loads some elements, etc.
                also worth noting that some dynamic content will update location bar with a bookmark which fires onload.  Sites that use this method may not require a webdriverwait.   (It's a technique that may be outdated, but allowed for bookmarking and proper forward back button functionality.)   The hashchange events fire dynamic loads.
– pcalkins
                Jun 4, 2019 at 19:31
Yes as per the best practices, you need to set a waiting time i.e. a waiter in the form of WebDriverWait in conjunction with either of the expected_conditions when using Selenium.
You can find a relevant detailed discussion in:
Do we have any generic funtion to check if page has completely loaded in Selenium
How to sleep webdriver in python for milliseconds
This usecase
As per your usecase to login within the website using a valid set of credentials you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC 
options = webdriver.ChromeOptions() 
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.wjx.cn/login.aspx")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.validate-input.user-name#UserName"))).send_keys("TheCoolestStacker")
driver.find_element_by_css_selector("input#Password[name='Password']").send_keys("TheCoolestStacker")
driver.find_element_by_css_selector("input#RememberMe[name='RememberMe']").click()
driver.find_element_by_css_selector("input.submitbutton#LoginButton").click()
Browser Snapshot:
NoSuchElementException
You can find a couple of detailed discussion on NoSuchElementException in:
Selenium “selenium.common.exceptions.NoSuchElementException” when using Chrome
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element while trying to click Next button with selenium
                Thanks! And do you know why the error didn't happen in my situation? It seems that the "find_element..." line won't execute until the loading is successful or reaches an error page. That's why I'm doubting the necessity of adding "waiting".
– Coo Lest
                Jun 4, 2019 at 12:13
                @TheCoolestStacker loading is successful term is pretty much subjective from Selenium's perspective. find_element_by_* in conjunction with Implicitly Wait won't be that effective in the bigger picture when you will start interacting with dynamic elements. Hence a better approach would be WebDriverWait which I have spoke about in my answer.
– undetected Selenium
                Jun 4, 2019 at 12:29
                if there is no additional requests sent after initial pageload, then you wouldn't need to wait.  so it's not always needed
– Corey Goldberg
                Jun 4, 2019 at 19:07
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.