Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

selenium.common.exceptions.TimeoutException locating an element using Selenium with Python

I'm trying to crawl a page that loads certain data using js request using selenium so I used "visibility_of_element_located" with a timeout. however, even setting it for 15 seconds doesn't work and I get a timeout error.

url = '''https://ukranews.com/en/news/850009-u-s-believes-russia-s-war-with-ukraine- 
will-last-until-end-of-2022-cnn'''        
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.page_load_strategy = "none"
browser = webdriver.Chrome(options=options,
                           executable_path='C:\\Users\\313-SsS\\Downloads\\Compressed\\chromedriver_win32\\chromedriver.exe',
browser.get(url)
timeout_in_seconds = 5
WebDriverWait(browser, timeout_in_seconds).until(ec.visibility_of_element_located((By.CLASS_NAME, 'article_content')))
browser.execute_script("window.stop();")
html = browser.page_source
Here is the error I got:
selenium.common.exceptions.TimeoutException: Message: 
Stacktrace:
Backtrace:
Ordinal0 [0x008E7413+2389011]
I also use the same code for another website that also loads the content using js and it works just fine.
Solution
To wait for the visibility of the <h1> element with text as U.S. Believes Russia's War With Ukraine Will Last Until End Of 2022 - CNN you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.article_content h1")))
Using XPATH:
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='article_content']//h1")))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
                still have the same problem. after 20 seconds I got selenium.common.exceptions.TimeoutException error.
– mohsen amini
                Apr 16, 2022 at 19:22
                @mohsenamini Sounds to be a different question all together. Feel free to raise a new question as per your new requirement.
– undetected Selenium
                Apr 17, 2022 at 19:55
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.