Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Also find the error below:

  File "/home/PycharmProjects/reditt/redit1.py", line 44, in get_links
    pst_hldr = wait.until(cond.visibility_of_element_located((By.XPATH, ".//*[@class='QBfRw7Rj8UkxybFpX-USO']")))
  File "/usr/local/lib/python3.9/dist-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
code:
def get_links(*keywords):
    urllist = []
    keystring = ''
    for kw in keywords:
        keystring += kw + "%20"
        keystring = keystring.strip()
    driver = webdriver.Chrome("chromedriver")
    driver.get('https://www.reddit.com/search/?q=' + keystring)
    driver.maximize_window()
    wait = WebDriverWait(driver, 10)
    tbody = wait.until(cond.presence_of_element_located(
        (By.XPATH, "//*[@class='_3ozFtOe6WpJEMUtxDOIvtU']//*[@class='q4a8asWOWdfdniAbgNhMh']")))
    tb_bar = tbody.find_element_by_xpath("//*[@class='_3ozFtOe6WpJEMUtxDOIvtU']//*["
                                         "@class='q4a8asWOWdfdniAbgNhMh']//*[@class='M7VDHU4AdgCc6tHaZ-UUy']")
    driver.execute_script("arguments[0].click();", tb_bar)
    print("end of bar")
    k = 0
    for i in range(200):
        newht = i * 500
        driver.execute_script("window.scrollTo(0, " + str(newht) + ");")
        time.sleep(0.1)
    wait = WebDriverWait(driver, 10)
    pst_hldr = wait.until(cond.visibility_of_element_located((By.XPATH, ".//*[@class='QBfRw7Rj8UkxybFpX-USO']")))
    pst_tiles = pst_hldr.find_elements_by_xpath(
        ".//*[@class='_1poyrkZ7g36PawDueRza-J']//*[@class='_2XDITKxlj4y3M99thqyCsO']//*["
        "@class='_1Y6dfr4zLlrygH-FLmr8x-']")
    for tl in pst_tiles:
        ttl = tl.find_element_by_xpath(".//*[@class='y8HYJ-y_lTUHkQIc1mdCq _2INHSNB8V5eaWp4P0rY_mE']//a")
        href = ttl.get_attribute('href')
        print(href)
    driver.close()
get_links('america', 'coronavirus', 'cases')
those classes looks dynamic in nature, try with below code :
you can use the below css_selector :
div[data-testid='search-results-subnav']+div
In code,
pst_hldr = wait.until(cond.visibility_of_element_located((By.CSS_SELECTOR, "div[data-testid='search-results-subnav']+div")))
However, this exception
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
is common, when you use Explicit wait which is WebDriverWait.
so try to use
driver.find_element_by_css("div[data-testid='search-results-subnav']+div")
and see what is the exact error you are getting.
Update 1 :
div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)
in code :-
pst_hldr = wait.until(cond.visibility_of_element_located((By.CSS_SELECTOR, "div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)")))
pst_tiles = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[data-click-id='body'] span")))
for title in pst_tiles:
    print(title.text)
Update 2 :
driver.implicitly_wait(30)
driver.maximize_window()
driver.get("https://www.reddit.com/search/?q=america%20coronavirus%20cases%20")
wait = WebDriverWait(driver, 20)
pst_hldr = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[class*='ListingLayout-outerContainer'] div:nth-of-type(2) div:nth-of-type(3)")))
pst_tiles = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[data-click-id='body']")))
for title in pst_tiles:
    print(title.get_attribute('href'))
Output :
Timeline suggests a link between Alzheimers, Prion Disease & Covid-19
Vaccines
+2,089 New Cases = 1,243,932 Total Cases in PA; +16 New Deaths = 27,941 Total Deaths in PA Take me out of Latam 🗺✈ is that a good bet?
r/CoronavirusDownunder random daily discussion thread - 13 August,
2021 The New York parade reappears Asian-Americans gather downstairs
to protest Yan Limeng spread rumors about the origin of the virus
Florida becomes epicentre of America’s pandemic as coronavirus cases
surge 50 per cent. 1 in 5 coronavirus cases nationally is found in
Florida Pre-market brief
+1,811 New Cases = 1,241,843 Total Cases in PA; +11 New Deaths = 27,925 Total Deaths in PA Florida becomes epicentre of America’s
pandemic as coronavirus cases surge 50 per cent What A Day:
Reconcilable BIFerences by Sarah Lazarus & Crooked Media (08/11/21)
r/CoronavirusDownunder random daily discussion thread - 12 August,
2021 Hungarian nationalism is not the answer - Slow Boring More
evidence suggests COVID-19 was in US by Christmas 2019 3 Major Reasons
Why I am All in GME Pre-market brief Know anyone who is PRO MASK / VAX
/ LOCKDOWN?
+2,076 New Cases = 1,240,032 Total Cases in PA; +11 New Deaths = 27,914 Total Deaths in PA What A Day: Cuom Over by Sarah Lazarus &
Crooked Media (08/10/21) r/CoronavirusDownunder random daily
discussion thread - 11 August, 2021 Democrats and their accomplices in
the media are working overtime in Texas and Florida to create panic in
an effort to pressure governors to give them back the power to
reinstate mask mandates and lockdowns in their communities. More
evidence suggests COVID-19 was in US by Christmas 2019
+1,280 New Cases = 1,237,956 Total Cases in PA; +1 New Deaths = 27,903 Total Deaths in PA Even if we don't compare the value of animal lives
to human lives, the Holocaust comparison is still valid. AMC Talks
Bitcoin, GameStop With Its Reddit Followers -- 2nd Update
(https://markets.qtrade.ca/news/story?t=iKXJizM-dSY,RB0cWxp8F57oYAx8Odv4T-UoxUxTcyQA&article=651d9d865e21f8f0#651d9d865e21f8f0)
Process finished with exit code 0
                I see you are on this page reddit.com/search/?q=america&type=link, what exactly you wanna do ? I have updated a bit above, try to run and share the exception if you face any.
– cruisepandey
                Aug 13, 2021 at 6:51
                i m on this page reddit.com/search/?q=america%20coronavirus%20cases%20 i want to fetch href link of each tiles reddit.com/r/AskReddit/comments/p2vtzs/…  like this
– pkk
                Aug 13, 2021 at 6:53
                @pkk : you should get in now if you run the updated 2 code above. that needed a little tweak
– cruisepandey
                Aug 13, 2021 at 8:22
This is expected behavior. You are using WebDriverWait, which waits for a fixed time (which you give) for an element to be visible. If the element was not found within that time, this exception is thrown. It is a way to tell you: "Hey, the element did not appear in the time you set".
Read more about this exception here: https://www.educative.io/edpresso/timeoutexception-in-selenium
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.