I'm trying to crawl this website, problem is it's dynamically loaded.
Basically I want what I can see from the browser console, not what I see when I right click > show sources.
I've tried some selenium examples but I can't get what I need. The code below uses selenium and get only what you get in right click -> show code. How can I get the content of the loaded page?
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium import webdriver import time # Start the WebDriver and load the page wd = webdriver.Firefox() wd.get("https://www.leforem.be/particuliers/offres-emploi-recherche-par-criteres.html?exParfullText=&exPar_search_=true& exParGeographyEdi=true") # Wait for the dynamically loaded elements to show up time.sleep(5) # And grab the page HTML source html_page = wd.page_source wd.quit() # Now you can use html_page as you like print(html_page)