This is a dynamically-loaded website https://www.gelbeseiten.de/suche/hotels/n%c3%bcrnberg.
I'm trying to follow every link from the results. I found //article[@class='mod mod-Treffer']/a to follow the search result links. But the problem is this XPATH works only for a couple of links. For the rest of the others, I don't find any Selector. Because the other are using probably JS to make this action. I'm not familiar with this kind of dynamic website. So, I don't know how to get the selector from this kind of website. Any suggestions will be highly appreciated.
How to get the XPATH or CSS selector from dynamically loaded website to follow links?
68 views Asked by Raisul Islam At
1
There are 1 answers
Related Questions in WEB-SCRAPING
- Using Puppeteer to scrape a public API only when the data changes
- Scraping information in a span located under nested span
- How to scrape website which loads json content dynamically?
- How can I find a button element and click on it?
- WebScraping doesnt work, even without error
- Need Help Extracting Redirect URL from a div Element with Specific Class Name in Python Selenium
- beautifulsoup library not showing below #document data inside iframe tag in python
- how to create robust scraper for specific website without updating code after develop?
- Optimizing Selenium script for faster execution
- Parse Dynamic Power BI table with selenium
- How to extract table from webpage that requires click/toggle?
- SSL Certificate Verification Error When Scraping Website and Inserting Data into MongoDB
- Scraping all links using BeautifulSoup
- How do I make it so all arrays are the same length?
- I am getting 'NoneType object is not subscriptable' error in web scraping method
Related Questions in XPATH
- How can I load all the elements of a webpage with Selenium?
- Why is the copied XPATH not working for selenium?
- When I'm typing an Xpath or CSS selector in the console why won't matching results appear while typing? Results only appear after pressing Enter
- Nokogiri only returning 5 results
- XPath - how to exclude text from child node
- xpath issue in nested div
- Question using XPath to look for a sibling of a td with a certain name
- How to separate XML tags in freemarker in body function
- Wait using Path (Puppeteer)
- why can't I retrieve the track of my Spotify playlist even i have given correct full xpath
- How do I click the correct link based on text contained in another element using Selenium + Python?
- PHP DOMDocument ignores first table's closing tag
- String tokenise an xpath expression
- Problem to get into the next page, Selenium
- Scrapy / extracting data across multiple HTML tags
Related Questions in SCRAPY
- pagination, next page with scrapy
- Scraping Text through sections using scrapy
- How to access Script Tag Variables From a Website using Python
- xpath issue in nested div
- How to fixed Crawled (403) forbbiden in scrapy?
- Cannot set LOG_LEVEL when using CrawlerRunner
- Scrapy handle closespider timeout in middleware
- Scrapy CrawlProcess is throwing reactor already installed
- Scrapy playwright non-headless browser always closing
- why can't I retrieve the track of my Spotify playlist even i have given correct full xpath
- Scrapy - how do I load data from the database in ItemLoader before sending it to the pipeline?
- Scrapy Playwright Page Method: Prevent timeout error if selector cannot be located
- Why scrapy shell did not return an output?
- Python Scrapy Function that does always work
- Scrapy / extracting data across multiple HTML tags
Related Questions in CSS-SELECTORS
- When I'm typing an Xpath or CSS selector in the console why won't matching results appear while typing? Results only appear after pressing Enter
- CSS responsive bug
- not able to target element
- How to handle regex in BeautifulSoup / CSS selector?
- selector name with symbol
- Create pattern repeating every 3 elements but only on those that contain a div with class
- Why can't I click on button within a embed element using Selenium C#?
- Select `td` in rows that are only inside a root table but not inside nested tables
- CSS Techniques to Hide Scroll Bars and Disable Text Selection?
- How can I make my "display: grid" responsive using media queries for mobile devices and tablets?
- Apply unknown (at time of programming) stylesheet to only one div
- How can i affect all content within a tag except a specific child tag
- hide nth child items after multiple of 7
- How to make every subsequent sibling after a button-wrapper disappear and restore their visibility again by pressing this very button?
- How to specify elements belonging to the same class in HTML?
Related Questions in SCRAPINGHUB
- Scraping 'offers' data from OpenSea
- Persistent error deploying Docker image to Scrapy Cloud
- Issue with Docker Image for Custom Deployment on Zyte (formerly Scrapinghub) Platform
- Scrapy Cloud Deployment Error: "Referer Header Missing"
- Webscraping yml files from Github
- I'm having issue while deploying scrapper to Zyte formerly (Scraping hub)
- Extract data from company sharepoint using Python
- Can't fetch url in scrapy shell with splash
- Is it possible to create a proxy failover with Python Scrapy?
- Web scraping using Octoparse
- How to get the XPATH or CSS selector from dynamically loaded website to follow links?
- How to save Scrapy Broad Crawl Results?
- Selenium Problem extracting Google business description
- Why error with installing csv when its part of python core package in scrapinghub
- YouTube Subscriptions List Scraping
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
I will post this as an answer, without actually giving you the code, as it might help you more in the long term.
First, load that page in browser with javascript disabled (there are ways with disabling js in browser directly, or use an extension like ublock origin, etc - look it up).
You will notice that only the first 2 hotels are fully loading - the rest are being loaded dynamically by javascript (which in this case is disabled). There are 13 hits for
//article[@class='mod mod-Treffer']/aselector, while there are more hotels on that page. However, each hotel is wrapped in an<article>tag, and that tag hasdata-realid="[...]"attribute. The url for each hotel would behttps://www.gelbeseiten.de/gsbiz/{data-realid}.This is how you can get all those hotels' profile links.