Scraping infinte scroll - GET url not working

56 views Asked by At

I am using import.io to scrape angel.co and as I usually do when there is an infinite scroll I'd open the devtools, look at the network and get the GET request with the right pagination.

Now when I do that with angel.co it simply doesnt work.

This is the GET request I have:

https://angel.co/companies/startups?ids%5B%5D=155618&ids%5B%5D=238203&ids%5B%5D=228828&ids%5B%5D=228837&ids%5B%5D=34454&ids%5B%5D=212959&ids%5B%5D=106075&ids%5B%5D=212446&ids%5B%5D=92216&ids%5B%5D=199453&ids%5B%5D=194318&ids%5B%5D=60461&ids%5B%5D=186506&ids%5B%5D=185905&ids%5B%5D=185820&ids%5B%5D=173350&ids%5B%5D=169237&ids%5B%5D=171703&ids%5B%5D=152063&ids%5B%5D=148409&total=149&page=5&sort=joined&new=false&hexdigest=302cb17792e051f215c6bbaac5786ee35415c894

Which does not work with import.io even if there is actually the right pagination.

Any idea?

Thank you a LOT!

Best,

0

There are 0 answers