I need to iterate through a series of webpages using Zyte (a rotating proxy service). I used a curl converter to convert Zyte's instructions to an R friendly command. Unfortunately, when I try to paste the webpage into the full body of instructions I get a status 400. I'm not sure exactly what is wrong here. When I look at the output from the paste object it's exactly the same as what the curl converter specifies and what works without the paste complication. I've tried wrapping data in eval(), that doesn't seem to help.
headers = c(
`Content-Type` = "application/json"
)
data = paste0("'",'{ \n "url": ',link.vec[i],'\n }',"'",sep="")
list_html.list[[i]] <- httr::POST(url = "https://api.zyte.com/v1/extract", httr::add_headers(.headers=headers), body = data, httr::authenticate("SPM-KEY", ""))
This is what the curl converter specifies:
headers = c(
`Content-Type` = "application/json"
)
data = '{ \n "url": "https://denver.craigslist.org/search/cta?bundleDuplicates=1&min_price=1000&postal=80003&purveyor=owner&search_distance=100#search=1~gallery~29~0"\n }'
res <- httr::POST(url = "https://api.zyte.com/v1/extract", httr::add_headers(.headers=headers), body = data, httr::authenticate("SPM-KEY", ""))