HtmlUnit: Handling Hidden Google reCAPTCHA Token in Login Automation

Question

I'm using HtmlUnit to automate the login process on a website. The website employs a Google reCAPTCHA to protect its login form, and the reCAPTCHA token is hidden in the HTML. I need to obtain and use this token for a successful login. The token is automatically sent in the login (POST) request http-form-urlencoded as the content type.

Here's my code:

try (final WebClient webClient = new WebClient()) {
webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setRedirectEnabled(true);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getCookieManager().setCookiesEnabled(true);

HtmlPage page = webClient.getPage(LOGIN_URL);
HtmlForm form = page.getFormByName("login-form");

form.getInputByName("login[username]").setValueAttribute(USERNAME);
form.getInputByName("login[password]").setValueAttribute(PASSWORD);

HtmlButton submitButton = form.getFirstByXPath("//*[@id=\"send2\"]");
HtmlPage loggedInPage = submitButton.click();

System.out.println("Page Body: " + loggedInPage.getBody().asText());

} catch (Exception e) {
    e.printStackTrace();
}

Why HtmlUnit is not working with this as intended I mean as I see it's not sending with the login request, but I see all necessary reCaptcha-related URLs are executing, this is even hidden reCaptcha so it doesn't need user interaction. Can anybody help me? Any help or example code would be greatly appreciated.

Answer 1

You can't login like this to that web page. Human visitor verification such as reCAPTCHA and cloudflare turnstile are designed to stop and prevent exactly what you are trying to do.

You will have to contact that website's management for access to an API that will allow you login using automation.

If there is none, you will have to login manually using your web browser and then copy the login session cookies and add them before making a request:

webClient.getCookieManager().addCookie(new Cookie(domain, name, value));