nginx load balancer on top of two selenium grids

Question

I am trying to setup an nginx load balancer on top of two selenium grids so that i can pass the nginx url instead of the grid url and the load can be effectively distributed on the two since one gets overloaded. This is my nginx configuration:

upstream selenium_grids {
    server <grid1_ip>:<grid1_port>;
    server <grid2_ip>:<grid2_port>;
}

server {
    listen 80;
    server_name <nginxIP>;

    location / {
        proxy_pass http://selenium_grids;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

when I hit the nginx IP I see it oscillating between the two grids, which seems correct, but when I pass the nginx url (http:///ui, the tests fail with browser connectivity issue. What am I missing?