http client: faster timeout when no network

Question

When making a http Get request in Go, it waits the full timeout time before returning an error, even when there is no network connection.

I assume in the internals it knows pretty quickly it has failed; I'd like that error to propagate up as soon as possible instead of waiting for the timeout time. I do want it to try for 20s when the network is there and just slow. How can I setup a client with this behaviour?

Code to see issue:

var client = &http.Client{
    Timeout: time.Second * 20,
}

response, err := client.Get(url)

If it matters I'm using gomobile and it's running on the iOS simulator.

Answer 1

The answer here ends up being quite simple: golang internals can and does know when there is no network, and propagates up the failure in a timely manner without waiting for the 20s timeout. Nothing is sent over the network, and there's nothing to wait on. Go seems to be doing everything properly, and there's no changes needed to the sample code.

The issue still reproduces consistently, but only on the iOS simulator. It seems to be an issue specific to how the iOS simulator mapped connections to the host OS. Not sure if this is a longstanding issue, or a one-off on my MacOS/simulator pairing. On the host MacOS and real iOS devices it works properly, timing out immediately, when there's no network interface.

There's no need for an extra request, as that's just another path to same conclusion, which adds the possibility of other failures. Might be helpful to differentiate network issues from issues with specific service, or get an indicator of real network status (past the existence of a connected network interface) sooner.

Answer 2

The initial TCP handshake will fail without a connected network interface, and at that point there's no chance of success. What's it doing for the next 19.99 seconds?

The Go's HTTP client waits for the full timeout duration before returning an error when there is no network connection, because the client does not know the state of the network when it initiates the request. It only knows that it has not received a response within the specified timeout period.
The client is waiting for the operating system's TCP stack to return an error when it fails to establish a connection, and that can take some time because of various factors like the network configuration, the operating system's TCP/IP implementation, etc.

I do not know of a direct way to set a faster timeout for the case where there's no network connection in Go's HTTP client.

Another approach would be to combining the use of a context with a timeout and a separate goroutine that checks the network connectivity status. The goroutine would cancel the context if it detects that there's no network connection.
The request would use NewRequestWithContext

For instance:

ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()

// Check network connection in a separate goroutine.
go func() {
    if !isNetworkAvailable() {
        cancel()
    }
}()

req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
    log.Fatal(err)
}

response, err := client.Do(req)
if err != nil {
    log.Fatal(err)
}

You would need to implement isNetworkAvailable(): a function to check if there is a network connection available. This can be platform-specific, and on iOS you would likely need to use a system API to check the network status.

Note that this approach might not work perfectly in all scenarios. For example, the network could become available just after isNetworkAvailable() checks it, but before the HTTP request is made. In this case, the request would be canceled even though the network is available. But it might help in cases where the network is known to be unavailable for a longer period.

---

For instance:

import (
    "net"
    "time"
)

func isNetworkAvailable() bool {
    timeout := 2 * time.Second
    conn, err := net.DialTimeout("tcp", "8.8.8.8:53", timeout)
    if err != nil {
        return false
    }
    if conn != nil {
        defer conn.Close()
    }
    return true
}

This function tries to establish a TCP connection to 8.8.8.8 on port 53 (the port used by DNS servers) with a timeout of 2 seconds.
(as JimB adds in the comments: "All you can do is try to make some network calls with shorter timeouts")

If the function can't establish a connection within that time, it assumes there's no network connection and returns false.
If it can establish a connection, it closes the connection and returns true.