Why does using the Google APIs Python client in appengine with GAE_USE_SOCKETS_HTTPLIB result in ResponseNotReady

Question

The code will work just fine in the SDK, but on Google AppEngine, it explodes:

  File "./oauth2client/util.py", line 137, in positional_wrapper
  File "./googleapiclient/discovery.py", line 197, in build
  File "./oauth2client/util.py", line 137, in positional_wrapper
  File "./oauth2client/client.py", line 563, in new_request
  File "./httplib2/__init__.py", line 1608, in request
  File "./httplib2/__init__.py", line 1350, in _request
  File "./httplib2/__init__.py", line 1306, in _conn_request
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/python_std_lib/httplib.py", line 1033, in getresponse
    raise ResponseNotReady()
ResponseNotReady

This only happens when GAE_USE_SOCKETS_HTTPLIB is on.

Answer 1

This happens because the Google API library is not aware of this and blindly uses a socket to connect to https://www.googleapis.com, which is not allowed. Further, the actual code hides the real error, which a permission denied on trying to create a socket.

I'm not aware of any workaround, except disabling sockets for httplib, or not using the provided libraries, but using a requests based one and installing an adapter that uses urlfetch for these domains.

Answer 2

Socket connections are only available for paid apps. Have you setup billing for your app?

From the documentation on sockets:

Sockets are only available for paid apps, and traffic from sockets is billed as outgoing bandwidth.

Why are you trying to use sockets anyway?

Answer 3

In addition to @gcbirzan's answer. Below would be steps to make it work :-

• Commenting GAE_USE_SOCKETS_HTTPLIB: True if it's set in app.yaml under env_variables
• For requests installing requests_toolbelt using pip install -t lib/ requests_toolbelt(make sure you have requests version >= 2.10.0 for this to work)
• Include below lines in main.py to enable requests_toolbelt adapter
  import requests_toolbelt.adapters.appengine requests_toolbelt.adapters.appengine.monkeypatch()

Above steps should be enough to solve this issue.