How to monkeypatch requests to save the responses?

Question

I have a package with a bunch of classes that act as wrappers for other systems. For example:

# mysystem.py
import requests

class MySystem():
   def __init__(self):
      self.session = requests.session()
   
   def login(self, username, password):
      self.session.post("https://mysystem.com/", data={"username": username, "password": password})

I also have integration tests:

# test_mysystem.py
from mysystem import MySystem

def test_login():
   system = MySystem()
   system.login(username="test", password="P@ssw0rd")

I now want to mock out the requests to the real systems so I wouldn't depend on them to run my tests. I will use responses to register the fake responses into a fixture (I'm using pytest).

The problem is that there are many classes (and more to come), so manually collecting every desired response would be a tedious task. My ideia to automate this, since the integration tests already do the job of requesting every single page I'm interested in mocking up, is to create a fixture that would save all the responses (along with the URL) after a full run of my tests. I could later use the saved information to register my responses more easily.

But how can I monkeypatch the requests made by the sessions that are properties of each system class during testing time?

Answer 1

It seems like you want to do a record/replay of the network calls and responses.

I know a couple of libraries you can use:

• pytest-automock
• vcrpy - which has a couple of pytest/unittest wrappers: pytest-recording, vcrpy-unittest, pytest-vcr

The basic principle is the same: each library has a "record mode", where you run your code once and the actual network calls are made and serialized into some file. When the tests are run afterward - the file is deserialized and used to replay the same response instead of making the network call.

With pytest-recording, all you need is to add the vcr mark to the test you want to be recorded:

# test_mysystem.py
import pytest
from mysystem import MySystem

# Alternatively, you can add the mark to pytestmark to mark all the methods in the module
# pytestmark = pytest.mark.vcr(filter_post_data_parameters=["password"])

@pytest.mark.vcr(filter_post_data_parameters=["password"])
def test_login():
   system = MySystem()
   system.login(username="test", password="P@ssw0rd")

Then, the first time you run your test using pytest test_mysystem.py --record-mode once, it will proceed as usual but also record all the network interactions in yaml files under the "cassettes" folder.

The next time you run the same command above, the cassettes will be loaded and no actual request will take place. You can make sure this is the case by using the option --block-network or even disconnecting your machine from the network.

Be advised that, by default, all the transmitted data will be recorded. In your specific example, you will probably want to leave out the password. Fortunately, vcrpy supports filtering. You can see I already did that in my example by passing filter_post_data_parameters=["password"] to the mark.