Detecting object usage in the least invasive and most invisible way

Question

Is there a way to automatically detect when a python object is being used (and possibly react to that)?

For example, let's say I have an object of type Foo. I did not write the class code for Foo, since it's coming from an external library.

I would like to "decorate" my object in such a way that, whenever one of its methods is used, or whenever its internal state (members) changes or is accessed, I get some logging info, like "Foo is being used".

I am using the "decorate" term to highlight that I wouldn't like to change all the interfaces where objects of type Foo are used. I would simply like to add some functionality to it.

Also I would avoid having to tinker with Foo's class code directly, i.e. by explicitly adding a print statement at the beginning of each of its methods (either way this wouldn't inform me of when its members are changing).

And I would not like to have to explicitly register my objects to some other objects, since that would be an "invasive" approach that would require to change the "client-side" code (the code that uses Foo objects) and it would be something that can be easily forgotten.

Answer 1

You can use monkey patching to achieve this. Re-assign one of the member functions on the object as a decorated function, which in turn calls the original function, along with some logging added.

For example:

a = Test() # An object you want to monitor
a.func() # A specific function of Test you want to decorate

# Your decorator
from functools import wraps
def addLogging(function):
    @wraps(function)
    def wrapper(*args, **kwargs):
        print 'Calling {}'.format(function.func_name)   
        return function(*args, **kwargs)
    return wrapper

a.func = addLogging(a.func)

Note, however, that monkey patching is best used only for unit testing, and not in production code. It can have unforeseen side-effects and should be used with caution.

As for identifying when a member variable's value changes, you can refer to this.

All of this does require you to modify the client side code -- if there's a way to achieve this without altering the client code, I don't know about it.

Answer 2

I can think of one solution, it isn't perfect but it's probably a start. We can capture instance attributes accesses via __getattribute__ and __setattribute__ in a class that will inherit from the decorated class:

import re

dunder_pattern = re.compile("__.*__")
protected_pattern = re.compile("_.*")

def is_hidden(attr_name):
    return dunder_pattern.match(attr_name) or protected_pattern.match(attr_name)


def attach_proxy(function=None):
    function = function or (lambda *a: None)

    def decorator(decorated_class):

        class Proxy(decorated_class):
            def __init__(self, *args, **kwargs):
                function("init", args, kwargs)
                super().__init__(*args, **kwargs)

            def __getattribute__(self, name):
                if not is_hidden(name):
                    function("acces", name)
                return object.__getattribute__(self, name)

            def __getattr__(self, name):
                if not is_hidden(name):
                    function("acces*", name)
                return object.__getattr__(self, name)

            def __setattribute__(self, name, value):
                if not is_hidden(name):
                    function("set", name, value)
                return object.__setattribute__(self, name, value)

            def __setattr__(self, name, value):
                if not is_hidden(name):
                    function("set*", name, value)
                return object.__setattr__(self, name, value)

        return Proxy

    return decorator

Which you can then use to decorate your class:

@attach_proxy(print)
class A:
    x = 1
    def __init__(self, y, msg="hello"):
        self.y = y

    @classmethod
    def foo(cls):
        print(cls.x)

    def bar(self):
        print(self.y)

Which will result in the following:

>>> a = A(10, msg="test")
init (10,) {'msg': 'test'}
set* y 10
>>> a.bar()
acces bar
acces y
10
>>> a.foo() # access to x is not captured
acces foo
1
>>> y = a.y
acces y
>>> x = A.x # access to x is not captured
>>> a.y = 3e5
set* y 300000.0

Problems:

1. Class attributes access are not captured (would need a metaclass for that but I don't see a way to do on the fly).

2. Type A is hidden (behind the type Proxy), this is probably simpler to solve:

>>> A
__main__.attach_proxy.<locals>.decorator.<locals>.Proxy

On the other hand that's not necessarily a problem as this will work as expected:

>>> a = A(10, msg="test")
>>> isinstance(a, A)
True

Edit note that I don't pass instances to function calls, but that would actually be a good idea, replacing calls from function("acces", name) to function("acces", self, name). That would allow to make much more funny things with your decorator.

Answer 3

You could combine the answer provided by @suicidalteddy with method inspection, resulting in something similar to the following:

# Your decorator
def add_logging(function):
    @wraps(function)
    def wrapper(*args, **kwargs):
        print 'Calling {}'.format(function.func_name)   
        return function(*args, **kwargs)
    return wrapper

instance = Test() # An object you want to monitor

# list of callables found in instance
methods_list = [
   method_name for method_name in dir(instance) if callable(
      getattr(instance, method_name)
   )
]

# replaces original method by decorated one
for method_name in methods_list:
    setattr(instance, method_name, add_logging(getattr(instance, method_name))

I haven't tested this but something similar to it should do the work, good luck!