ubelt.util_memoize module

This module exposes decorators for in-memory caching of functional results. This is particularly useful when prototyping dynamic programming algorithms.

Either memoize(), memoize_method(), and memoize_property() should be used depending on what type of function is being wrapped. The following example demonstrates this.

In Python 3.8+ memoize() works similarly to the standard library functools.cache(), but the ubelt version makes use of ubelt.util_hash.hash_data(), which is slower, but handles inputs containing mutable containers.

Example

>>> import ubelt as ub
>>> # Memoize a function, the args are hashed
>>> @ub.memoize
>>> def func(a, b):
>>>     return a + b
>>> #
>>> class MyClass:
>>>     # Memoize a class method, the args are hashed
>>>     @ub.memoize_method
>>>     def my_method(self, a, b):
>>>         return a + b
>>>     #
>>>     # Memoize a property: there can be no args,
>>>     @ub.memoize_property
>>>     @property
>>>     def my_property1(self):
>>>         return 4
>>>     #
>>>     # The property decorator is optional
>>>     def my_property2(self):
>>>         return 5
>>> #
>>> func(1, 2)
>>> func(1, 2)
>>> self = MyClass()
>>> self.my_method(1, 2)
>>> self.my_method(1, 2)
>>> self.my_property1
>>> self.my_property1
>>> self.my_property2
>>> self.my_property2
ubelt.util_memoize.memoize(func)[source]

memoization decorator that respects args and kwargs

In Python 3.9. The functools introduces the cache method, which is currently faster than memoize for simple functions [FunctoolsCache]. However, memoize can handle more general non-natively hashable inputs.

Parameters:

func (Callable) – live python function

Returns:

memoized wrapper

Return type:

Callable

References

Example

>>> import ubelt as ub
>>> closure = {'a': 'b', 'c': 'd'}
>>> incr = [0]
>>> def foo(key):
>>>     value = closure[key]
>>>     incr[0] += 1
>>>     return value
>>> foo_memo = ub.memoize(foo)
>>> assert foo('a') == 'b' and foo('c') == 'd'
>>> assert incr[0] == 2
>>> print('Call memoized version')
>>> assert foo_memo('a') == 'b' and foo_memo('c') == 'd'
>>> assert incr[0] == 4
>>> assert foo_memo('a') == 'b' and foo_memo('c') == 'd'
>>> print('Counter should no longer increase')
>>> assert incr[0] == 4
>>> print('Closure changes result without memoization')
>>> closure = {'a': 0, 'c': 1}
>>> assert foo('a') == 0 and foo('c') == 1
>>> assert incr[0] == 6
>>> assert foo_memo('a') == 'b' and foo_memo('c') == 'd'
class ubelt.util_memoize.memoize_method(func)[source]

Bases: object

memoization decorator for a method that respects args and kwargs

References

Variables:

__func__ (Callable) – the wrapped function

Note

This is very thread-unsafe, and has an issue as pointed out in [ActiveState_Miller_2010], next version may work on fixing this.

Example

>>> import ubelt as ub
>>> closure1 = closure = {'a': 'b', 'c': 'd', 'z': 'z1'}
>>> incr = [0]
>>> class Foo(object):
>>>     def __init__(self, instance_id):
>>>         self.instance_id = instance_id
>>>     @ub.memoize_method
>>>     def foo_memo(self, key):
>>>         "Wrapped foo_memo docstr"
>>>         value = closure[key]
>>>         incr[0] += 1
>>>         return value, self.instance_id
>>>     def foo(self, key):
>>>         value = closure[key]
>>>         incr[0] += 1
>>>         return value, self.instance_id
>>> self1 = Foo('F1')
>>> assert self1.foo('a') == ('b', 'F1')
>>> assert self1.foo('c') == ('d', 'F1')
>>> assert incr[0] == 2
>>> #
>>> print('Call memoized version')
>>> assert self1.foo_memo('a') == ('b', 'F1')
>>> assert self1.foo_memo('c') == ('d', 'F1')
>>> assert incr[0] == 4, 'should have called a function 4 times'
>>> #
>>> assert self1.foo_memo('a') == ('b', 'F1')
>>> assert self1.foo_memo('c') == ('d', 'F1')
>>> print('Counter should no longer increase')
>>> assert incr[0] == 4
>>> #
>>> print('Closure changes result without memoization')
>>> closure2 = closure = {'a': 0, 'c': 1, 'z': 'z2'}
>>> assert self1.foo('a') == (0, 'F1')
>>> assert self1.foo('c') == (1, 'F1')
>>> assert incr[0] == 6
>>> assert self1.foo_memo('a') == ('b', 'F1')
>>> assert self1.foo_memo('c') == ('d', 'F1')
>>> #
>>> print('Constructing a new object should get a new cache')
>>> self2 = Foo('F2')
>>> self2.foo_memo('a')
>>> assert incr[0] == 7
>>> self2.foo_memo('a')
>>> assert incr[0] == 7
>>> # Check that the decorator preserves the name and docstring
>>> assert self1.foo_memo.__doc__ == 'Wrapped foo_memo docstr'
>>> assert self1.foo_memo.__name__ == 'foo_memo'
>>> print(f'self1.foo_memo = {self1.foo_memo!r}, {hex(id(self1.foo_memo))}')
>>> print(f'self2.foo_memo = {self2.foo_memo!r}, {hex(id(self2.foo_memo))}')
>>> #
>>> # Test for the issue in the active state recipe
>>> method1 = self1.foo_memo
>>> method2 = self2.foo_memo
>>> assert method1('a') == ('b', 'F1')
>>> assert method2('a') == (0, 'F2')
>>> assert method1('z') == ('z2', 'F1')
>>> assert method2('z') == ('z2', 'F2')
Parameters:

func (Callable) – method to wrap

ubelt.util_memoize.memoize_property(fget)[source]

Return a property attribute for new-style classes that only calls its getter on the first access. The result is stored and on subsequent accesses is returned, preventing the need to call the getter any more.

This decorator can either be used by itself or by decorating another property. In either case the method will always become a property.

Note

implementation is a modified version of [estebistec_memoize].

References

Parameters:

fget (property | Callable) – A property or a method.

Example

>>> import ubelt as ub
>>> class C(object):
...     load_name_count = 0
...     @ub.memoize_property
...     def name(self):
...         "name's docstring"
...         self.load_name_count += 1
...         return "the name"
...     @ub.memoize_property
...     @property
...     def another_name(self):
...         "name's docstring"
...         self.load_name_count += 1
...         return "the name"
>>> c = C()
>>> c.load_name_count
0
>>> c.name
'the name'
>>> c.load_name_count
1
>>> c.name
'the name'
>>> c.load_name_count
1
>>> c.another_name