The PACT for classes

Note

This version of the package was tagged as v.0.0.3 in the project repository, and can be examined there if desired.

The test-expectations and goals for members of classes are, fundamentally, identical to those for functions. Specifically:

Goal

1. Every method of a class should have a corresponding happy-paths test method, testing all permutations of a rational set of arguments across the parameters of the method.
2. Every method should also have a corresponding unhappy-path test-method for each parameter.

The main differentiator is not in what kinds of expectations are present, but in how those expectations are defined and applied to members of classes. With the exception of the main test-method (test_source_class_has_expected_test_methods), the implementation of the expected_test_entities and source_entiteis properties of the ExaminesSourceClass class, and the addition of an INVALID_DEL_SUFFIX used to identify test methods for invalid property and data-descriptor delete-methods, the relationships between the members of the ExaminesSourceClass class are pretty much identical to the members and relationships between them of the ExaminesSourceFunction class from the previous article. Diagrammed, those members’ relationships to each other are:

The key reason behind the differences between setting expectations between classes and functions is that classes have members, while functions do not. The members of classes that can be meaningfully tested to the extent that setting a testing expectation for them are limited to methods and data-descriptors. Methods, when it comes right down to it, are just functions that may have a common scope parameter: self for instance methods, indicating which object instance the method should act in relation to, or cls for methods that the @classmethod decorator has been applied to, indicating which class the method should act in relation to. There is also the @staticmethod decorator, which is used to attach a method to a class without either an instance or a class scope expectation. In all of these cases, the code for the method looks like a simple function, for example:

class MyClass:

    def instance_method(self):
        pass

    @classmethod
    def class_method(cls):
        pass

    @staticmethod
    def static_method():
        pass

The identification of any method in the target_class by the source_entities property uses the built-in inspect.getmembers function to find members, using the built-in callable function to detect callables, and inspect.isclass to filter out any callables that are classes rather than functions or methods. Similarly, the source_entities property uses inspect.getmembers in conjunction with the inspect module’s isdatadescriptor function to identify members defined as properties with the property decorator.

Warning

As I was writing this article, and doing the relevant review, I noticed a discrepancy between how source_entities behaves in comparison with expected_test_entities. I’m not sure if it is significant or not as I’m writing this, but since this article is being written about v.0.0.3 and the current repo version is v.0.0.6, it won’t really be addressed until v.0.0.7 if it is a problem.

Properties, and data descriptors in general, are objects attached to the classes they are members of, following an interface structure that is hinted at in the Pure Python Equivalents: Properties example of the Descriptor HowTo Guide. Descriptors have __get__, __set__ and __delete__ methods, and property objects augment that with fget, fset and fdel members that contain the getter-, setter-, and deleter-methods that are written in user-generated code like so:

class MyClass:

    @property
    def name(self):
        ...

    @name.setter
    def name(self, value):
        ...

    @name.deleter
    def name(self):
        ...

Because a data-descriptor may be a property or a custom descriptor type, and thus may or may not have the property-specific fget, fset and fdel members, the expectations for test-methods for properties and descriptors may not be able to reliably determine if the related actions need to be tested. That is, a property might be defined with a getter and setter, but no deleter, in which case its fdel member will be None. A non-property descriptor is expected to always have the __get__, __set__ and __delete__ member methods, but they might not be implemented, and there may not be a good way to make that distinction without simply requiring that all non-property descriptors test all three methods’ actions. The fact that all of those members and methods are, themselves, defined as methods still allows the same parameter/argument expectations to be determined: the example name property above could be expected to have test_name_happy_paths and test_set_name_bad_value test-methods, testing the happy-path set/get processes, and unhappy set-scenarios, respectively. It would also need a test_name_invalid_del test-method, assuming that the property has a deleter. If name were defined as a more generic data-descriptor.

Tip

There are other implementations that behave in ways similar to property and general descriptor objects that may not implement the descriptor interface. I know of one example offhand: the various Field types provided by Pydantic. Those will have to be handled with more specific functionality later, but that won’t be a consideration until the v.1.0.0 version of this package is complete.

With those properties added to the MyClass class, running the test-suite before adding any of the expected test-methods results in the expected failures:

================================================================
...
AssertionError: False is not true : 
  Missing expected test-method - test_class_method_happy_paths
================================================================
...
AssertionError: False is not true :
  Missing expected test-method - test_name_set_bad_value
================================================================
...
AssertionError: False is not true :
  Missing expected test-method - test_name_invalid_del
================================================================
...
AssertionError: False is not true :
  Missing expected test-method - test_instance_method_happy_paths
================================================================
...
AssertionError: False is not true :
  Missing expected test-method - test_static_method_happy_paths
================================================================
...
AssertionError: False is not true :
  Missing expected test-method - test_name_happy_paths
----------------------------------------------------------------

After adding all of the expected test-methods reported as missing, the example project’s structure looks like this:

project-name/
├─ Pipfile
├─ Pipfile.lock
├─ .env
├─ src/
│   └─ my_package/
│      └─ module.py
│         ├─ ::MyClass
│         │  ├─ ::instance_method() # instance method
│         │  ├─ ::name              # property
│         │  ├─ ::class_method()    # class method
│         │  └─ ::static_method()   # static method
│         └─ ::my_function()
└─ tests/
    └─ unit/
       └─ test_my_package/
          ├─ test_project_test_modules_exist.py
          │  └─ ::test_ProjectTestModulesExist
          └─ test_module.py
             ├─ ::test_MyClass
             │  │  # Property tests
             │  ├─ ::test_name_happy_paths
             │  ├─ ::test_name_invalid_del
             │  ├─ ::test_name_invalid_del
             │  │  # Method tests
             │  ├─ ::test_instance_method_happy_paths
             │  ├─ ::test_class_method_happy_paths
             │  └─ ::test_static_method_happy_paths
             └─ ::test_my_function

And, finally, the complete module_memberes class-diagram looks like this:

At this point, the package does everything that the most basic interpretation of my initial desires required: With adequate inclusion of the project- and module-level tests, and some short but relatively tedious manual effort to stub out the test-suite for the package itself, there are 108 test methods defined, 96 of which are still pending implementation. The results of the test-suite show that clearly:

===================== test session starts ======================
tests/unit/test_goblinfish/test_testing/test_pact/test_abcs.py 
                                              ..ssssssss  [  9%]
tests/unit/test_goblinfish/test_testing/test_pact/
    test_module_members.py 
           .ssss.sssssssssssssssssssss.ssssssssssssssss.  [ 50%]
tests/unit/test_goblinfish/test_testing/test_pact/
    test_modules.py              .ssss.sssssssssssssssss  [ 72%]
tests/unit/test_goblinfish/test_testing/test_pact/
    test_pact_logging.py                               .  [ 73%]
tests/unit/test_goblinfish/test_testing/test_pact/
    test_project_test_modules_exist.py                 .  [ 74%]
tests/unit/test_goblinfish/test_testing/test_pact/
    test_projects.py                ss.sssssssssssssssss  [100%]
================= 12 passed, 96 skipped in 0.05s ===============

Running a coverage report shows about what I’d expect at this point as well: a fair bit of the code is being called in most cases, but nowhere near what I’d like. The module_mambers.py missing-lines report is most of the source file, modules.py has 7 substantial chunks identified as currently untested, and projects.py has 4. Even just the simple percentages reported are a good indicator:

Name	Stmts	Miss	Cover
src/goblinfish/testing/pact/abcs.py	13	3	77%
src/goblinfish/testing/pact/module_members.py	131	129	2%
src/goblinfish/testing/pact/modules.py	74	44	41%
src/goblinfish/testing/pact/pact_logging.py	18	5	72%
src/goblinfish/testing/pact/projects.py	63	15	76%

Still, as I noted in the first article in this series, I hadn’t originally planned to get tests actually implemented until after this point had been reached. After the little bits of chaos that interfered with that original plan, which I’ll get into more detail about in the next article, actual test-implementations and the fixes that would come of that were deferred until v.0.0.7, which I’ll get into in the article after next.