NML Says

Intro

Unittest

Pytest

Introduction to Testing

Before we start, a look ahead: The next lesson will be along the lines of a paedagogical style called flipped classroom. This basically means that at home, prior to the class, you study what would normally be taught in the classroom, and next time in the classroom we do what would normally be the class’ subsequent assignments/exercises. It is therefore imperative that you carefully and thoroughly work through the following youtube video:

PyTest • REST API Integration Testing with Python

But first things first, on to today’s work.

Test Driven Development

Background

Computer systems must be tested. Hardware, or as in our context, systems, software. This is about testing.

There are several test concepts in system engineering, development, but they all have in common that the test should ascertain that the software does what it is supposed to do. This begs the question: What is it supposed to do?

What is it supposed to do? You will find the answer to that in the specs. This means that there is a proces preceding writing code. The developer gets a system specification explaining in detail what the requirements are. There are usually also a number of use cases involved. Use cases are part of user scenarios.

Use cases are defined in UML, Unified Modelling Language1

Use Cases I

  • Requirements in a system entails:
    • A need of classes, specific classes.
    • A need of functionality, methods of the classes of the system.
  • Use cases define the interface to the system.
  • Use cases also define the entities of the system. Who needs it?

Use Cases II

  • A use case may be high level, quite abstract.
  • It might be very detailed.
  • It might be a diagram with few words.
  • It might be verbose.
  • It describes the what of the system.
  • The developer uses it to write the how in the form of code.

Use Cases III

  • The high level use case: Alt text

Use Cases IV

A use case could be verbose:

  1. Fido barks meaning “I wanna get out!”
  2. Todd or Gina hears Fido barking.
  3. Todd or Gina aktivate the remote control.
  4. The door opens, sesame, sesame …
  5. Fido exits.
  6. Fido does what must be done.
    • The door closes automatically
    • Fido barks to signal “I wanna come back inside!”
    • Todd or Gina hears Fido bark (again.)
    • One of them activates the remote control (again.)
    • The door reopens (again.)
  7. Fido enters.
  8. The door closes automatically.

Use Cases V

  • Point no 1 was a start condition.
  • Subsequently a series of steps illustrate the interface behavior.
  • The interface is between an actor and the system.
  • A use case ends with an end condition.
  • The detailed use case may be very concrete begging detailed code. The end condition must be testable.

Use Cases VI

  • A payment system example Alt text

Aspects of Testing

Let us look at some focal points of testing by referring to https://www.sitepoint.com/python-unit-testing-unittest-pytest/

We generally distinguish between different levels of software testing:

Unit testing
testing specific lines of code, functions, etc.
Integration testing
tests integration between units
System testing
tests the entire system.
Acceptance testing
checks the compliance with business goals as expressed in specs.

Generally speaking the former two are done by the developers, while the latter two involve testers and users.

Test Strategy

Apart from the professional pride of the developer, if that still exists, there are some overall limits that should be considered for a strategy. They include:

Risks
What are the business consequences if a bug were to affect this component?
Time
How soon must you deliver? Do you have a deadline?
Budget
How much money are you willing to spend on testing?

Risk, time, and budget are factors not just in software development, but are general project management topics.

Unit Test Characteristics

The main characteristics of unit tests are

Speed
Unit tests are mostly executed automatically, which means they must be fast. Slow unit tests are more likely to be skipped by developers because they don’t provide instant feedback.
Isolation
Unit tests are standalone by definition. Each test tests one thing. They don’t depend on anything external like a file or a network resource, if they depend on any other function, they are by definition an integration test.
Repeatable
Unit tests are executed repeatedly. They are contained in a program that you run.
Reliable
Unit tests themselves are tested before use. They may fail only if there’s a bug in the system being tested. They must be independent of environment, or the order of execution.
Naming
Proper naming is as essential in tests as in software development generally. The name of a test should be clear and unambiguous.

Test Pattern

Sitepoint advocates what they call the AAA pattern.

The Arrange, Act and Assert pattern is a common strategy used to write and organize unit tests. It works in the following way:

  • During the Arrange phase, all the objects and variables needed for the test are set.
  • Next, during the Act phase, the function/method/class under test is called.
  • In the end, during the Assert phase, we verify the outcome of the test.

This strategy provides a clean approach to organizing unit tests by separating all the main parts of a test: setup, execution and verification. Plus, unit tests are easier to read, because they all follow the same structure.

Now, let us proceed in more concrete direction.

TDD, Test Driven Development Overview

Computer systems must be tested. Hardware, or as in our context, systems, i.e. software. This section is about testing.

Wikipedia has the following about test-driven development. It cites Beck022 for this and for being the father of TDD. The quote has been cleaned from Wikipedia internal links. The Wikipedia article is very well referenced. Use the link above.

1. Add a test
The adding of a new feature begins by writing a test that passes if and only if the feature’s specifications are met. The developer can discover these specifications by asking about use cases and user stories. A key benefit of test-driven development is that it makes the developer focus on requirements before writing code. This is in contrast with the usual practice, where unit tests are only written after code.
2. Run all tests
The new test should fail for expected reasons This shows that new code is actually needed for the desired feature. It validates that the test harness is working correctly. It rules out the possibility that the new test is flawed and will always pass.
3.Write the simplest code that passes the new test
Inelegant or hard code is acceptable, as long as it passes the test. The code will be honed anyway in Step 5. No code should be added beyond the tested functionality.
4. All tests should now pass
If any fail, the new code must be revised until they pass. This ensures the new code meets the test requirements and does not break existing features.
5. Refactor as needed, using tests after each refactor
Code is refactored for readability and maintainability. In particular, hard-coded test data should be removed. Running the test suite after each refactor helps ensure that no existing functionality is broken.
  • Examples of refactoring:
    • moving code to where it most logically belongs
    • removing duplicate code
    • making names self-documenting
    • splitting methods into smaller pieces
    • re-arranging inheritance hierarchies
Repeat
The cycle above is repeated for each new piece of functionality. Tests should be small and incremental, and commits made often. That way, if new code fails some tests, the programmer can simply undo or revert rather than debug excessively. When using external libraries, it is important not to write tests that are so small as to effectively test merely the library itself, unless there is some reason to believe that the library is buggy or not feature-rich enough to serve all the needs of the software under development.

Unit Testing with unittest

Test Functions

Steps 1 and 2

Create the unittest and run it before there’s anything to test. First rule of TDD.

coursecode/pyth/project0/tests.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

import unittest
from lib0 import *

''' class must be subclass of unittest.TestCase 
    class name of your choice
    methods of the class are tests
'''
class Testing(unittest.TestCase):
    def test_add_numbers(self):                     # unittest, name arbitrary 
        self.assertEqual(add_numbers(2, 3), 5)
        self.assertEqual(add_numbers(5, 7), 12)

Execute the test by

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

$ python -m unittest tests.py
E
======================================================================
ERROR: test_add_numbers (tests.Testing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nml/public_html/courses/coursecode/pyth/project0/tests.py", line 6, in test_add_numbers
    self.assertEqual(add_numbers(2, 3), 5)
NameError: name 'add_numbers' is not defined

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Steps 3 and 4

Second rule of TDD:

coursecode/pyth/project0/lib0.py:
1
2

def add_numbers(a, b):
    return a + b

Execute the test by

1
2
3
4
5
6

$ python -m unittest tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Expand Test

Create the first unittest and run it.

coursecode/pyth/project1/tests.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

import unittest
from lib0 import *

''' class must be subclass of unittest.TestCase 
    class name of your choice
    methods of the class are tests
'''
class Testing(unittest.TestCase):
    def test_add_numbers101(self):                  # unittest, name arbitrary 
        self.assertEqual(add_numbers(2, 3), 5)
        self.assertEqual(add_numbers(5, 7), 12)
        
    def test_again(self):                           # next test
        self.assertEqual(add_numbers(15, -7), 8)
        self.assertEqual(add_numbers(-15, 7), -8)
        self.assertEqual(add_numbers(-35, -7), -42)

Execute the test by:

1
2
3
4
5
6

python -m unittest tests.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Unit Testing with unittest

Test a Class

First create the unittestand run it.

coursecode/pyth/project20/tests.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import unittest
from Person import *
from datetime import datetime

''' 
    Wrong number of arguments will automatically fail
    Testing logics in constructor
'''
class Testing(unittest.TestCase):
    def test_constructor(self):                     # unittest, name arbitrary 
        o = Person('firstname', 'lastname', datetime.now().year - 39)
        self.assertEqual(o.old, False, msg='Checking for not old failed') 
        o = Person('firstname', 'lastname', datetime.now().year - 40)
        self.assertEqual(o.old, True, msg='Checking for old failed')

Execute it with

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

python -m unittest tests.py
E
======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests
Traceback (most recent call last):
  File "/nix/store/xf54733x4chbawkh1qvy9i1i4mlscy1c-python3-3.10.11/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/home/nml/public_html/courses/coursecode/pyth/project20/tests.py", line 2, in <module>
    from Person import *
ModuleNotFoundError: No module named 'Person'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Then we write the class to test with its constructor:

coursecode/pyth/project21/Person.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# Person.py

from datetime import datetime

class Person:
    thehill = 39
    
    def __init__(self, first, last, yearofbirth):
        self.first = first
        self.last = last
        self.yearofbirth = yearofbirth
        self.old = False
        if datetime.now().year - self.yearofbirth > self.thehill:
            self.old = True

This is the minimal requirement to satisfy the test. Now test again:

1
2
3
4
5
6

python -m unittest tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Unit Testing with pytest

Test Functions

Steps 1-4, Demo

Create the pytest and the function to test.

coursecode/pyth/project4/test_sample.py:
1
2
3
4
5
6
7

# content of test_sample.py
def inc(x):
    return x + 1


def test_answer():
    assert inc(3) == 5

The execution of the test

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

pytest
============================= test session starts =============================
platform linux -- Python 3.10.11, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/nml/public_html/courses/coursecode/pyth/project4
collected 1 item                                                              

test_sample.py F                                                        [100%]

================================== FAILURES ===================================
_________________________________ test_answer _________________________________

    def test_answer():
>       assert inc(3) == 5
E       assert 4 == 5
E        +  where 4 = inc(3)

test_sample.py:7: AssertionError
=========================== short test summary info ===========================
FAILED test_sample.py::test_answer - assert 4 == 5
============================== 1 failed in 0.02s ==============================

Expand a Test

Again, create a pytest and run it.

coursecode/pyth/project41/test_sample.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# content of test_sample.py
def inc(x):
    return x + 1


def test_answer():
    assert inc(3) == 4

def test_answer1():
    assert inc(-1) == 0

def test_answer2():
    assert inc(9) == 9

Execution

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

pytest
============================= test session starts =============================
platform linux -- Python 3.10.11, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/nml/public_html/courses/coursecode/pyth/project41
collected 3 items                                                             

test_sample.py ..F                                                      [100%]

================================== FAILURES ===================================
________________________________ test_answer2 _________________________________

    def test_answer2():
>       assert inc(9) == 9
E       assert 10 == 9
E        +  where 10 = inc(9)

test_sample.py:13: AssertionError
=========================== short test summary info ===========================
FAILED test_sample.py::test_answer2 - assert 10 == 9
========================= 1 failed, 2 passed in 0.02s =========================

Test a Class

Steps 1 and 2

Create the pytest and run it before there’s anything to test. First rule of TDD.

coursecode/pyth/project50/tests/test_distance.py:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# test_distance.py, the 'test_' makes it a test to pytest

from Point import Point

def test_distance():
    # Arrange
    a = Point(100, 100)
    b = Point(400, 500)

    # Act
    dist = a.distance(b)

    # Assert
    assert dist == 500

Execute:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

pytest
============================= test session starts =============================
platform linux -- Python 3.10.11, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/nml/public_html/courses/coursecode/pyth/project50
collected 0 items / 1 error                                                   

=================================== ERRORS ====================================
___________________ ERROR collecting tests/test_distance.py ___________________
ImportError while importing test module '/home/nml/public_html/courses/coursecode/pyth/project50/tests/test_distance.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/lwzzgbnj41d657lpxczk6l5f7d5zcnj1-python3-3.10.11/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_distance.py:3: in <module>
    from Point import Point
E   ModuleNotFoundError: No module named 'Point'
=========================== short test summary info ===========================
ERROR tests/test_distance.py
!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
============================== 1 error in 0.07s ===============================

Steps 3 and 4

Second rule of TDD. Make just enough to satisfy tests.

coursecode/pyth/project51/Point.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

# Point.py
import math

class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __repr__(self):
        return "[{}, {}]".format(self.x, self.y)

    def distance(self, o):
        return math.sqrt((self.x - o.x)**2 + (self.y - o.y)**2)


if __name__ == "__main__":
    a = Point(100, 100)
    print(a)
    b = Point(400, 500)
    print(b)
    print(a.distance(b))

Execute from relevant directory

1
2
3
4
5
6
7
8
9

pytest
============================= test session starts =============================
platform linux -- Python 3.10.11, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/nml/public_html/courses/coursecode/pyth/project51
collected 1 item                                                              

tests/test_distance.py .                                                [100%]

============================== 1 passed in 0.01s ==============================

Expand the Test

Improving/maintaining software calls for additional tests. We add a test

coursecode/pyth/project52/test1/test_2.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# test_distance.py, the 'test_' makes it a test to pytest

from Point import Point

def test_2():
    # Arrange
    a = Point(100, 100)

    # Act


    # Assert
    assert str(a) == "[100, 100]"

Execute from the CLI by

1
2
3
4
5
6
7
8
9

pytest
============================= test session starts =============================
platform linux -- Python 3.10.11, pytest-7.2.1, pluggy-1.0.0
rootdir: /home/nml/public_html/courses/coursecode/pyth/project52
collected 2 items                                                             

tests/test_2.py .                                                       [ 50%]
tests/test_distance.py .                                                [100%]
============================== 2 passed in 0.01s ==============================

Do Not Forget

to watch the video before continuing your study of this material.

  1. http://en.m.wikipedia.org/wiki/Unified_Modeling_Language ↩︎

  2. Kent Beck. Test-driven Development by Example. Vaseem: Addison-Wesley, 2002 ↩︎