unittest.rst

:mod:`unittest` --- Unit testing framework
==========================================

.. module:: unittest
   :synopsis: Unit testing framework for Python.

.. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
.. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
.. sectionauthor:: Raymond Hettinger <python@rcn.com>

**Source code:** :source:`Lib/unittest/__init__.py`

--------------

(If you are already familiar with the basic concepts of testing, you might want
to skip to :ref:`the list of assert methods <assert-methods>`.)

The :mod:`unittest` unit testing framework was originally inspired by JUnit
and has a similar flavor as major unit testing frameworks in other
languages.  It supports test automation, sharing of setup and shutdown code
for tests, aggregation of tests into collections, and independence of the
tests from the reporting framework.

To achieve this, :mod:`unittest` supports some important concepts in an
object-oriented way:

test fixture
   A :dfn:`test fixture` represents the preparation needed to perform one or more
   tests, and any associated cleanup actions.  This may involve, for example,
   creating temporary or proxy databases, directories, or starting a server
   process.

test case
   A :dfn:`test case` is the individual unit of testing.  It checks for a specific
   response to a particular set of inputs.  :mod:`unittest` provides a base class,
   :class:`TestCase`, which may be used to create new test cases.

test suite
   A :dfn:`test suite` is a collection of test cases, test suites, or both.  It is
   used to aggregate tests that should be executed together.

test runner
   A :dfn:`test runner` is a component which orchestrates the execution of tests
   and provides the outcome to the user.  The runner may use a graphical interface,
   a textual interface, or return a special value to indicate the results of
   executing the tests.


.. seealso::

   Module :mod:`doctest`
      Another test-support module with a very different flavor.

   `Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_
      Kent Beck's original paper on testing frameworks using the pattern shared
      by :mod:`unittest`.

   `pytest <https://docs.pytest.org/>`_
      Third-party unittest framework with a lighter-weight syntax for writing
      tests.  For example, ``assert func(10) == 42``.

   `The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_
      An extensive list of Python testing tools including functional testing
      frameworks and mock object libraries.

   `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_
      A special-interest-group for discussion of testing, and testing tools,
      in Python.

   The script :file:`Tools/unittestgui/unittestgui.py` in the Python source distribution is
   a GUI tool for test discovery and execution.  This is intended largely for ease of use
   for those new to unit testing.  For production environments it is
   recommended that tests be driven by a continuous integration system such as
   `Buildbot <https://buildbot.net/>`_, `Jenkins <https://jenkins.io/>`_,
   `GitHub Actions <https://github.com/features/actions>`_, or
   `AppVeyor <https://www.appveyor.com/>`_.


.. _unittest-minimal-example:

Basic example
-------------

The :mod:`unittest` module provides a rich set of tools for constructing and
running tests.  This section demonstrates that a small subset of the tools
suffice to meet the needs of most users.

Here is a short script to test three string methods::

  import unittest

  class TestStringMethods(unittest.TestCase):

      def test_upper(self):
          self.assertEqual('foo'.upper(), 'FOO')

      def test_isupper(self):
          self.assertTrue('FOO'.isupper())
          self.assertFalse('Foo'.isupper())

      def test_split(self):
          s = 'hello world'
          self.assertEqual(s.split(), ['hello', 'world'])
          # check that s.split fails when the separator is not a string
          with self.assertRaises(TypeError):
              s.split(2)

  if __name__ == '__main__':
      unittest.main()


A testcase is created by subclassing :class:`unittest.TestCase`.  The three
individual tests are defined with methods whose names start with the letters
``test``.  This naming convention informs the test runner about which methods
represent tests.

The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse`
to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a
specific exception gets raised.  These methods are used instead of the
:keyword:`assert` statement so the test runner can accumulate all test results
and produce a report.

The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you
to define instructions that will be executed before and after each test method.
They are covered in more detail in the section :ref:`organizing-tests`.

The final block shows a simple way to run the tests. :func:`unittest.main`
provides a command-line interface to the test script.  When run from the command
line, the above script produces an output that looks like this::

   ...
   ----------------------------------------------------------------------
   Ran 3 tests in 0.000s

   OK

Passing the ``-v`` option to your test script will instruct :func:`unittest.main`
to enable a higher level of verbosity, and produce the following output::

   test_isupper (__main__.TestStringMethods.test_isupper) ... ok
   test_split (__main__.TestStringMethods.test_split) ... ok
   test_upper (__main__.TestStringMethods.test_upper) ... ok

   ----------------------------------------------------------------------
   Ran 3 tests in 0.001s

   OK

The above examples show the most commonly used :mod:`unittest` features which
are sufficient to meet many everyday testing needs.  The remainder of the
documentation explores the full feature set from first principles.

.. versionchanged:: 3.11
   The behavior of returning a value from a test method (other than the default
   ``None`` value), is now deprecated.


.. _unittest-command-line-interface:

Command-Line Interface
----------------------

The unittest module can be used from the command line to run tests from
modules, classes or even individual test methods::

   python -m unittest test_module1 test_module2
   python -m unittest test_module.TestClass
   python -m unittest test_module.TestClass.test_method

You can pass in a list with any combination of module names, and fully
qualified class or method names.

Test modules can be specified by file path as well::

   python -m unittest tests/test_something.py

This allows you to use the shell filename completion to specify the test module.
The file specified must still be importable as a module. The path is converted
to a module name by removing the '.py' and converting path separators into '.'.
If you want to execute a test file that isn't importable as a module you should
execute the file directly instead.

You can run tests with more detail (higher verbosity) by passing in the -v flag::

   python -m unittest -v test_module

When executed without arguments :ref:`unittest-test-discovery` is started::

   python -m unittest

For a list of all the command-line options::

   python -m unittest -h

.. versionchanged:: 3.2
   In earlier versions it was only possible to run individual test methods and
   not modules or classes.


Command-line options
~~~~~~~~~~~~~~~~~~~~

:program:`unittest` supports these command-line options:

.. program:: unittest

.. cmdoption:: -b, --buffer

   The standard output and standard error streams are buffered during the test
   run. Output during a passing test is discarded. Output is echoed normally
   on test fail or error and is added to the failure messages.

.. cmdoption:: -c, --catch

   :kbd:`Control-C` during the test run waits for the current test to end and then
   reports all the results so far. A second :kbd:`Control-C` raises the normal
   :exc:`KeyboardInterrupt` exception.

   See `Signal Handling`_ for the functions that provide this functionality.

.. cmdoption:: -f, --failfast

   Stop the test run on the first error or failure.

.. cmdoption:: -k

   Only run test methods and classes that match the pattern or substring.
   This option may be used multiple times, in which case all test cases that
   match any of the given patterns are included.

   Patterns that contain a wildcard character (``*``) are matched against the
   test name using :meth:`fnmatch.fnmatchcase`; otherwise simple case-sensitive
   substring matching is used.

   Patterns are matched against the fully qualified test method name as
   imported by the test loader.

   For example, ``-k foo`` matches ``foo_tests.SomeTest.test_something``,
   ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``.

.. cmdoption:: --locals

   Show local variables in tracebacks.

.. versionadded:: 3.2
   The command-line options ``-b``, ``-c`` and ``-f`` were added.

.. versionadded:: 3.5
   The command-line option ``--locals``.

.. versionadded:: 3.7
   The command-line option ``-k``.

The command line can also be used for test discovery, for running all of the
tests in a project or just a subset.


.. _unittest-test-discovery:

Test Discovery
--------------

.. versionadded:: 3.2

Unittest supports simple test discovery. In order to be compatible with test
discovery, all of the test files must be :ref:`modules <tut-modules>` or
:ref:`packages <tut-packages>` importable from the top-level directory of
the project (this means that their filenames must be valid :ref:`identifiers
<identifiers>`).

Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
used from the command line. The basic command-line usage is::

   cd project_directory
   python -m unittest discover

.. note::

   As a shortcut, ``python -m unittest`` is the equivalent of
   ``python -m unittest discover``. If you want to pass arguments to test
   discovery the ``discover`` sub-command must be used explicitly.

The ``discover`` sub-command has the following options:

.. program:: unittest discover

.. cmdoption:: -v, --verbose

   Verbose output

.. cmdoption:: -s, --start-directory directory

   Directory to start discovery (``.`` default)

.. cmdoption:: -p, --pattern pattern

   Pattern to match test files (``test*.py`` default)

.. cmdoption:: -t, --top-level-directory directory

   Top level directory of project (defaults to start directory)

The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in
as positional arguments in that order. The following two command lines
are equivalent::

   python -m unittest discover -s project_directory -p "*_test.py"
   python -m unittest discover project_directory "*_test.py"

As well as being a path it is possible to pass a package name, for example
``myproject.subpackage.test``, as the start directory. The package name you
supply will then be imported and its location on the filesystem will be used
as the start directory.

.. caution::

    Test discovery loads tests by importing them. Once test discovery has found
    all the test files from the start directory you specify it turns the paths
    into package names to import. For example :file:`foo/bar/baz.py` will be
    imported as ``foo.bar.baz``.

    If you have a package installed globally and attempt test discovery on
    a different copy of the package then the import *could* happen from the
    wrong place. If this happens test discovery will warn you and exit.

    If you supply the start directory as a package name rather than a
    path to a directory then discover assumes that whichever location it
    imports from is the location you intended, so you will not get the
    warning.

Test modules and packages can customize test loading and discovery by through
the `load_tests protocol`_.

.. versionchanged:: 3.4
   Test discovery supports :term:`namespace packages <namespace package>`
   for the start directory. Note that you need to specify the top level
   directory too (e.g.
   ``python -m unittest discover -s root/namespace -t root``).

.. versionchanged:: 3.11
   Python 3.11 dropped the :term:`namespace packages <namespace package>`
   support. It has been broken since Python 3.7. Start directory and
   subdirectories containing tests must be regular package that have
   ``__init__.py`` file.

   Directories containing start directory still can be a namespace package.
   In this case, you need to specify start directory as dotted package name,
   and target directory explicitly. For example::

      # proj/  <-- current directory
      #   namespace/
      #     mypkg/
      #       __init__.py
      #       test_mypkg.py

      python -m unittest discover -s namespace.mypkg -t .


.. _organizing-tests:

Organizing test code
--------------------

The basic building blocks of unit testing are :dfn:`test cases` --- single
scenarios that must be set up and checked for correctness.  In :mod:`unittest`,
test cases are represented by :class:`unittest.TestCase` instances.
To make your own test cases you must write subclasses of
:class:`TestCase` or use :class:`FunctionTestCase`.

The testing code of a :class:`TestCase` instance should be entirely self
contained, such that it can be run either in isolation or in arbitrary
combination with any number of other test cases.

The simplest :class:`TestCase` subclass will simply implement a test method
(i.e. a method whose name starts with ``test``) in order to perform specific
testing code::

   import unittest

   class DefaultWidgetSizeTestCase(unittest.TestCase):
       def test_default_widget_size(self):
           widget = Widget('The widget')
           self.assertEqual(widget.size(), (50, 50))

Note that in order to test something, we use one of the :meth:`assert\*`
methods provided by the :class:`TestCase` base class.  If the test fails, an
exception will be raised with an explanatory message, and :mod:`unittest`
will identify the test case as a :dfn:`failure`.  Any other exceptions will be
treated as :dfn:`errors`.

Tests can be numerous, and their set-up can be repetitive.  Luckily, we
can factor out set-up code by implementing a method called
:meth:`~TestCase.setUp`, which the testing framework will automatically
call for every single test we run::

   import unittest

   class WidgetTestCase(unittest.TestCase):
       def setUp(self):
           self.widget = Widget('The widget')

       def test_default_widget_size(self):
           self.assertEqual(self.widget.size(), (50,50),
                            'incorrect default size')

       def test_widget_resize(self):
           self.widget.resize(100,150)
           self.assertEqual(self.widget.size(), (100,150),
                            'wrong size after resize')

.. note::
   The order in which the various tests will be run is determined
   by sorting the test method names with respect to the built-in
   ordering for strings.

If the :meth:`~TestCase.setUp` method raises an exception while the test is
running, the framework will consider the test to have suffered an error, and
the test method will not be executed.

Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
after the test method has been run::

   import unittest

   class WidgetTestCase(unittest.TestCase):
       def setUp(self):
           self.widget = Widget('The widget')

       def tearDown(self):
           self.widget.dispose()

If :meth:`~TestCase.setUp` succeeded, :meth:`~TestCase.tearDown` will be
run whether the test method succeeded or not.

Such a working environment for the testing code is called a
:dfn:`test fixture`.  A new TestCase instance is created as a unique
test fixture used to execute each individual test method.  Thus
:meth:`~TestCase.setUp`, :meth:`~TestCase.tearDown`, and :meth:`~TestCase.__init__`
will be called once per test.

It is recommended that you use TestCase implementations to group tests together
according to the features they test.  :mod:`unittest` provides a mechanism for
this: the :dfn:`test suite`, represented by :mod:`unittest`'s
:class:`TestSuite` class.  In most cases, calling :func:`unittest.main` will do
the right thing and collect all the module's test cases for you and execute
them.

However, should you want to customize the building of your test suite,
you can do it yourself::

   def suite():
       suite = unittest.TestSuite()
       suite.addTest(WidgetTestCase('test_default_widget_size'))
       suite.addTest(WidgetTestCase('test_widget_resize'))
       return suite

   if __name__ == '__main__':
       runner = unittest.TextTestRunner()
       runner.run(suite())

You can place the definitions of test cases and test suites in the same modules
as the code they are to test (such as :file:`widget.py`), but there are several
advantages to placing the test code in a separate module, such as
:file:`test_widget.py`:

* The test module can be run standalone from the command line.

* The test code can more easily be separated from shipped code.

* There is less temptation to change test code to fit the code it tests without
  a good reason.

* Test code should be modified much less frequently than the code it tests.

* Tested code can be refactored more easily.

* Tests for modules written in C must be in separate modules anyway, so why not
  be consistent?

* If the testing strategy changes, there is no need to change the source code.


.. _legacy-unit-tests:

Re-using old test code
----------------------

Some users will find that they have existing test code that they would like to
run from :mod:`unittest`, without converting every old test function to a
:class:`TestCase` subclass.

For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
This subclass of :class:`TestCase` can be used to wrap an existing test
function.  Set-up and tear-down functions can also be provided.

Given the following test function::

   def testSomething():
       something = makeSomething()
       assert something.name is not None
       # ...

one can create an equivalent test case instance as follows, with optional
set-up and tear-down methods::

   testcase = unittest.FunctionTestCase(testSomething,
                                        setUp=makeSomethingDB,
                                        tearDown=deleteSomethingDB)

.. note::

   Even though :class:`FunctionTestCase` can be used to quickly convert an
   existing test base over to a :mod:`unittest`\ -based system, this approach is
   not recommended.  Taking the time to set up proper :class:`TestCase`
   subclasses will make future test refactorings infinitely easier.

In some cases, the existing tests may have been written using the :mod:`doctest`
module.  If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
automatically build :class:`unittest.TestSuite` instances from the existing
:mod:`doctest`\ -based tests.


.. _unittest-skipping:

Skipping tests and expected failures
------------------------------------

.. versionadded:: 3.1

Unittest supports skipping individual test methods and even whole classes of
tests.  In addition, it supports marking a test as an "expected failure," a test
that is broken and will fail, but shouldn't be counted as a failure on a
:class:`TestResult`.

Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
or one of its conditional variants, calling :meth:`TestCase.skipTest` within a
:meth:`~TestCase.setUp` or test method, or raising :exc:`SkipTest` directly.

Basic skipping looks like this::

   class MyTestCase(unittest.TestCase):

       @unittest.skip("demonstrating skipping")
       def test_nothing(self):
           self.fail("shouldn't happen")

       @unittest.skipIf(mylib.__version__ < (1, 3),
                        "not supported in this library version")
       def test_format(self):
           # Tests that work for only a certain version of the library.
           pass

       @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
       def test_windows_support(self):
           # windows specific testing code
           pass

       def test_maybe_skipped(self):
           if not external_resource_available():
               self.skipTest("external resource not available")
           # test code that depends on the external resource
           pass

This is the output of running the example above in verbose mode::

   test_format (__main__.MyTestCase.test_format) ... skipped 'not supported in this library version'
   test_nothing (__main__.MyTestCase.test_nothing) ... skipped 'demonstrating skipping'
   test_maybe_skipped (__main__.MyTestCase.test_maybe_skipped) ... skipped 'external resource not available'
   test_windows_support (__main__.MyTestCase.test_windows_support) ... skipped 'requires Windows'

   ----------------------------------------------------------------------
   Ran 4 tests in 0.005s

   OK (skipped=4)

Classes can be skipped just like methods::

   @unittest.skip("showing class skipping")
   class MySkippedTestCase(unittest.TestCase):
       def test_not_run(self):
           pass

:meth:`TestCase.setUp` can also skip the test.  This is useful when a resource
that needs to be set up is not available.

Expected failures use the :func:`expectedFailure` decorator. ::

   class ExpectedFailureTestCase(unittest.TestCase):
       @unittest.expectedFailure
       def test_fail(self):
           self.assertEqual(1, 0, "broken")

It's easy to roll your own skipping decorators by making a decorator that calls
:func:`skip` on the test when it wants it to be skipped.  This decorator skips
the test unless the passed object has a certain attribute::

   def skipUnlessHasattr(obj, attr):
       if hasattr(obj, attr):
           return lambda func: func
       return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))

The following decorators and exception implement test skipping and expected failures:

.. decorator:: skip(reason)

   Unconditionally skip the decorated test.  *reason* should describe why the
   test is being skipped.

.. decorator:: skipIf(condition, reason)

   Skip the decorated test if *condition* is true.

.. decorator:: skipUnless(condition, reason)

   Skip the decorated test unless *condition* is true.

.. decorator:: expectedFailure

   Mark the test as an expected failure or error.  If the test fails or errors
   in the test function itself (rather than in one of the :dfn:`test fixture`
   methods) then it will be considered a success.  If the test passes, it will
   be considered a failure.

.. exception:: SkipTest(reason)

   This exception is raised to skip a test.

   Usually you can use :meth:`TestCase.skipTest` or one of the skipping
   decorators instead of raising this directly.

Skipped tests will not have :meth:`~TestCase.setUp` or :meth:`~TestCase.tearDown` run around them.
Skipped classes will not have :meth:`~TestCase.setUpClass` or :meth:`~TestCase.tearDownClass` run.
Skipped modules will not have :func:`setUpModule` or :func:`tearDownModule` run.


.. _subtests:

Distinguishing test iterations using subtests
---------------------------------------------

.. versionadded:: 3.4

When there are very small differences among your tests, for
instance some parameters, unittest allows you to distinguish them inside
the body of a test method using the :meth:`~TestCase.subTest` context manager.

For example, the following test::

   class NumbersTest(unittest.TestCase):

       def test_even(self):
           """
           Test that numbers between 0 and 5 are all even.
           """
           for i in range(0, 6):
               with self.subTest(i=i):
                   self.assertEqual(i % 2, 0)

will produce the following output::

   ======================================================================
   FAIL: test_even (__main__.NumbersTest.test_even) (i=1)
   Test that numbers between 0 and 5 are all even.
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "subtests.py", line 11, in test_even
       self.assertEqual(i % 2, 0)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
   AssertionError: 1 != 0

   ======================================================================
   FAIL: test_even (__main__.NumbersTest.test_even) (i=3)
   Test that numbers between 0 and 5 are all even.
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "subtests.py", line 11, in test_even
       self.assertEqual(i % 2, 0)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
   AssertionError: 1 != 0

   ======================================================================
   FAIL: test_even (__main__.NumbersTest.test_even) (i=5)
   Test that numbers between 0 and 5 are all even.
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "subtests.py", line 11, in test_even
       self.assertEqual(i % 2, 0)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^
   AssertionError: 1 != 0

Without using a subtest, execution would stop after the first failure,
and the error would be less easy to diagnose because the value of ``i``
wouldn't be displayed::

   ======================================================================
   FAIL: test_even (__main__.NumbersTest.test_even)
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "subtests.py", line 32, in test_even
       self.assertEqual(i % 2, 0)
   AssertionError: 1 != 0


.. _unittest-contents:

Classes and functions
---------------------

This section describes in depth the API of :mod:`unittest`.


.. _testcase-objects:

Test cases
~~~~~~~~~~

.. class:: TestCase(methodName='runTest')

   Instances of the :class:`TestCase` class represent the logical test units
   in the :mod:`unittest` universe.  This class is intended to be used as a base
   class, with specific tests being implemented by concrete subclasses.  This class
   implements the interface needed by the test runner to allow it to drive the
   tests, and methods that the test code can use to check for and report various
   kinds of failure.

   Each instance of :class:`TestCase` will run a single base method: the method
   named *methodName*.
   In most uses of :class:`TestCase`, you will neither change
   the *methodName* nor reimplement the default ``runTest()`` method.

   .. versionchanged:: 3.2
      :class:`TestCase` can be instantiated successfully without providing a
      *methodName*. This makes it easier to experiment with :class:`TestCase`
      from the interactive interpreter.

   :class:`TestCase` instances provide three groups of methods: one group used
   to run the test, another used by the test implementation to check conditions
   and report failures, and some inquiry methods allowing information about the
   test itself to be gathered.

   Methods in the first group (running the test) are:

   .. method:: setUp()

      Method called to prepare the test fixture.  This is called immediately
      before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`,
      any exception raised by this method will be considered an error rather than
      a test failure. The default implementation does nothing.


   .. method:: tearDown()

      Method called immediately after the test method has been called and the
      result recorded.  This is called even if the test method raised an
      exception, so the implementation in subclasses may need to be particularly
      careful about checking internal state.  Any exception, other than
      :exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be
      considered an additional error rather than a test failure (thus increasing
      the total number of reported errors). This method will only be called if
      the :meth:`setUp` succeeds, regardless of the outcome of the test method.
      The default implementation does nothing.


   .. method:: setUpClass()

      A class method called before tests in an individual class are run.
      ``setUpClass`` is called with the class as the only argument
      and must be decorated as a :func:`classmethod`::

        @classmethod
        def setUpClass(cls):
            ...

      See `Class and Module Fixtures`_ for more details.

      .. versionadded:: 3.2


   .. method:: tearDownClass()

      A class method called after tests in an individual class have run.
      ``tearDownClass`` is called with the class as the only argument
      and must be decorated as a :meth:`classmethod`::

        @classmethod
        def tearDownClass(cls):
            ...

      See `Class and Module Fixtures`_ for more details.

      .. versionadded:: 3.2


   .. method:: run(result=None)

      Run the test, collecting the result into the :class:`TestResult` object
      passed as *result*.  If *result* is omitted or ``None``, a temporary
      result object is created (by calling the :meth:`defaultTestResult`
      method) and used. The result object is returned to :meth:`run`'s
      caller.

      The same effect may be had by simply calling the :class:`TestCase`
      instance.

      .. versionchanged:: 3.3
         Previous versions of ``run`` did not return the result. Neither did
         calling an instance.

   .. method:: skipTest(reason)

      Calling this during a test method or :meth:`setUp` skips the current
      test.  See :ref:`unittest-skipping` for more information.

      .. versionadded:: 3.1


   .. method:: subTest(msg=None, **params)

      Return a context manager which executes the enclosed code block as a
      subtest.  *msg* and *params* are optional, arbitrary values which are
      displayed whenever a subtest fails, allowing you to identify them
      clearly.

      A test case can contain any number of subtest declarations, and
      they can be arbitrarily nested.

      See :ref:`subtests` for more information.

      .. versionadded:: 3.4


   .. method:: debug()

      Run the test without collecting the result.  This allows exceptions raised
      by the test to be propagated to the caller, and can be used to support
      running tests under a debugger.

   .. _assert-methods:

   The :class:`TestCase` class provides several assert methods to check for and
   report failures.  The following table lists the most commonly used methods
   (see the tables below for more assert methods):

   +-----------------------------------------+-----------------------------+---------------+
   | Method                                  | Checks that                 | New in        |
   +=========================================+=============================+===============+
   | :meth:`assertEqual(a, b)                | ``a == b``                  |               |
   | <TestCase.assertEqual>`                 |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertNotEqual(a, b)             | ``a != b``                  |               |
   | <TestCase.assertNotEqual>`              |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertTrue(x)                    | ``bool(x) is True``         |               |
   | <TestCase.assertTrue>`                  |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertFalse(x)                   | ``bool(x) is False``        |               |
   | <TestCase.assertFalse>`                 |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIs(a, b)                   | ``a is b``                  | 3.1           |
   | <TestCase.assertIs>`                    |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIsNot(a, b)                | ``a is not b``              | 3.1           |
   | <TestCase.assertIsNot>`                 |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIsNone(x)                  | ``x is None``               | 3.1           |
   | <TestCase.assertIsNone>`                |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIsNotNone(x)               | ``x is not None``           | 3.1           |
   | <TestCase.assertIsNotNone>`             |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIn(a, b)                   | ``a in b``                  | 3.1           |
   | <TestCase.assertIn>`                    |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertNotIn(a, b)                | ``a not in b``              | 3.1           |
   | <TestCase.assertNotIn>`                 |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertIsInstance(a, b)           | ``isinstance(a, b)``        | 3.2           |
   | <TestCase.assertIsInstance>`            |                             |               |
   +-----------------------------------------+-----------------------------+---------------+
   | :meth:`assertNotIsInstance(a, b)        | ``not isinstance(a, b)``    | 3.2           |
   | <TestCase.assertNotIsInstance>`         |                             |               |
   +-----------------------------------------+-----------------------------+---------------+

   All the assert methods accept a *msg* argument that, if specified, is used
   as the error message on failure (see also :data:`longMessage`).
   Note that the *msg* keyword argument can be passed to :meth:`assertRaises`,
   :meth:`assertRaisesRegex`, :meth:`assertWarns`, :meth:`assertWarnsRegex`
   only when they are used as a context manager.

   .. method:: assertEqual(first, second, msg=None)

      Test that *first* and *second* are equal.  If the values do not
      compare equal, the test will fail.

      In addition, if *first* and *second* are the exact same type and one of
      list, tuple, dict, set, frozenset or str or any type that a subclass
      registers with :meth:`addTypeEqualityFunc` the type-specific equality
      function will be called in order to generate a more useful default
      error message (see also the :ref:`list of type-specific methods
      <type-specific-methods>`).

      .. versionchanged:: 3.1
         Added the automatic calling of type-specific equality function.

      .. versionchanged:: 3.2
         :meth:`assertMultiLineEqual` added as the default type equality
         function for comparing strings.


   .. method:: assertNotEqual(first, second, msg=None)

      Test that *first* and *second* are not equal.  If the values do
      compare equal, the test will fail.

   .. method:: assertTrue(expr, msg=None)
               assertFalse(expr, msg=None)

      Test that *expr* is true (or false).

      Note that this is equivalent to ``bool(expr) is True`` and not to ``expr
      is True`` (use ``assertIs(expr, True)`` for the latter).  This method
      should also be avoided when more specific methods are available (e.g.
      ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they
      provide a better error message in case of failure.


   .. method:: assertIs(first, second, msg=None)
               assertIsNot(first, second, msg=None)

      Test that *first* and *second* are (or are not) the same object.

      .. versionadded:: 3.1


   .. method:: assertIsNone(expr, msg=None)
               assertIsNotNone(expr, msg=None)

      Test that *expr* is (or is not) ``None``.

      .. versionadded:: 3.1


   .. method:: assertIn(member, container, msg=None)
               assertNotIn(member, container, msg=None)

      Test that *member* is (or is not) in *container*.

      .. versionadded:: 3.1


   .. method:: assertIsInstance(obj, cls, msg=None)
               assertNotIsInstance(obj, cls, msg=None)

      Test that *obj* is (or is not) an instance of *cls* (which can be a
      class or a tuple of classes, as supported by :func:`isinstance`).
      To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.

      .. versionadded:: 3.2



   It is also possible to check the production of exceptions, warnings, and
   log messages using the following methods:

   +---------------------------------------------------------+--------------------------------------+------------+
   | Method                                                  | Checks that                          | New in     |
   +=========================================================+======================================+============+
   | :meth:`assertRaises(exc, fun, *args, **kwds)            | ``fun(*args, **kwds)`` raises *exc*  |            |
   | <TestCase.assertRaises>`                                |                                      |            |
   +---------------------------------------------------------+--------------------------------------+------------+
   | :meth:`assertRaisesRegex(exc, r, fun, *args, **kwds)    | ``fun(*args, **kwds)`` raises *exc*  | 3.1        |
   | <TestCase.assertRaisesRegex>`                           | and the message matches regex *r*    |            |
   +---------------------------------------------------------+--------------------------------------+------------+
   | :meth:`assertWarns(warn, fun, *args, **kwds)            | ``fun(*args, **kwds)`` raises *warn* | 3.2        |
   | <TestCase.assertWarns>`                                 |                                      |            |
   +---------------------------------------------------------+--------------------------------------+------------+
   | :meth:`assertWarnsRegex(warn, r, fun, *args, **kwds)    | ``fun(*args, **kwds)`` raises *warn* | 3.2        |
   | <TestCase.assertWarnsRegex>`                            | and the message matches regex *r*    |            |
   +---------------------------------------------------------+--------------------------------------+------------+
   | :meth:`assertLogs(logger, level)                        | The ``with`` block logs on *logger*  | 3.4        |
   | <TestCase.assertLogs>`                                  | with minimum *level*                 |            |
   +---------------------------------------------------------+--------------------------------------+------------+
   | :meth:`assertNoLogs(logger, level)                      | The ``with`` block does not log on   | 3.10       |
   | <TestCase.assertNoLogs>`                                |  *logger* with minimum *level*       |            |
   +---------------------------------------------------------+--------------------------------------+------------+

   .. method:: assertRaises(exception, callable, *args, **kwds)
               assertRaises(exception, *, msg=None)

      Test that an exception is raised when *callable* is called with any
      positional or keyword arguments that are also passed to
      :meth:`assertRaises`.  The test passes if *exception* is raised, is an
      error if another exception is raised, or fails if no exception is raised.
      To catch any of a group of exceptions, a tuple containing the exception
      classes may be passed as *exception*.

      If only the *exception* and possibly the *msg* arguments are given,
      return a context manager so that the code under test can be written