We covered what doctests are in the post about documentation. To summarize, they are IEx examples that are pasted into the
@doc documentation for a function. IN a doctest, those examples in the documentation can automatically be run in IEx and the result from the documentation will be compared with the actual result. If there is a mismatch, that results in a test failure and you'll need to go update your documentation. It's meant to serve as a supplement to the typical code-based unit testing, not a replacement. I think that this is truly a great feature to help test not only the code, but also the documentation.
Let's create some doctests. I'm going to be building on the example project from lwm 65, where I created a project with unit tests. The modified project can be found in the the "lwm 66 - Doctests" folder in the code examples in the Learn With Me: Elixir repository on Github.
To have doctests, I first need to create documentation. That documentation must have IEx examples that are indented with 4 spaces (two indentations by standard Elixir coding convention of 2 spaces each). Here's the modified math_module.ex file containing documentation and examples.
defmodule MathModule do @doc """ Adds two numbers together. This is actually a totally unnecessary function, since Elixir already has a `Kernel.+/2` function that does exactly the same thing. ## Examples iex> MathModule.add(3, 4) 12 iex> MathModule.add(-2, -4) -6 """ def add(x, y), do: x + y @doc """ Subtracts one number from another number This is actually a totally unnecessary function, since Elixir already has a `Kernel.-/2` function that does exactly the same thing. ## Examples iex> MathModule.subtract(6, 2) 4 iex> MathModule.subtract(-2, -4) -6 """ def subtract(x, y), do: x - y end
The IEx examples (which are typically added to function documentation in Elixir) double as examples for a human reader and as doctests. For example, the testing tool will run
MathModule.add(3, 4) in IEx and verify that it does actually result in a
12 and will then run
MathModule.add(-2, -4) and verify that the result is a
If you're paying close attention, you may have noticed some mistakes in the examples. I did this on purpose so that we can see what a failing doctest looks like.
Now to run the doctests, I'm going to modify math_module_test.exs in the "test" directory. I just need to add one line:
doctest MathModule. Here is the whole file.
defmodule MathModuleTest do use ExUnit.Case, async: true doctest MathModule import MathModule, only: [add: 2, subtract: 2] describe "Testing the add function" do test "Add two positive numbers" do assert add(102, 45) == 147 end test "Add a positive and a negative number" do assert add(4, -4) == 0 end test "Add two negative numbers" do assert add(-6, -8) == -14 end end describe "Testing the subtract function" do test "Subtract two positive numbers to get a positive result" do assert subtract(20, 18) == 2 end test "Subtract two positive numbers to get a zero result" do assert subtract(31, 31) == 0 end test "Subtract two positive numbers to get a negative result" do assert subtract(14, 21) == -7 end test "Subtract a negative from a positive number" do assert subtract(4, -4) == 8 end test "Subtract a positive from a negative number" do assert subtract(-6, 8) == -14 end test "Subtract two negative numbers" do assert subtract(-6, -8) == 2 end test "Subtract a zero from a positive number" do assert subtract(6, 0) == 6 end test "Subtract a zero from a negative number" do assert subtract(-3, 0) == -3 end test "Subtract a zero from a zero" do assert subtract(0, 0) == 0 end end end
doctest MathModule statement causes the
MathModule module to be searched for IEx examples and any that are found are run as doctests.
Note that if you are in a situation where you don't already have a unit test file for the module being doctested, you can create a very simple one that only runs doctests and add code-based unit tests later.
Now let's run the unit tests, which will also run the doctests.
> mix test Compiling 1 file (.ex) Generated unit_test_app app ........ 1) doctest MathModule.subtract/2 (2) (MathModuleTest) test/math_module_test.exs:3 Doctest failed code: MathModule.subtract(-2, -4) === -6 left: 2 right: -6 stacktrace: lib/math_module.ex:23: MathModule (module) .... 2) doctest MathModule.add/2 (1) (MathModuleTest) test/math_module_test.exs:3 Doctest failed code: MathModule.add(3, 4) === 12 left: 7 right: 12 stacktrace: lib/math_module.ex:9: MathModule (module) Finished in 0.07 seconds 2 doctests, 12 tests, 2 failures Randomized with seed 706000
We can see that there were two failures in the doctests: one in the subtraction doctests and one in the addition doctests. I'm going to fix those doctests and run the unit tests again.
> mix test Compiling 1 file (.ex) .............. Finished in 0.04 seconds 2 doctests, 12 tests, 0 failures Randomized with seed 381000
That's much better. All my tests are now passing. Note that the "12 tests" mentioned in the output are the code-based tests only. The doctests are listed on their own as "2 doctests".