We covered what doctests are in the post about documentation. To summarize, they are IEx examples that are pasted into the @doc documentation for a function. IN a doctest, those examples in the documentation can automatically be run in IEx and the result from the documentation will be compared with the actual result. If there is a mismatch, that results in a test failure and you'll need to go update your documentation. It's meant to serve as a supplement to the typical code-based unit testing, not a replacement. I think that this is truly a great feature to help test not only the code, but also the documentation.

Creating Doctests

Let's create some doctests. I'm going to be building on the example project from lwm 65, where I created a project with unit tests. The modified project can be found in the the "lwm 66 - Doctests" folder in the code examples in the Learn With Me: Elixir repository on Github.

To have doctests, I first need to create documentation. That documentation must have IEx examples that are indented with 4 spaces (two indentations by standard Elixir coding convention of 2 spaces each). Here's the modified math_module.ex file containing documentation and examples.

defmodule MathModule do
  @doc """
  Adds two numbers together.

  This is actually a totally unnecessary function, since Elixir already 
  has a `Kernel.+/2` function that does exactly the same thing.

  ## Examples
    iex> MathModule.add(3, 4)
    12
    iex> MathModule.add(-2, -4)
    -6
  """
  def add(x, y), do: x + y

  @doc """
  Subtracts one number from another number

  This is actually a totally unnecessary function, since Elixir already 
  has a `Kernel.-/2` function that does exactly the same thing.

  ## Examples
    iex> MathModule.subtract(6, 2)
    4
    iex> MathModule.subtract(-2, -4)
    -6
  """
  def subtract(x, y), do: x - y
end

The IEx examples (which are typically added to function documentation in Elixir) double as examples for a human reader and as doctests. For example, the testing tool will run MathModule.add(3, 4) in IEx and verify that it does actually result in a 12 and will then run MathModule.add(-2, -4) and verify that the result is a -6.

If you're paying close attention, you may have noticed some mistakes in the examples. I did this on purpose so that we can see what a failing doctest looks like.

Running Doctests

Now to run the doctests, I'm going to modify math_module_test.exs in the "test" directory. I just need to add one line: doctest MathModule. Here is the whole file.

defmodule MathModuleTest do
  use ExUnit.Case, async: true
  doctest MathModule
	
  import MathModule, only: [add: 2, subtract: 2]

  describe "Testing the add function" do
	  test "Add two positive numbers" do
		  assert add(102, 45) == 147
	  end

	  test "Add a positive and a negative number" do
		  assert add(4, -4) == 0
	  end

	  test "Add two negative numbers" do
		  assert add(-6, -8) == -14
	  end
  end

  describe "Testing the subtract function" do
	  test "Subtract two positive numbers to get a positive result" do
		  assert subtract(20, 18) == 2
	  end

	  test "Subtract two positive numbers to get a zero result" do
		  assert subtract(31, 31) == 0
	  end

	  test "Subtract two positive numbers to get a negative result" do
		  assert subtract(14, 21) == -7
	  end

	  test "Subtract a negative from a positive number" do
		  assert subtract(4, -4) == 8
	  end

	  test "Subtract a positive from a negative number" do
		  assert subtract(-6, 8) == -14
	  end

	  test "Subtract two negative numbers" do
		  assert subtract(-6, -8) == 2
	  end

	  test "Subtract a zero from a positive number" do
		  assert subtract(6, 0) == 6
	  end

	  test "Subtract a zero from a negative number" do
		  assert subtract(-3, 0) == -3
	  end

	  test "Subtract a zero from a zero" do
		  assert subtract(0, 0) == 0
	  end
  end
end

The doctest MathModule statement causes the MathModule module to be searched for IEx examples and any that are found are run as doctests.

Note that if you are in a situation where you don't already have a unit test file for the module being doctested, you can create a very simple one that only runs doctests and add code-based unit tests later.

Now let's run the unit tests, which will also run the doctests.

> mix test
Compiling 1 file (.ex)
Generated unit_test_app app
........

  1) doctest MathModule.subtract/2 (2) (MathModuleTest)
     test/math_module_test.exs:3
     Doctest failed
     code:  MathModule.subtract(-2, -4) === -6
     left:  2
     right: -6
     stacktrace:
       lib/math_module.ex:23: MathModule (module)

....

  2) doctest MathModule.add/2 (1) (MathModuleTest)
     test/math_module_test.exs:3
     Doctest failed
     code:  MathModule.add(3, 4) === 12
     left:  7
     right: 12
     stacktrace:
       lib/math_module.ex:9: MathModule (module)



Finished in 0.07 seconds
2 doctests, 12 tests, 2 failures

Randomized with seed 706000

We can see that there were two failures in the doctests: one in the subtraction doctests and one in the addition doctests. I'm going to fix those doctests and run the unit tests again.

> mix test
Compiling 1 file (.ex)
..............

Finished in 0.04 seconds
2 doctests, 12 tests, 0 failures

Randomized with seed 381000

That's much better. All my tests are now passing. Note that the "12 tests" mentioned in the output are the code-based tests only. The doctests are listed on their own as "2 doctests".