We're going to move beyond the Elixir language for a little while and focus on the Elixir tooling. Today we're going to talk about Elixir projects.
Previously we've been looking at code in IEx or in isolated code files, but if you want to develop a real library or application, you need to use an Elixir project. Elixir projects provide a structure around the code, data, build, configuration, and test files that make up an Elixir library or application.
Introducing Mix
Mix is a command-line tool that is installed along with Elixir. Mix can perform a variety of tasks related to Elixir projects, including creating and building projects. You can think of mix as similar to npm, yarn, or git, in that it has multiple sub-commands that can be run to create a project, build a project, run tests, and so forth. These sub-commands are called "mix tasks". There are a variety of built-in mix tasks and you have the ability to create your own mix tasks if you want to.
Let's look at the built in mix tasks by running mix help on the command line.
> mix help
mix # Runs the default task (current: "mix run")
mix app.start # Starts all registered apps
mix app.tree # Prints the application tree
mix archive # Lists installed archives
mix archive.build # Archives this project into a .ez file
mix archive.install # Installs an archive locally
mix archive.uninstall # Uninstalls archives
mix clean # Deletes generated application files
mix cmd # Executes the given command
mix compile # Compiles source files
mix deps # Lists dependencies and their status
mix deps.clean # Deletes the given dependencies' files
mix deps.compile # Compiles dependencies
mix deps.get # Gets all out of date dependencies
mix deps.tree # Prints the dependency tree
mix deps.unlock # Unlocks the given dependencies
mix deps.update # Updates the given dependencies
mix do # Executes the tasks separated by comma
mix escript # Lists installed escripts
mix escript.build # Builds an escript for the project
mix escript.install # Installs an escript locally
mix escript.uninstall # Uninstalls escripts
mix format # Formats the given files/patterns
mix help # Prints help information for tasks
mix hex # Prints Hex help information
mix hex.audit # Shows retired Hex deps for the current project
mix hex.build # Builds a new package version locally
mix hex.config # Reads, updates or deletes local Hex config
mix hex.docs # Fetches or opens documentation of a package
mix hex.info # Prints Hex information
mix hex.organization # Manages Hex.pm organizations
mix hex.outdated # Shows outdated Hex deps for the current project
mix hex.owner # Manages Hex package ownership
mix hex.publish # Publishes a new package version
mix hex.repo # Manages Hex repositories
mix hex.retire # Retires a package version
mix hex.search # Searches for package names
mix hex.user # Manages your Hex user account
mix loadconfig # Loads and persists the given configuration
mix local # Lists local tasks
mix local.hex # Installs Hex locally
mix local.public_keys # Manages public keys
mix local.rebar # Installs Rebar locally
mix new # Creates a new Elixir project
mix profile.cprof # Profiles the given file or expression with cprof
mix profile.eprof # Profiles the given file or expression with eprof
mix profile.fprof # Profiles the given file or expression with fprof
mix run # Starts and runs the current application
mix test # Runs a project's tests
mix xref # Performs cross reference checks
iex -S mix # Starts IEx and runs the default task
So you can see that there are quite a few mix tasks that you can use on your project. They all look fairly useful, but I wouldn't be surprised if some of these are rarely used.
Creating a New Projects
You can create a new Elixir project by running "mix new [project name]" on the command line. I created an example project by running mix new example_project. The tool creates a subdirectory called "example_project" that contains the project files and directories.
> mix new example_project
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating config
* creating config/config.exs
* creating lib
* creating lib/example_project.ex
* creating test
* creating test/test_helper.exs
* creating test/example_project_test.exs
Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:
cd example_project
mix test
Run "mix help" for more commands.
Mix tells us which files it created and gives us instructions on how to build the project.
Here's the directory listing of the "example_project" directory.
> ls
README.md config/ lib/ mix.exs test/
Although they are not listed in the directory listing because they're dot files, mix helpfully created .gitignore and .formatter.exs files as well.
You can also find the generated example project in the the "lwm 57 - Elixir Projects" folder in the code examples in the Learn With Me: Elixir repository on Github.
The Project Structure
Now that we have a nice project that was generated by mix for us, let's go over the individual files and folders and learn what they are for.
The Dot Files
As we mentioned earlier, mix created a .gitignore and a .formatter.exs file.
Those of you who are familiar with git will know that a .gitignore file tells git which files and directories to ignore when tracking files. These are typically build directories and build artifacts that don't need to be added to source control since they can be generated by the build process. Mix generated one for us so that we don't have to create one ourselves. How considerate.
Let's look at the contents of .gitignore.
# The directory Mix will write compiled artifacts to.
/_build/
# If you run "mix test --cover", coverage assets end up here.
/cover/
# The directory Mix downloads your dependencies sources to.
/deps/
# Where 3rd-party dependencies like ExDoc output generated docs.
/doc/
# Ignore .fetch files in case you like to edit your project deps locally.
/.fetch
# If the VM crashes, it generates a dump, let's ignore it too.
erl_crash.dump
# Also ignore archive artifacts (built via "mix archive.build").
*.ez
# Ignore package tarball (built via "mix hex.build").
example_project-*.tar
The things that are to be ignored, which are mostly directories, are nicely commented so that we know what those things are and why they are being ignored. That's a nice feature because I would have no idea what most of these things were otherwise.
Now let's take a look at the second dot file that was generated, .formatter.exs. This appears to be some settings for the Elixir source code formatting tool. I don't know much about this tool other than it exists and it will format your Elixir code in some standard way. I've seen similar tools for other languages like Javascript.
# Used by "mix format"
[
inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"]
]
From the comment it look like mix format is what you use to invoke the source code form formatting tool. It looks to me like this tells the formatter which files to format. From what I see here, it looks like mix.exs, .formatter.exs, and any .ex or .exs file found underneath the config, lib, or test directories will be formatted. Good to know. I don't plan to dive into the formatting tool now, but it's probably something I'll revisit in the future.
The lib Directory
The lib directory contains the source code for the project. All the code can sit directly within this directory or be organized in a hierarchy of sub-directories. It's up to you how you structure it, just as long as it lives in the lib directory. The build task will build any code files found in lib. It's the equivalent of the src directory in some other projects I've seen, although I've also seen lib used as the source code directory in other languages as well.
A single code file was generated for us, lib/example_project.ex. Remember that the .ex file extension indicates that this file is intended to be compiled by the Elixir compiler, not loaded into an interpreter. Here are the contents of that file.
defmodule ExampleProject do
@moduledoc """
Documentation for ExampleProject.
"""
@doc """
Hello world.
## Examples
iex> ExampleProject.hello()
:world
"""
def hello do
:world
end
end
Let's go over the generated code. The module is called "ExampleProject" to match the name of the project. The @moduledoc attribute followed by the heredoc string appears to be where you put the documentation for the module. The following @doc attribute followed by the heredoc string appears to be where the documentation for the hello/0 function is located. This documentation already contains an example of how to call this function.
I'm not familiar with Elixir documentation yet, so I don't know how the documentation should be formatted. All I know is that we put the documentation in the code files and that there's something that uses that to generate the nice documentation HTML pages we see when we look at the Elixir documentation. I believe that all that documentation is generated from code as well. I like well-documented code, so I'm looking forward to learning more about Elixir documentation in the future.
Finally, there is the code for the hello/0 function, which just returns the atom :world. This is the Elixir version of Hello World.
There's nothing particularly special regarding this file. It's just a placeholder that provides an example until you can put some real code into the project.
The test Directory
The test directory is where we'll find the unit tests for the project. Mix generated a unit test file test/example_project_test.exs. This is a .exs file, indicating that it is loaded into an interpreter and run rather than being compiled. That makes sense to me, since the tests will not need to be compiled when they are run during the development process.
Here's what the unit test module looks like:
defmodule ExampleProjectTest do
use ExUnit.Case
doctest ExampleProject
test "greets the world" do
assert ExampleProject.hello() == :world
end
end
The unit tests are Elixir code just like any other Elixir code. This unit test looks similar to other unit tests I've written in the past. In fact, it reminds me of jest tests I've written for a Node.js project.
Let's examine the elements of this unit test module. The unit tests are contained in a module, like most other code in Elixir. The tests in this module are clearly meant to test the ExampleProject module, , which is the code module from the "lib" directory we were previously looking at. The use ExUnit.Case statement provides some of the magic that makes testing possible. I don't know much about the use directive in Elixir, but I do know that it allows code in another module to be injected into this module and run.
The doctest ExampleProject statement runs some doctests in the ExampleProject module, which is the code module from the "lib" directory we were previously looking at. I don't know much about doctests, but from what I understand so far, the test runner will look for the examples in the documentation and then run them, matching the actual results with the results from the documentation. That sounds slightly magical to me, and I'm sure I'll be learning more about them.
Finally, there is a standard code-based test to be run. It looks like the test is associated with the "greets the world" string being passed to test, which is probably displayed when the test fails. The test has a single assertion that verifies that the ExampleProject.hello/0 function returns the atom :world. It does do that, so this test should pass.
I have no idea at this point when I would use the code tests and when I would use doctests and what the trade-offs are. I will certainly be diving into that in more detail in the future.
The second file in the test directory is test_helper.exs. This is a very simple file.
ExUnit.start()
A compiler wouldn't like this, since there is no module here. Since this is an Elixir script file, however, this just gets loaded into an interpreter. I'd say it's highly likely that this statement is what triggers the unit tests to start running after the test modules have been loaded into the interpeter.
I believe that any test configuration code can also be put in this file.
Running the Tests
Let's run the test! Just type mix test into the command line and watch the tests get run.
> mix test
Compiling 1 file (.ex)
Generated example_project app
..
Finished in 0.04 seconds
1 doctest, 1 test, 0 failures
Randomized with seed 108000
The mix tool builds any code that needs to be built and then finds the tests that need to be run. In this example, a single unit test was run and a single doctest was run. All tests passed.
The tests get run in random order, so if you need to run the tests again in that exact same order, the seed number can somehow be used to repeat that exact order of tests. Tests should be completely independent of each other and able to be run in random order, but that doesn't always happen. Running the tests randomly can help find any order dependency issues in your tests and being able to use the seed number to run the tests again in the exact same order can help you reproduce (and debug and fix) any errors found during a particular test run.
Now let's change some code and watch the unit test fail. I'm going to temporarily change the code in "lib/example_project.ex" so that it returns :word instead of :world, which is an example of a typical typo. Let's see what happens when a test fails.
> mix test
Compiling 1 file (.ex)
1) test greets the world (ExampleProjectTest)
test/example_project_test.exs:5
Assertion with == failed
code: assert ExampleProject.hello() == :world
left: :word
right: :world
stacktrace:
test/example_project_test.exs:6: (test)
2) doctest ExampleProject.hello/0 (1) (ExampleProjectTest)
test/example_project_test.exs:3
Doctest failed
code: ExampleProject.hello() === :world
left: :word
right: :world
stacktrace:
lib/example_project.ex:11: ExampleProject (module)
Finished in 0.06 seconds
1 doctest, 1 test, 2 failures
Randomized with seed 239000
Both the code test and the doctest failed. The test failure output is quite good, and provides a lot of useful information. I'm starting to get a feeling that Elixir has a high-quality unit test tool. I like writing tests, so I'm looking forward to diving into unit testing in Elixir.
I'll dive deeper into unit testing later in its own topic.
The config Directory
The config directory is where the application configuration is specified. The configuration takes the form of Elixir code, but the config.exs file that was generated for this project contains no configuration code. However, the file does have a lot of comments telling you about configuration, which is very informative and an interesting read. There's more to configuration than I can cover here, so I'll dive deeper into configuration in its own topic.
The mix.exs File
The mix.exs file contains the project configuration information that the mix tool uses to work with the project. This is where you specify things like application metadata and dependencies. You can think of this being similar to a package.json file in Node.js applications.
Here's what the generated mix.exs looks like. Like everything else, it consists of Elixir code. It looks like Elixir favors doing everything in code, and not using some other format like JSON or XML.
defmodule ExampleProject.MixProject do
use Mix.Project
def project do
[
app: :example_project,
version: "0.1.0",
elixir: "~> 1.7",
start_permanent: Mix.env() == :prod,
deps: deps()
]
end
# Run "mix help compile.app" to learn about applications.
def application do
[
extra_applications: [:logger]
]
end
# Run "mix help deps" to learn about dependencies.
defp deps do
[
# {:dep_from_hexpm, "~> 0.3.0"},
# {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"},
]
end
end
I'm able to get an idea of what mix.exs file contains by looking at it, but I don't know much in the way of details at this point. I have no doubt that there is much that can be configured in this file, and I expect to learn more about it as I progress.
Building a Project
Although running mix test will automatically build the project before it runs, you can just build it by simply running mix in the project directory.
Here's what that looks like in the generated example project if you haven't previously built anything.
> mix
Compiling 1 file (.ex)
Generated example_project app
All artifacts from the compilation process are put into the "_build" subdirectory underneath the project directory.
Here's a recursive directory listing of the compiled code for this example project.
> ls -R _build
_build:
dev/
_build/dev:
lib/
_build/dev/lib:
example_project/
_build/dev/lib/example_project:
consolidated/ ebin/
_build/dev/lib/example_project/consolidated:
Elixir.Collectable.beam Elixir.IEx.Info.beam Elixir.List.Chars.beam
Elixir.Enumerable.beam Elixir.Inspect.beam Elixir.String.Chars.beam
_build/dev/lib/example_project/ebin:
Elixir.ExampleProject.beam example_project.app
It looks like the compiled project code was placed in "_build/dev/lib/example_project/ebin". I see two files, a .beam file and a .app file. I know that BEAM is the Erlang virtual machine, so I'm guessing that Elixir.ExampleProject.beam contains the ExampleProject module compiled into BEAM bytecode. I imagine that "example_project.app" contains information relevant to the application as a while, and is perhaps the starting point for execution.
It looks like "_build/dev/lib/example_project/consolidated" contains the BEAM bytecode for various Elixir standard library modules. I'm guessing that these are all the Elixir dependencies that the application needs. I'm also presuming that the Elixir libraries aren't magically loaded into BEAM, which understandably has no internal knowledge of Elixir, so the Elixir modules must be loaded by the application that is running them.
I notice that the directory structure contains "dev" in it, so I'm going to guess that it's possible to have differences between versions build for the Dev and Prod environments, and this is a build for the Dev environment.
I'm really just making educated guesses at a lot of this at this point until I can dive more in depth into what this is all about.
Running the Code
So now that we have a project that is built, how do we run the code in an Elixir project? Well, that answer seems to be a little obscure at this point. I've learned two ways of running the code so far, but these only apply to a development environment and not a production environment. So I've still much to learn, but I'll cover what I've learned so far.
Before I run the code, however, I'm going to make a small change to to the hello/0 function in example_project.ex so that it will output some text to the console.
This is so we'll get some kind of feedback when we run it.
def hello do
IO.puts "Hello, World!"
:world
end
That's better. Now we should see a message on the screen when the code is run. The tests still pass, although the output looks weird because "Hello, World!" is being included in the test output. There must be a better way to handle side effects like that, but I haven't looked at unit testing in depth yet.
I'm currently aware of two ways of running an Elixir in a development environment. It would not surprise me if there were other ways I haven't heard of yet.
Running the Project From the Command Line
The first way of running your code is by using the mix "run" task to run a particular function that's the starting point in the application.
Here's what it looks like for our example project.
> mix run -e "ExampleProject.hello()"
Hello, World!
That's not too bad. It's a bit annoying to have to specify the starting function, but it gets the job done.
Running the Project From Within IEx
The second way of running your code is to simply load the entire application into IEx and run it from there. You do this by running iex -S mix, which builds the project and feeds it into IEx. From there you can run the function that forms the application entry point.
> iex -S mix
Compiling 1 file (.ex)
Generated example_project app
Interactive Elixir (1.7.2) - press Ctrl+C to exit (type h() ENTER for help)
iex> ExampleProject.hello()
Hello, World!
:world
This is more typing and clearly is only suited for a developer, but it allows us to run any function we like in our application. If some part of the code is giving us trouble, we can call the relevant functions directly from IEx. I really like this feature because we can just concentrate on particular functions rather than rerunning the entire application every time, and we can see what those functions are returning. This is very good for debugging purposes.
The "-S" option indicates to IEx that it is running a script. I imagine that the mix tool is probably implemented as an Elixir script, so this would be how mix interacts with the IEx environment.
It sounds like from what I've been reading that running the application in the context of IEx is a common way of running an application while developing. It certainly provides a nice way to call various functions in the application to help you debug and do some one-time tests.
From what I understand, running an application in a production environment is a bit different than running it in a development environment. I haven't seen any information about that yet, but I assume that I'll eventually get there.
Conclusion
In a very short time, I just went from feeling like I'm understanding Elixir pretty well to feeling like I know very little. Branching out to learning the Elixir project structure and toolset has made me aware of how much more there is to learn. In learning the basics of Elixir projects, I caught a glimpse of a lot I hadn't previously been aware of. I feel like I'm barely scraping the surface of what there is to learn about building and running Elixir applications.
It feels a bit overwhelming, but the best thing is to accept there's a lot I don't know, there's a lot that will seem like incomprehensible magic for now, and there's a lot I will simply not understand yet at this point. The best strategy is to keep going and keep learning: it will eventually all become familiar and make sense. At least that's been my experience for other technologies.