Learn With Me: Elixir - Managing Dependencies (#58)
We previously learned about Elixir projects, and we're going to expand upon that by learning about managing dependencies for those projects. For managing dependencies, Node.js has npm, Python has pip, Ruby has RubyGems, and C# has NuGet. These are known as package managers because each dependency is encapsulated in a package that contains the files for that dependency.
Elixir has hex, which manages dependencies for Elixir. If you're familiar with another package manager, then the concepts behind hex will be familiar to you.
Digression Time - Dependencies and Packages
I'm going to assume that most of you have used package managers before to manage dependencies for your projects, but for those of you who haven't or who just need a refresher, I'm going to briefly talk about dependencies and packages.
A dependency is another bundle of code, usually a library of some kind, that was written by someone else and is used by your code. This dependency usually provides some kind of functionality that you don't want to implement yourself. They're called dependencies because your code is dependent on them to run. It's generally a good thing to use dependencies since someone else (or a team of someone elses) spent a lot of time creating the dependency and they usually did it better than you can do it. Open source library dependencies also typically benefit from bug fixes and contributions from other people using that library.
Dependencies can sometimes be misused in that someone will only use a single function in a large dependency to do something that would have been trivial to do themselves. This can result in a large amount of code being transmitted or loaded when most of that code is not actually needed. This is particularly acute in the web client world, where page sizes can balloon due to unneeded code being downloaded, which has led to code pruning tools like webpack. It's not as big a deal for a server application where disk space is fairly plentiful and the code stays on the server, but anything that gets downloaded and installed can become very large very quickly due to large dependencies. That doesn't mean you have to avoid large dependencies entirely, but be aware of the cost/benefit tradeoff. If you're using significant parts of a large dependency, then there's nothing wrong with that.
The code and files that constitute a dependency are typically packaged into a package, which are managed by a package managers. In the bad old days, you just to download any dependencies and include them in your project yourself. You also had to manually download sub-dependencies (dependencies of the dependency) and do any updates manually. Nowadays, most languages have their own package managers, which will automatically download and install dependencies and all their dependencies, track what's installed, the versions that are installed, and handle updating those dependencies for you. It's very nice and it can be automatically done when your project is built.
Generally, a project will descibe which dependencies it needs and which version (or a range of versions) and the package manager will retrieve the latest dependency that matches. If a later version comes out with bug fixes, it can be updated easily. Automated builds will retrieve the latest version in whatever the allowed version range is. On a development machine, where the dependencies have already been retrieved, there's usually a command available that will update the dependency to the latest allowed version. It can make things a little complicated when packages depend on other packages, which depend on yet other packages, and so forth. A large dependency chain can cause a lot of packages to be downloaded.
The advantage of all this is that it makes dependencies easy to manage and acquire, but the disadvantage is that things can break if a bug is introduced into a dependent package and it's not easy to diagnose and fix, especially if it's deep down in the depedency hierarchy. This can be helped somewhat by being more specific when it comes to specifying the dependency versions, but you have to balance between getting bug fixes and security fixes that new versions bring vs bugs introduced by new versions. I believe the most common policy is to specify the major and minor version numbers, but let the dependency manage retrieve the latest patch version. You can then manually upgrade to a newer major or minor version, giving you the ability to test, but it will never pull in that version automatically. Another common policy is to allow minor version updates to automatically be downloaded, but that does make it more likely that a new minor version can cause unexpected issues. In theory, a minor version update should not be a problem, but that doesn't always work out in reality.
All package managers I've seen so far use SemVer, which is a versioning standard. SemVer version numbers look like X.Y.Z, where X is the major version number, Y is the minor version number, and Z is the patch version number. According to the SemVer standard, a package needs a new major version number whenever breaking changes are made, a new minor version number when new functionality (but non-breaking) is added, and a new patch number when the functionality remains the same, but some bug fixes were applied. This is mostly followed, but there can be exceptions. The most common exception is someone putting breaking changes in a minor version update.
Specifying Dependencies
The dependencies in Elixir are specified in the deps/0 function in the mix.exs project file. To add a dependency, you have to add the dependency and version to the list in the deps function. Unlike package managers like npm and yarn, you cannot add dependencies from the command line.
A dependency entry looks something like this: { :dependency, "~> 1.0.0" }. The atom is a unique identifier for a package in the hex package archive and the string indicates which versions can be used. This example specifies that the major and minor versions are fixed at 1.0, but the latest patch version can be applied. So when the dependencies are downloaded, version 1.0.0, 1.0.1, 1.0.2, etc can be downloaded, depending on what the most recent version is.
Pre-release versions can be specified by appending a dash and an identifier to the version number. It's common to see versions like "1.0.0-dev" or "1.0.0-alpha.4" or "1.0.0-beta.1". By default, the package manager ignores pre-release versions.
You can read about versioning by typing "h Version" into IEx. That brings up some useful information about versioning, including the following table.
~> | Translation
~> 2.0.0 | >= 2.0.0 and < 2.1.0
~> 2.1.2 | >= 2.1.2 and < 2.2.0
~> 2.1.3-dev | >= 2.1.3-dev and < 2.2.0
~> 2.0 | >= 2.0.0 and < 3.0.0
~> 2.1 | >= 2.1.0 and < 3.0.0
If you want to, you can even construct a version expression using boolean operators to create a more complex dependency version specification, but I'm won't go into the details of that here.
Displaying Dependencies
Project dependencies can be displayed by entering "mix deps" to run the "deps" mix task. This is an example is from another Elixir project I had lying around from when I was following some examples in a book. I'm going to do a walkthrough of using dependencies with our own project later on, so you'll eventually have an opportunity to do this yourself.
> mix deps
* ex_doc (Hex package) (mix)
the dependency is not locked. To generate the "mix.lock" file run "mix deps.get"
* poison (Hex package) (mix)
the dependency is not locked. To generate the "mix.lock" file run "mix deps.get"
* earmark (Hex package) (mix)
the dependency is not locked. To generate the "mix.lock" file run "mix deps.get"
* httpoison (Hex package) (mix)
the dependency is not locked. To generate the "mix.lock" file run "mix deps.get"
In this example, there are four dependencies: ex_doc (a documentation generator), poison (a JSON library), earmark (used for generating HTML documentation), and httpoison (an HTTP library).
Retrieving Dependencies
Project dependencies can be retrieved by entering "mix deps.get" to run the "deps.get" mix task. Here's an example from the same project with the "httpoison" dependency.
> mix deps.get
Resolving Hex dependencies...
Dependency resolution completed:
certifi 2.4.2
hackney 1.14.3
httpoison 1.0.0
idna 6.0.0
metrics 1.0.1
mimerl 1.0.2
parse_trans 3.3.0
ssl_verify_fun 1.1.4
unicode_util_compat 0.4.1
* Getting ssl_verify_fun (Hex package)
Checking package (https://repo.hex.pm/tarballs/ssl_verify_fun-1.1.4.tar)
Fetched package
* Getting unicode_util_compat (Hex package)
Checking package (https://repo.hex.pm/tarballs/unicode_util_compat-0.4.1.tar)
Fetched package
* Getting parse_trans (Hex package)
Checking package (https://repo.hex.pm/tarballs/parse_trans-3.3.0.tar)
Fetched package
Note that not only was the httpoison package retrieved, but all of its dependencies are retrieved as well. The project needs all these packages in order to use the functionality in httpoison.
Once the dependencies have been retrieved, information regarding the dependencies that have been retrieved is stored in the mix.lock file. Here's the mix.lock file that was generated after we retrieved httpoison and its dependencies.
%{
"certifi": {:hex, :certifi, "2.4.2", "75424ff0f3baaccfd34b1214184b6ef616d89e420b258bb0a5ea7d7bc628f7f0", [:rebar3], [{:parse_trans, "~>3.3", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"},
"hackney": {:hex, :hackney, "1.14.3", "b5f6f5dcc4f1fba340762738759209e21914516df6be440d85772542d4a5e412", [:rebar3], [{:certifi, "2.4.2", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "1.0.2", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.4", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
"httpoison": {:hex, :httpoison, "1.0.0", "1f02f827148d945d40b24f0b0a89afe40bfe037171a6cf70f2486976d86921cd", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
"idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
"metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"},
"mimerl": {:hex, :mimerl, "1.0.2", "993f9b0e084083405ed8252b99460c4f0563e41729ab42d9074fd5e52439be88", [:rebar3], [], "hexpm"},
"parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [], [], "hexpm"},
"ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.4", "f0eafff810d2041e93f915ef59899c923f4568f4585904d010387ed74988e77b", [], [], "hexpm"},
"unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [], [], "hexpm"},
}
It looks like mix.lock contains metadata regarding each package that was retrieved, including the name, version, and what looks like a package hash. This appears to be valid Elixir code, with all the data contained in a map structure.
The mix.lock file "locks" down the dependency versions to those particular versions. So even if a later dependency is released that is within version range specified in config.exs, it won't be automatically updated until the mix.lock is removed or you run "mix deps.update" to update the dependencies. Dependencies will only be updated if they are in the allowed version range.
The mix.lock file is the equivalent of package.lock in a Node.js project. If you want to lock down exactly which versions are retrieved, which is a good idea, since your production build could conceivably retrieve a different version of a package than what you were using in your dev and test environments, then you need to add mix.lock to version control.
If we run the "mix deps" task again, we'll see that all the packages are now listed and that the exact version is locked in until it is changed later on. That's because there's a mix.lock file present.
> mix deps
* parse_trans 3.3.0 (Hex package) (rebar3)
locked at 3.3.0 (parse_trans) 09765507
ok
* mimerl 1.0.2 (Hex package) (rebar3)
locked at 1.0.2 (mimerl) 993f9b0e
ok
* metrics 1.0.1 (Hex package) (rebar3)
locked at 1.0.1 (metrics) 25f094de
ok
* unicode_util_compat 0.4.1 (Hex package) (rebar3)
locked at 0.4.1 (unicode_util_compat) d869e4c6
ok
* idna 6.0.0 (Hex package) (rebar3)
locked at 6.0.0 (idna) 689c46cb
ok
* ssl_verify_fun 1.1.4 (Hex package) (rebar3)
locked at 1.1.4 (ssl_verify_fun) f0eafff8
ok
* certifi 2.4.2 (Hex package) (rebar3)
locked at 2.4.2 (certifi) 75424ff0
ok
* hackney 1.14.3 (Hex package) (rebar3)
locked at 1.14.3 (hackney) b5f6f5dc
ok
* httpoison 1.0.0 (Hex package) (mix)
locked at 1.0.0 (httpoison) 1f02f827
ok
Updating Dependencies
Your dependencies will probably be updated at some point and newer versions become available. As long as you have a mix.lock file that locks in which versions are being used, your project will ignore any newer versions. This is a good thing, since updates will happen when you choose to apply them and not at some arbitrary time.
To figure out which packages can be updated, run mix hex.outdated. Here's what this looks like for the Elixir project I have.
> mix hex.outdated
Dependency Current Latest Update possible
earmark 1.3.1 1.3.1
ex_doc 0.19.2 0.19.3 Yes
httpoison 1.0.0 1.5.0 No
poison 3.1.0 4.0.1 No
This example has four dependencies, and three of them have updates available. Pay attention to the "Update possible" column, which can be blank or display "Yes" or "No". A blank value means that no update is available. A "Yes" value means that the latest version falls within the allowed version range in mix.exs. You can update it with the update tool. In the example above, a new patch version has been released for the "ex_doc" dependency, and that falls within the range allowed in mix.exs. A "No" value means that the latest version does not fall within that allowed range, so it cannot be updated with the update tool. To update this dependency, we would have to change the allowed version range in mix.exs and then update. This makes a significant update a deliberate act that can't be done accidentally.
To perform the update, you'll need to run the deps.update mix task. You can run mix deps.update --all to upgrade all the dependencies or you can run mix deps.update [package_name] to update a single dependency.
Let's try updating all the dependencies in this project and see what happens.
> mix deps.update --all
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
certifi 2.4.2
earmark 1.3.1
httpoison 1.0.0
idna 6.0.0
makeup 0.8.0
makeup_elixir 0.13.0
metrics 1.0.1
mimerl 1.0.2
nimble_parsec 0.5.0
parse_trans 3.3.0
poison 3.1.0
ssl_verify_fun 1.1.4
unicode_util_compat 0.4.1
Upgraded:
ex_doc 0.19.2 => 0.19.3
hackney 1.14.3 => 1.15.0
* Updating ex_doc (Hex package)
* Updating hackney (Hex package)
> mix hex.outdated
Dependency Current Latest Update possible
earmark 1.3.1 1.3.1
ex_doc 0.19.3 0.19.3
httpoison 1.0.0 1.5.0 No
poison 3.1.0 4.0.1 No
It updated ex_doc to the latest patch version like I expected. It also updated hackney as well. The hackney dependency is not a direct dependency, but is actually a dependency of httpoison. So it looks to me like it goes through the entire tree of dependencies and automatically upgrades any packages that can be upgraded within the allowed version range. I notice that hackney was upgraded from 1.14.4 to 1.15.0. That's a minor version upgrade. I'm pretty certain that it was looking at the allowed dependency version range in the httpoison package, which specified that hackney could be updated to any minor version.
The packages where an update isn't possible without a change to mix.exs (a "No" in the Update possible column) were not updated. I wonder what would happen if the latest version of a dependency was outside the allowed range, but there were later versions within that range. Would it be updated to the latest allowed version? I suspect that it would. So if poison had a version 3.1.2 package published, it would probably update from 3.1.0 to 3.1.2, but not to 4.0.1.
Anyway, let's update poison to the latest version by updating mix.exs to specify that we want to depend on version 4 now.
defp deps do
[
{:httpoison, "~> 1.0.0"},
{:poison, "~> 4.0"},
{:ex_doc, "~> 0.19.2"},
{:earmark, "~> 1.3"}
]
end
This time I'm going to let it update to any minor version (4..) by specifying "~> 4.0". Now I'll run the tool to look at the outdated packages.
> mix hex.outdated
Dependency Current Latest Update possible
earmark 1.3.1 1.3.1
ex_doc 0.19.3 0.19.3
httpoison 1.0.0 1.5.0 No
poison 3.1.0 4.0.1 Yes
An update is possible now. Let's run the deps.update mix task.
> mix deps.update --all
Resolving Hex dependencies...
Dependency resolution completed:
Unchanged:
certifi 2.4.2
earmark 1.3.1
ex_doc 0.19.3
hackney 1.15.0
httpoison 1.0.0
idna 6.0.0
makeup 0.8.0
makeup_elixir 0.13.0
metrics 1.0.1
mimerl 1.0.2
nimble_parsec 0.5.0
parse_trans 3.3.0
ssl_verify_fun 1.1.4
unicode_util_compat 0.4.1
Upgraded:
poison 3.1.0 => 4.0.1
* Updating poison (Hex package)
> mix hex.outdated
Dependency Current Latest Update possible
earmark 1.3.1 1.3.1
ex_doc 0.19.3 0.19.3
httpoison 1.0.0 1.5.0 No
poison 4.0.1 4.0.1
Alright! It updated the poison dependency to 4.0.1. Now I just have to run the unit tests in the project (and any necessary integration tests) to verify that the upgrade didn't break anything.
By the way, some of this information about updating dependencies comes from a great post written by Toby Osborn, which talks about updating dependencies. It's short, and you may find it worth reading.
Updating Hex
While I was running some of the mix tasks while writing this, some of the output was displaying a message that said "A new Hex version is available (0.17.1 < 0.19.0), please update with mix local.hex". So it looks like my version of hex is out of date. Let's follow its suggestion and update hex using the "local.hex" mix task.
> mix local.hex
Found existing entry: ~/.mix/archives/hex-0.17.1
Are you sure you want to replace it with "https://repo.hex.pm/installs/1.8.0/hex-0.19.0.ez"? [Yn] Y
* creating ~/.mix/archives/hex-0.19.0
That was easy. My version of hex is now updated.
Building Dependencies
Once you've added dependencies, you have to first run the deps.get mix task to retrieve the dependencies before you can build the project. If you didn't, here's what it would look like:
> mix
Unchecked dependencies for environment dev:
* httpoison (Hex package)
the dependency is not locked. To generate the "mix.lock" file run "mix deps.get"
** (Mix) Can't continue due to errors on dependencies
Running mix dep.get creates the mix.lock file. The project will not build if it has dependencies, but no mix.lock file.
Once you've retrieved the dependencies, you can build the project by running mix. Not only will this build the project, but it will also build the dependencies.
> mix
===> Compiling parse_trans
===> Compiling mimerl
==> nimble_parsec
Compiling 4 files (.ex)
Generated nimble_parsec app
==> makeup
Compiling 45 files (.ex)
Generated makeup app
===> Compiling metrics
===> Compiling unicode_util_compat
===> Compiling idna
warning: String.strip/1 is deprecated. Use String.trim/1 instead
c:/Development/Elixir/issues/deps/poison/mix.exs:4
==> poison
Compiling 4 files (.ex)
warning: Integer.to_char_list/2 is deprecated. Use Integer.to_charlist/2 instead
lib/poison/encoder.ex:173
Generated poison app
==> ssl_verify_fun
Compiling 7 files (.erl)
Generated ssl_verify_fun app
===> Compiling certifi
===> Compiling hackney
==> earmark
Compiling 1 file (.yrl)
Compiling 2 files (.xrl)
Compiling 3 files (.erl)
Compiling 25 files (.ex)
Generated earmark app
==> httpoison
Compiling 2 files (.ex)
Generated httpoison app
==> makeup_elixir
Compiling 4 files (.ex)
Generated makeup_elixir app
==> ex_doc
Compiling 18 files (.ex)
Generated ex_doc app
==> example
Compiling 4 files (.ex)
Generated example app
It built all the dependencies and then built the main application. By the way, I reverted the poison dependency to 3.1 before I ran this. I see that some warnings were emitted for deprecated functions. I suspect that if I upgraded poison and httpoison to the latest versions, these warnings would disappear.
It looks like the entire package is compiled into its own application. I'm starting to think that libraries are also considered separate applications, but that just leads me to wonder what exactly an "application" is in the context of Elixir.
Storing Dependencies
All the dependencies that we retrieved were placed in the "deps" directory. Here's a recursive file listing showing what is there.
> ls -R deps
deps:
certifi/ hackney/ httpoison/ idna/ metrics/ mimerl/ parse_trans/ ssl_verify_fun/ unicode_util_compat/
deps/certifi:
ebin/ LICENSE priv/ README.md rebar.config src/ test/
deps/certifi/ebin:
certifi.app certifi.beam
deps/certifi/priv:
cacerts.pem
deps/certifi/src:
certifi.app.src certifi.erl
deps/certifi/test:
certifi_tests.erl
deps/hackney:
ebin/ include/ LICENSE MAINTAINERS NEWS.md NOTICE README.md rebar.config rebar.lock src/
deps/hackney/ebin:
hackney.app hackney_date.beam hackney_manager.beam hackney_response.beam hackney_trace.beam
hackney.beam hackney_headers.beam hackney_metrics.beam hackney_socks5.beam hackney_url.beam
hackney_app.beam hackney_headers_new.beam hackney_multipart.beam hackney_ssl.beam hackney_util.beam
hackney_bstr.beam hackney_http.beam hackney_pool.beam hackney_stream.beam
hackney_connect.beam hackney_http_connect.beam hackney_pool_handler.beam hackney_sup.beam
hackney_cookie.beam hackney_local_tcp.beam hackney_request.beam hackney_tcp.beam
deps/hackney/include:
hackney.hrl hackney_lib.hrl
deps/hackney/src:
hackney.app.src hackney_date.erl hackney_local_tcp.erl hackney_pool_handler.erl hackney_sup.erl
hackney.erl hackney_headers.erl hackney_manager.erl hackney_request.erl hackney_tcp.erl
hackney_app.erl hackney_headers_new.erl hackney_methods.hrl hackney_response.erl hackney_trace.erl
hackney_bstr.erl hackney_http.erl hackney_metrics.erl hackney_socks5.erl hackney_url.erl
hackney_connect.erl hackney_http_connect.erl hackney_multipart.erl hackney_ssl.erl hackney_util.erl
hackney_cookie.erl hackney_internal.hrl hackney_pool.erl hackney_stream.erl
deps/httpoison:
CHANGELOG.md lib/ LICENSE mix.exs README.md
deps/httpoison/lib:
httpoison/ httpoison.ex
deps/httpoison/lib/httpoison:
base.ex
deps/idna:
ebin/ LICENSE README.md rebar.config rebar.config.script rebar.lock src/ tmp/
deps/idna/ebin:
idna.app idna_bidi.beam idna_data.beam idna_table.beam punycode.beam
idna.beam idna_context.beam idna_mapping.beam idna_ucs.beam
deps/idna/src:
idna.app.src idna_bidi.erl idna_data.erl idna_mapping.erl idna_ucs.erl
idna.erl idna_context.erl idna_logger.hrl idna_table.erl punycode.erl
deps/idna/tmp:
deps/metrics:
ebin/ LICENSE README.md rebar.config rebar.lock src/
deps/metrics/ebin:
metrics.app metrics.beam metrics_dummy.beam metrics_exometer.beam metrics_folsom.beam
deps/metrics/src:
metrics.app.src metrics.erl metrics_dummy.erl metrics_exometer.erl metrics_folsom.erl
deps/mimerl:
ebin/ LICENSE README.md rebar.config rebar.lock src/
deps/mimerl/ebin:
mimerl.app mimerl.beam
deps/mimerl/src:
mimerl.app.src mimerl.erl mimerl.erl.src
deps/parse_trans:
ebin/ include/ LICENSE README.md rebar.config rebar.lock src/
deps/parse_trans/ebin:
ct_expand.beam parse_trans.app parse_trans_codegen.beam parse_trans_pp.beam
exprecs.beam parse_trans.beam parse_trans_mod.beam
deps/parse_trans/include:
codegen.hrl exprecs.hrl
deps/parse_trans/src:
ct_expand.erl parse_trans.app.src parse_trans_codegen.erl parse_trans_pp.erl
exprecs.erl parse_trans.erl parse_trans_mod.erl
deps/ssl_verify_fun:
ebin/ LICENSE Makefile mix.exs README.md rebar.config src/
deps/ssl_verify_fun/ebin:
ssl_verify_fingerprint.beam ssl_verify_fun_cert_helpers.beam ssl_verify_hostname.beam ssl_verify_string.beam
ssl_verify_fun.app ssl_verify_fun_encodings.beam ssl_verify_pk.beam ssl_verify_util.beam
deps/ssl_verify_fun/src:
ssl_verify_fingerprint.erl ssl_verify_fun_cert_helpers.erl ssl_verify_hostname.erl ssl_verify_string.erl
ssl_verify_fun.app.src ssl_verify_fun_encodings.erl ssl_verify_pk.erl ssl_verify_util.erl
deps/unicode_util_compat:
ebin/ LICENSE README.md rebar.config src/
deps/unicode_util_compat/ebin:
unicode_util_compat.app unicode_util_compat.beam
deps/unicode_util_compat/src:
unicode_util_compat.app.src unicode_util_compat.erl
Each dependency directory looks like its own mini project directory, with .beam files indicating compiled code. There is also source code, so it looks like the source code is downloaded and then compiled along with the project. I see some .ex files, indicating that some dependencies were written in Elixir, but there are a lot more .erl files, which almost certainly means the dependencies were written in Erlang. It's interesting to see how well Elixir has integrated itself into the Erlang ecosystem. We can use Erlang packages without even knowing that they were written in Erlang.
I notice that unlike Node.js dependencies, all Elixir dependencies are located in the same directory. That made me wonder what would happen if you had two dependencies that were dependent on two different versions of the same package. After some experimentation, I found that if both packages have a version range overlap in both of their mix.exs files, the latest version that matches both ranges will be retrieved.
For example, let's say your project has two dependencies: package A and package B. Both have Package J as a dependency. If package A is dependent on package J 1.4 and later and package B is dependent on package J 1.8 and later, hex will retrieve the latest version of package J and everything will be fine. If instead package A had a dependency on Package J versions 1.4 - 1.9 and package B had a dependency Package J 1.6 - 1.11, I believe that hex would retrieve Package J 1.9, because that's the latest version that both packages could use. I didn't see this specifically mentioned, but I believe this is how it would work based on what I read. On the other hand, if Package A had a dependency on Package J 1.8 and any later minor version and Package B had a dependency on Package J 2.2 and any later minor version, there is no version of Package J that would satisfy both version ranges. There would be an error in that case and you would be unable to build your project.
I've noticed that hex packages tend to keep their dependencies updated and that many of them allow any later minor version when specifying the allowed verson range of their dependencies. This looser specification may mean a greater possibility that a later minor version will break something, but it also means greater flexibility when there are duplicate dependencies.
From what I understand, this one-package-version-allowed behavior is in part due to the way the Erlang VM works. Instead of being an integrated part of your application, dependencies run as a separate entity called an "application" that runs in the Erlang VM. Having multiple versions of the same application in the same context would confuse things. At least, that's my admittedly vague understanding of the situation.
While we're looking at that, I'm going to take a look to see if I can find the allowed version ranges for the dependencies of httpoison. Look, there it is in deps/httpoison/mix.exs.
defp deps do
[
{:hackney, "~> 1.8"},
{:exjsx, "~> 3.1", only: :test},
{:httparrot, "~> 1.0", only: :test},
{:meck, "~> 0.8.2", only: :test},
{:earmark, "~> 1.0", only: :dev},
{:ex_doc, "~> 0.14", only: :dev},
]
end
That definitely confirms my suspicion regarding why the deps.update task updated hackney to the next minor version. The "~> 1.8" version specification indicates that all minor releases starting at version 1.8 are in the allowed version range. The current version of hackney is now 1.15, which is within the allowed version range. I'm feeling I'm understanding better and better how updating works.
It also looks like we can specify dev and test dependencies with only: :test and only: :dev. I'm guessing that it only retrieves and builds those dependencies when running in a :dev or :test environment. I believe a :test environment is only present when running unit tests, and a :dev environment is present during normal development. I'm not sure yet how you would specify a :prod environment instead of a :dev environment.
In the example above, :earmark and :ex_doc are documentation generating tools, so they wouldn't need to be retrieved when not doing development on httpoison. I'm guessing that :httparrot and :meck are probably some kind of mocking frameworks, so they would only be needed during testing.
Git Dependencies
It's also possible to create a dependency from a git repository, where instead of a version number, you specify the git URL. You can also optionally specify a tag, which will likely be some sort of version specification. Hex will download and install the files from the git repository and make it a dependency.
The default mix.exs that is created by "mix new" contains a commented-out example of a git dependency.
{`:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}`.
The dependency is called :dep_from_git and the files associated with the "0.1.0" tag will be pulled from the "https://github.com/elixir-lang/my_dep.git" repository instead of being retrieved from the hex package repository.
There are other options as well, and if you're interested, take a look at https://hexdocs.pm/mix/Mix.Tasks.Deps.html, which documents the options for git dependencies. Git dependencies can be useful for internal private dependencies that you don't want to be made available for the public in the hex package repository.
Conclusion
We've covered a lot about dependencies here. Next time, I'm going to go through creating our own project and writing code that makes use of a dependency. You'll be able to follow along on your own.