Learn With Me: Elixir - The Enum Module Part 2 (#31)

It's time to learn some more about the Enum module. This is Part 2 of learning about the Enum module, picking up from where Part 1 left off.

Enum.fetch!/2

The Enum.fetch!/2 function is similar to the Enum.at/3 function in that it retrieves the value found at a particular index in an enumerable. Unlike Enum.at/3, which returns a nil when a value is not found at that index, Enum.fetch/2 throws an error. That's suggested by the presence of "!" in the function name, which is an Elixir naming convention that indicates that the function throws an error.

Let's see an example of how this function is used.

iex> Enum.fetch!([1, 6, 10, 12], 2)
10
iex> Enum.fetch!([1, 6, 10, 12], 0)
1
iex> Enum.fetch!([1, 6, 10, 12], 3)
12
iex> Enum.fetch!([1, 6, 10, 12], 4)
** (Enum.OutOfBoundsError) out of bounds error
    (elixir) lib/enum.ex:863: Enum.fetch!/2
iex> Enum.fetch!([1, 6, 10, 12], -1)
12
iex> Enum.fetch!([1, 6, 10, 12], -3)
6
iex> Enum.fetch!([1, 6, 10, 12], -4)
1
iex> Enum.fetch!([1, 6, 10, 12], -5)
** (Enum.OutOfBoundsError) out of bounds error
    (elixir) lib/enum.ex:863: Enum.fetch!/2

A negative index indicates that the function should start at the end and count backward. Elixir throws an Enum.OutOfBoundsError when an index does not exist.

Since this function throws an error, it should be used when you do not expect it to ever fail under normal circumstances. If it does fail, then the error that is thrown will attract attention to the situation.

The equivalent in Javascript is just the array indexer.

C# has an equivalent LINQ extension method IEnumerable.ElementAt(), which uses only the methods defined in the IEnumerable interface.

List<int> numberList = new List<int>() {1, 3, 4};

int value = numberList.ElementAt(2);

If the index exists, the value is returned. If the index does not exist, an exception is thrown.

Enum.fetch/2

As its name suggests, the Enum.fetch/2 function is the same as Enum.fetch!/2, but it does not throw an error. I can deduce that from the function name, which is almost the same, but does not have a "!" character in it. By Elixir convention, that means that it does not throw an error.

When Enum.fetch/2 finds an index, it will always return a tuple, with the first value being :ok and the second value being the value that it found at that index. When Enum.fetch/2 sees that an index does not exist, it returns the atom :error. We can use pattern matching to provide different logic depending on whether the function was able to retrieve a value or not.

iex> Enum.fetch([1, 6, 10, 12], 2)
{:ok, 10}
iex> Enum.fetch([1, 6, 10, 12], 0)
{:ok, 1}
iex> Enum.fetch([1, 6, 10, 12], 3)
{:ok, 12}
iex> Enum.fetch([1, 6, 10, 12], 4)
:error
iex> Enum.fetch([1, 6, 10, 12], -1)
{:ok, 12}
iex> Enum.fetch([1, 6, 10, 12], -3)
{:ok, 6}
iex> Enum.fetch([1, 6, 10, 12], -4)
{:ok, 1}
iex> Enum.fetch([1, 6, 10, 12], -5)
:error

Unlike the version that throws errors, Enum.fetch/2 just gives us an atom indicating success or failure. This function should be used when you expect the function to fail to find anything under normal operating conditions. This gives you an opportunity to shape the flow of the logic when Enum.fetch/2 fails to find anything and continue on normally.

The difference between this function and Enum.at/3 is very small, with Enum.at/3 returning nil when an index was not found and allowing you to optionally specify a value that will be returned instead of nil. Enum.fetch/2, on the other hand, always returns :error when an index was not found.

The fetch functions are similar to the array indexer in Javascript and the IEnumerable.ElementAt() LINQ extension function in C#. I already covered these when discussing Enum.at/3 in Part 1.

Javascript does not have an exact equivalent to this function. The closest is the array indexer.

C# does not have an exact equivalent to this function. The closest is IEnumerable.ElementAtOrDefault().

Enum.filter/2

The Enum.filter/2 function is a higher-order function that receives an enumerable and a function, filters the elements in the enumerable, and returns a list containing the elements that met the criteria specified in the parameter function. If the function passed in as a parameter returns a truthy value when it is given an element, the element is kept; otherwise, the element is discarded.

This allows us to extract data from an enumerable that meets the criteria we provide.

Let's see an example of Enum.filter/2 at work.

iex> is_even = &(rem(&1, 2) == 0)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> is_odd = &(rem(&1, 2) == 1)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> Enum.filter([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], is_even)
[2, 4, 6, 8, 10]
iex> Enum.filter([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], is_odd)
[1, 3, 5, 7, 9]
iex> Enum.filter([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], &(&1 <= 5))
[1, 2, 3, 4, 5]
iex> Enum.filter([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], &(&1 * &1 <= 10))
[1, 2, 3]

In the first example, I filtered out the odd elements and kept the even elements. Everything else is discarded. In the second example I used the filter function to get a list of the odd numbers. In the third example, I filtered out anything that was greater than 5. In the fourth example, I filtered out any number whose square is greater than 10.

This function is equivalent to the array filter function in Javascript.

const isEven = number => number % 2 == 0;
let numbers = [1, 2, 3, 4, 5];

const evenNumbers = numbers.filter(isEven);

The C# LINQ extension functions have an equivalent function, IEnumerable.Where().

Func<int, bool> isEven = number => number % 2 == 0;
List<int> numbers = new List<int>() {1, 2, 3, 4, 5};

var evenNumbers = numbers.Where(isEven).ToList();

Enum.find/3

The Enum.find/3 is a higher-order function that returns the first element in an enumerable that meets the criteria, where the criteria are defined using a function that is passed in as a parameter.

Enum.find/3 has three parameters:

The enumerable
The value that will be returned if no elements were found that meet the criteria. This is an optional parameter and defaults to nil.
The function that defines the criteria. The first item that this function returns a truthy value for will be returned from Enum.find/3.

Here are some examples.

#Find the first number whose square is greater than 10
iex> Enum.find(1..10, &(&1 * &1 > 10))
4
#Find the first number greater than 5
iex> Enum.find(1..10, fn number -> number > 5 end)
6
#Find the first even number
iex> is_even = &(rem(&1, 2) == 0)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> Enum.find(1..10, is_even)
2
#Find the first negative number. Since there are no negative numbers, 
#a nil is returned
iex> Enum.find(1..10, fn number -> number < 0 end)
nil
#Find the first negative number, specifying an atom that will be 
#returned if no negative numbers are found
iex> Enum.find(1..10, :not_found, fn number -> number < 0 end)
:not_found

This function is useful when you just want to find just the first item that meets some criteria.

The equivalent to this function in Javascript is the native find() function on an array

let numbers = [1, 2, 3, 4, 5];

//This will return 2, since the first even number is 2
let firstEvenNumber = numbers.find(number => number % 2 === 0)

The C# equivalent is IEnumerable.FirstOrDefault() in the LINQ extension methods.

Func<int, bool> isEven = number => number % 2 == 0;
List<int> numbers = new List<int>() {1, 2, 3, 4, 5};

//This will return 2, since the first even number is 2
var firstEvenNumber = numbers.FirstOrDefault(isEven).ToList();

Enum.find_index/2

The Enum.find_index/2 function works a lot like Enum.find/3. It's also a higher-order function that finds the first element that meets the criteria. The difference is that Enum.find_index/2 returns the index instead of the value and there is no default value parameter.

If no items could be found that match the criteria, the function returns nil.

Here are some examples.

iex> Enum.find_index(1..10, &(&1 * &1 > 10))
3
iex> Enum.find_index(1..10, fn number -> number > 5 end)
5
iex> is_even = &(rem(&1, 2) == 0)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> Enum.find_index(1..10, is_even)
1
iex> Enum.find_index(1..10, fn number -> number < 0 end)
nil

The Javascript equivalent of this function is the findIndex() function on an array.

let numbers = [1, 2, 3, 4, 5];

//This will return 1, since the first even number is at index 1
let firstEvenNumberIndex = numbers.findIndex(number => number % 2 === 0)

The IEnumerable LINQ extension methods in C# do not have an equivalent, but the List type does: List<T>.FindIndex().

Func<int, bool> isEven = number => number % 2 == 0;
List<int> numbers = new List<int>() {1, 2, 3, 4, 5};

//This will return 2, since the first even number is 2
var firstEvenNumber = numbers.FindIndex(isEven).ToList();

Enum.find_value/3

The Enum.find_value/3 is a bit weird. It acts just like Enum.find/3 in that it feeds items from the enumerable into the parameter function until it returns a truthy value, but it returns the truthy value that was returned from the function instead of the item in the enumerable. Like Enum.find/3, it also has an optional parameter for the value that is returned when no items were found for which the parameter function returns a truthy value.

Let's look at an example of using this weird find function variant.

#When the number 5 is passed to the parameter function, it returns true. 
#That true value is what is returned because that was the first truthy value returned
iex> Enum.find_value(1..10, fn number -> number > 5 end)
true
#This parameter function just returns the same item that was passed into it
#So the first truthy item is 0, which is what is returned
iex> Enum.find_value([nil, 0, false, "Bob", nil], &(&1))
0
#The parameter function never returns a truthy value, so a nil is returned
iex> Enum.find_value(1..10, fn number -> number < 0 end)
nil
#The parameter function never returns a truthy value, so a :not_found atom is returned
iex> Enum.find_value(1..10, :not_found, fn number -> number < 0 end)
:not_found

I've been thinking about this function trying to determine how I would use it. I haven't been able to think of an instance where I would use this function instead of another one. It doesn't seem all that useful to me, but I'm sure it's there for a good reason. Perhaps it's an opportunity to transform the data somehow before returning it. I probably just haven't thought of a good use case yet.

I'm unaware of any equivalents to this unusual function in C# or Javascript.

Enum.map/2

So far, I've covered each function in the Enum module in alphabetical order, but I'm going to go out of order occasionally when I need to introduce some functions that are necessary to understand before we dive into other functions. That's happening here, where I jump to the Enum.map/2 function.

We've seen Enum.map/2 make an appearance in earlier topics. It's a higher-order order function that does a one to one transformation of every item in an enumerable, where each item in the enumerable is run though a transformation function that was passed in as a parameter, and the result is added to the list that will be returned. So the number of elements in the result is the same as the number of elements that were passed in.

Here are some examples.

#Map each number to its string equivalent
iex> Enum.map(1..10, &(Integer.to_string(&1)))
["1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]
#Double the numbers in the range
iex> Enum.map(1..10, &(&1 * 2))
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
#Square the numbers in the range
iex> Enum.map(1..10, &(&1 * &1))
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
#Do no transformation at all, just passing back the number unaltered.
#It's useless, but it can be done.
iex> Enum.map(1..10, &(&1))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

Mapping is a common functional operation, and Javascript has an equivalent function: the map() function on an array.

let numbers = [1, 2, 3, 4, 5];

//Double the numbers
let doubleNumbers = numbers.map(number => number * number);

The equivalent in C# is the IEnumerable.Select() in the LINQ extension methods.

List<int> numbers = new List<int>() {1, 2, 3, 4, 5};

//Double the numbers
var doubleNumbers = numbers.Select(number => number * 2).ToList();

Enum.map_every/3

The Enum.map_every/3 function closely resembles Enum.map/2, except that the mapping function is applied to every Nth element instead of every element. The value N is controlled by the second parameter.

The first item in the enumerable is always mapped, unless the second parameter is 0. So if you pass in 3 as the second parameter, the mapping function will be applied to items at indexes 0, 3, 6, 9, etc. until all the items have been iterated over.

If the second parameter is 0, then all the elements in the enumerable will returned in a list as-is, without any mapping having been applied.

Let's look at some examples.

iex> numbers = 1..20
1..20
#Map every 3rd element to a string starting with the first element
iex> Enum.map_every(numbers, 3, &(Integer.to_string(&1)))
["1", 2, 3, "4", 5, 6, "7", 8, 9, "10", 11, 12, "13", 14, 15, "16", 17, 18,
 "19", 20]
#Map every 5th element to a string starting with the first element
iex> Enum.map_every(numbers, 5, &(Integer.to_string(&1)))
["1", 2, 3, 4, 5, "6", 7, 8, 9, 10, "11", 12, 13, 14, 15, "16", 17, 18, 19, 20]
#Map every 2nd element to a string starting with the first element
iex> Enum.map_every(numbers, 2, &(Integer.to_string(&1)))
["1", 2, "3", 4, "5", 6, "7", 8, "9", 10, "11", 12, "13", 14, "15", 16, "17",
 18, "19", 20]
#Map every element to a string starting with the first element
iex> Enum.map_every(numbers, 1, &(Integer.to_string(&1)))
["1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14",
 "15", "16", "17", "18", "19", "20"]
#Passing a 0 as the second parameter means that no items are transformed
iex> Enum.map_every(numbers, 0, &(Integer.to_string(&1)))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
#No negative numbers!
iex> Enum.map_every(numbers, -1, &(Integer.to_string(&1)))
** (FunctionClauseError) no function clause matching in Enum.map_every/3

    The following arguments were given to Enum.map_every/3:

        # 1
        1..20

        # 2
        -1

        # 3
        &:erlang.integer_to_binary/1

    (elixir) lib/enum.ex:1352: Enum.map_every/3

Calling Enum.map_every/3 with a second parameter of 1 is just the same as calling Enum.map/2: all the elements are transformed. Negative numbers won't pattern match with any function clauses, and we'll get a FunctionClauseError when attempting to use a negative number.

Neither Javascript nor C# have an equivalent function.

Enum.flat_map/2

The Enum.flat_map/2 function works just like Enum.map/2 in that it transforms each element to another element. The difference between map and flat_map is that it will "flatten" the results into a single list. It unpacks any enumerables within the main enumerable and places all the elements in the main enumerable. So if the results are a list of lists, flat_map will take the items from all those sub-lists and put them in a single list.

This flat map functionality has been useful to me on several occasions when working in other languages, so I can see why it would be useful to have in Elixir.

Let's take a look at several examples using both Enum.map/2 and Enum.flat_map/2, so that you can compare the difference in behavior between the two.

iex> double = &(&1 * 2)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> data = [1..10, 1..5, 1..10, [44]]
[1..10, 1..5, 1..10, ',']
#Map the data, doubling all the elements in the sub-lists
iex> Enum.map(data, &(Enum.map(&1, double)))
[
  [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
  [2, 4, 6, 8, 10],
  [2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
  'X'
]
#Do the same thing with flat_map. The results are the same, but
#they are "flattened" into a single list
iex> Enum.flat_map(data, &(Enum.map(&1, double)))
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 2, 4, 6, 8, 10, 2, 4, 6, 8, 10, 12, 14, 16,
 18, 20, 88]
#Map the data without transforming it
iex> Enum.map(data, &(&1))
[1..10, 1..5, 1..10, ',']
#Flat map the data without transforming it. The data is the same, but it's
#been enumerated and placed in a single list
iex> Enum.flat_map(data, &(&1))
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
 44]

So the flat map function behaves just like the map function except that it flattens the items in any sub-enumerables to all be in the same list.

The Javascript language does not have an equivalent to Enum.flat_map/2, but Lodash does with its flatMap function.

let data = [[1, 2], [5, -3], [12, 12, 9]];
const double = number => number * 2;

//The resulting data is [2, 4, 10, -6, 24, 24, 18]
flattenedData = _.flatMap(data, double);

C# has an equivalent in the LINQ extension functions, IEnumerable.SelectMany().

var data = new List<List<int>>()
{
	new List<int> {1, 2},
	new List<int> {5, -3},
	new List<int> {12, 12, 9}
};

Func<int, bool> double = number => number * 2;

//The resulting data is {2, 4, 10, -6, 24, 24, 18}
List<int> flattenedData = data.SelectMany(double).ToList();

Enum.join/2

The Enum.join/2 function will convert each item in an enumerable to a binary (this is usually a string) and then join the items together using a separator in between each pair of items, which is also a binary. The enumeration is passed in as the first parameter and the separator is passed in as an optional parameter. If no separator is specified, then the items are just joined together without a separator.

This is the equivalent of a string join function that is commonly found in other languages, and this function is clearly targeted at joining stuff together to form a string. Since a string is a binary in Elixir, any function that will work with strings will also work with binaries, hence why the Elixir documentation talks about binaries rather than strings. However, at least 99% of the time, this function will be applied to string values rather than blobs of bytes.

Let's see this in action, with both strings and binary literals. If you don't know what I've been talking about so far, these examples should make it clear.

#Create a comma-separated string containing all the numbers
iex> Enum.join(1..10, ", ")
"1, 2, 3, 4, 5, 6, 7, 8, 9, 10"
#Create a string where the separator is an "x"
iex> Enum.join(1..10, "x")
"1x2x3x4x5x6x7x8x9x10"
#Join the items together to create a string where we didn't specify a separator
iex> Enum.join(1..10)
"12345678910"
#Join the numbers with a single-byte binary separator
iex> Enum.join(1..10, <<0x01>>)
<<49, 1, 50, 1, 51, 1, 52, 1, 53, 1, 54, 1, 55, 1, 56, 1, 57, 1, 49, 48>>
#Join the numbers with a single-byte binary separator that corresponds to a string character.
iex> Enum.join(1..10, <<0x56>>)
"1V2V3V4V5V6V7V8V9V10"

This function is useful for converting collections of data into strings. Notice that when we specify a binary as the separator, the numbers are still converted to their string character values. When we specify a binary literal that's the equivalent to the "V" character, IEx interprets the results as a string literal instead of a binary literal.

I keep having to remind myself that strings are exactly the same thing as binaries, no matter how IEx chooses to display them. Several times I found myself thinking of strings as being different data types than binaries because of how IEx chose to display them.

An array in Javascript has the equivalent function, also called "join".

let numbers = [1, 2, 3, 4, 5];

//The result is "1, 2, 3, 4, 5"
let joinedString = numbers.join(", ");

C# has an equivalent method, which is a static method located in the String class.

List<int> numbers = new List<int>() {1, 2, 3, 4, 5};

//The result is "1, 2, 3, 4, 5"
string joinedString = string.Join(", ", numbers);

Enum.map_join/3

The Enum.map_join/3 function combines the functionality of Enum.map/2 and Enum.join/2 to create a function that maps and joins in a single pass through the enumerable. It first applies the mapping function to transform the item and then converts that item to a binary and joins it to the other items using a separator. It's the equivalent of calling Enum.map/2 and then feeding the results to Enum.join/2, except that it's more efficient by doing that in a single pass instead of two separate passes.

Enum.map_join/3 requires an enumerable, a binary separator, and a mapping function. Here are some examples.

#Double the numbers and then join them as a comma-separated list
iex> double = &(&1 * 2)
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> Enum.map_join(1..10, ", ", double)
"2, 4, 6, 8, 10, 12, 14, 16, 18, 20"
#Subtract 5 from the numbers and join them like a multiplication expression
iex> Enum.map_join(1..10, " * ", &(&1 - 5))
"-4 * -3 * -2 * -1 * 0 * 1 * 2 * 3 * 4 * 5"
#Subtract 5 from the numbers and don't provide a separator.
iex> Enum.map_join(1..10, &(&1 - 5))
"-4-3-2-1012345"

That's a pretty interesting function, allowing us to do two thing in one pass. It's a more specialized function, so I doubt I'll use it anywhere near as often calling Enum.map/2 and Enum.join/2 individually. I'll need to do an individual map or a join operation much more often than I will need to do both at the same time.

That's not the only function that combines a map operation with another operation. There's also Enum.map_reduce/3, but before we look at that function, we need to understand what a reduce operation is.

Neither Javascript nor C# have any functions that combine map and join functionality.

Enum.reduce/3

The Enum.reduce/3 function implements a reduce operation. A reduce operation (sometimes called "fold" or "aggregate") iterates over a collection, applies a function to each item in the collection, and keeps a stateful object called an accumulator that is returned by one application of the function and is passed to the next. This allows the reduce operation to build up some sort of data in the accumulator, which is finally returned as the result.

A reduce function is more generic than other functions such as Enum.map/2 or Enum.filter/2 as it allows you to decide what you do with the data that is derived for each item in the collection. In fact, you could implement a lot of more specific functions with reduce such as counting, summing, mapping, filtering, etc. A reduce function is what you turn to when none of the other functions will do what you want. It's harder to read and understand than more specific functions, but it's much more flexible.

Reduce can reduce a collection to a single number (such as a max, sum, or count), or it can reduce it to another collection. Despite the name "reduce", the data that comes out of a reduce function does not have to be smaller than the original collection. You have control over what sort of data a reduce function returns.

The Enum.reduce/3 function has three parameters.

The enumerable
The initial accumulator to be passed into the parameter function when it is called on the first item
The function to be called for every item in the enumerable. The first parameter is the current item and the second parameter is the accumulator that was returned from the previous call to this function.

So the reduce operation will work like this.

The function that is passed in will be called, being passed the first item and the initial accumulator that was passed in as a parameter.
The function will return an accumulator value that will be passed to itself when the function is called for the next item
The function is called with the next item and the accumulator from the previous function call.
This will continue until there are no more elements in the enumerable.
The accumulator that was returned from the final function call will be returned from the reduce function.

That's all there is to reduce. A reduce operation is the same in Elixir as it is in many other languages. It tends to be less common in imperative languages and more common in functional languages. The reduce operation is something that took a long time for me to wrap my head around when I first learned it when doing functional Javascript programming. It wasn't until I looked at some examples and started playing around with it did I gain an understanding of how and when to use it.

Let's look at some examples. I'm going to use Enum.reduce/3 to calculate the sum of a collection, to find the max number in a collection, to filter a collection, and to duplicate the numbers in a collection. Finally, I'm going to create a more complicated function that returns a tuple that contains both the average of the numbers in the collection and a list containing the string representation of all the numbers.

Let's start off with calculating the sum of an enumerable:

iex> iex> Enum.reduce(1..10, 0, fn (number, sum) -> sum + number end)
55

When the parameter function is first called, it is passed the value 1 as the first parameter, since this is the current item in the enumerable, and it is passed the value 0 as the second parameter, since this was what I passed into the reduce function as the initial accumulator value. This makes sense, since the sum so far is 0. I add the current item (1) to the sum so far (0), which creates a new sum. The new sum is returned as the accumulator value.

The next time, the same function is called with the value 2 as the first parameter, since this is the current item in the enumerable, and it is passed the value 1 as the second parameter, since that was the sum that was returned that last time we called the function. The function adds the current item with the sum so far (1 + 2), and returns the new sum, which is 3. The value 3 will be passed in as the accumulator the next time the function is called.

Finally, after the paramter function has been applied to all the elements in the enumerable, the value 55 is returned from the last call to the function when the last item has been reached. That's the value that will be returned from the call to reduce.

I called the accumulator parameter "sum" in this code, since that's what the accumulator represents. However, you'll often seen it called "acc", since that's short for "accumulator". I prefer to name the accumulator parameter something more meaningful to make the function a little easier to understand, but I'll fall back to "acc" if I can't come up with a good name. Naming things can be hard.

I can actually improve upon this example a little. Many operators in Elixir are actually just shortcuts for functions in the Kernel module. We can actually use the function that corresponds to the addition operator "+", which is +/2. Yes, that is an actual function that adds two numbers together and it is called by the code whenever we use an addition operator. So sum = 1 + 2 is equivalent to sum = Kernel.+(1, 2). We have to refer to it using the Kernel module name or it is assumed to be the "+" operator, which can't be called like a function directly.

iex> 1 + 2
3
iex> +(1, 2)
** (SyntaxError) iex:2: syntax error before: ')'

iex> Kernel.+(1,2)
3

Since the addition function already does the same thing that my anonymous function did in the reduce operation to sum the numbers, I can just reuse the already-existing addition function that's part of the Elixir language.

iex> Enum.reduce(1..10, 0, fn (number, sum) -> sum + number end)
55
iex> Enum.reduce(1..10, 0, &Kernel.+/2)
55

Elixir almost certainly noticed the two functions were the same and just referred to &Kernel.+/2 behind the scenes, but it's much better to make that explicit and save yourself some coding.

Now for the next example, finding the max value in an enumerable.

iex> Enum.reduce(1..10, 0, fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end
...> )
10
iex> Enum.reduce([3, -4, 12, 8, 0, -1, 10], 0, fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end)
12

This parameter function has two clauses, one for when the current item is larger than the max number so far, where the current item will be passed back as the new max number so far, and one for when the number does not qualify to be the new max number so far. The max number will make it to the very end and then be returned from the reduce function.

The ability to pass an accumulator down the line of function calls is very powerful, and allows us to implement a wide variety of operations on an enumerable.

You also may have noticed that I passed in a 0 for the initial accumulator value. That works if any of the numbers are positive, but it won't work if the numbers are all negative.

iex> Enum.reduce([-3, -4, -12, -8, -1, -10], 0, fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end)
0

In a language like C#, where there are integer data types of a fixed bit size and there is a smallest possible value, I just make the initial value the smallest possible integer. That won't work here. Elixir can store any integer of any size as long as it can allocate enough memory to hold it. So there's no smallest possible value. For now, the only solution is to retrieve the first element in the enumerable and use that as the initial value.

iex> data = [-3, -4, -12, -8, -1, -10]
[-3, -4, -12, -8, -1, -10]
iex> Enum.reduce(data, Enum.at(data, 0), fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end)
-1

That solves our problem, but it just isn't as elegant. It returns nil if an empty enumerable is used, which is what we want. The Enum.at/2 function returns nil when the index does not exist. Since the enumerable doesn't have any items, the initial accumulator value is returned.

The next example is filtering a collection for even numbers.

iex> data = [3, -4, 12, 8, 0, -1, 10]
[3, -4, 12, 8, 0, -1, 10]
iex> Enum.reverse(Enum.reduce(data, [], fn
...>    (number, filtered_numbers) when rem(number, 2) == 0 -> [number | filtered_numbers]
...>    (number, filtered_numbers) when rem(abs(number), 2) == 1 -> filtered_numbers
...> end))
[-4, 12, 8, 0, 10]

The initial accumulator is an empty list. When the current item is an even number, I prepend it to the previous accumulator list. If the current item is odd, I ignore it and return the previous accumulator list. By the time the reduce operation is complete, I've constructed a list containing only even numbers. Since I'm prepending, the list is reversed, so I had to add a call to Enum.reverse/1.

I initially messed this one up because I forgot that dividing an odd negative number by 2 will result in -1. I solved that by taking the absolute value of the number using the abs/1 function.

For the next example, I'm going to use the reduce function to double every item in the enumerable.

iex> data = [3, -4, 12, 8, 0, -1, 10]
[3, -4, 12, 8, 0, -1, 10]
iex> Enum.reverse(Enum.reduce(data, [], fn
...>    (number, acc) -> [number, number | acc]
...> end))
[3, 3, -4, -4, 12, 12, 8, 8, 0, 0, -1, -1, 10, 10]

This time I used "acc" for the accumulator name, since that's often what it's called. In each call to the parameter function, the function adds the current item to the accumulator list twice. Since we prepend to the list, we have to reverse it all at the end with a call to Enum.reverse/1.

For the final reduce example, I created a function that returns a tuple that contains both the average of the numbers in the collection and a list containing the string representation of all the numbers. This means that an accumulator will have to store the average of the numbers so far, a count of the numbers that represent that average, and the transformed list of items.

I'm going to store these in a map for the sake of clarity because maps have keys that will associate the value with a meaningful name, making the code easier to read. I could also have easily used a tuple in this situation. Although modifying tuples is generally more expensive than modifying a map, it isn't a big deal here because the tuple would be so small. It doesn't take much to copy a tiny tuple and it's probably about the same performance as updating a tiny map. It's at larger sizes where the difference becomes significant, although I don't have any data on how large.

I'm used to calculating averages all at once by adding up all the numbers and dividing the total by the number count. That's not possible here, so I had to look up how to calculate the average incrementally, like I wanted to do in the reduce operation. It turns out that the formula to calculate the average incrementally is `new_average = current_average + ((new_number - current_average) / new_count). I'll be using that in the reduce function.

Here's what I came up with.

iex> data = [3, -4, 12, 8, 0, -1, 10]
[3, -4, 12, 8, 0, -1, 10]
result = Enum.reduce(data, %{avg: Enum.at(data, 0), count: 0, list: []}, fn
...>    (number, %{avg: average, count: count, list: list}) -> %{
...>            avg: average + ((number - average) / (count + 1)),
...>            count: count + 1,
...>            list: [Integer.to_string(number) | list]
...>    }
...> end)
%{avg: 4.0, count: 7, list: ["10", "-1", "0", "8", "12", "-4", "3"]}
iex> result = %{ result | list: Enum.reverse(result.list) }
%{avg: 4.0, count: 7, list: ["3", "-4", "12", "8", "0", "-1", "10"]}

I pass the reduce function an initial accumulator where the average is the first element in the enumerator, the count is initially 0, and the list of transformed numbers is empty. When the parameter function is called, I use pattern matching to extract the data I need from the map and create a new map with the new average, the new count, and a new item prepended to the list.

At the end, I got the correct results, but the list was reversed due to prepending. Updating the result with a reversal of that list fixed that problem.

Some of these examples needed to use the first element of the enumerable as the initial accumulator, and I had to call Enum.at/2 to get it. We can simplify things a bit in this case by using Enum.reduce/2, which we will cover next.

Javascript has a reduce() function on the array. Here's an example of summing an array using reduce().

numbers = [1, 2, 3, 4, 5];

//The result will be 15
let sum = numbers.reduce((sum, number) => sum + number, 0);

C# has a reduce operation in the LINQ extension methods, but it's called IEnumerable.Aggregate().

Here's the same example of summing a list of numbers using IEnumerable.Aggregate().

List<int> numbers = new List<int> {1, 2, 3, 4, 5};

//The result will be 15
int sumOfNumbers = numbers.Aggregate(0, (sum, number) => sum + number, sum => sum);

In addition to the initial accumulator value and the accumulator function, IEnumerable.Aggregate() offers a third parameter that can transform the final accumulator into something else. So if the accumulator contains data that isn't needed for the final result, you can transform it into something more useful before it is returned. I didn't need to do any transformation, so I just returned the accumulator as-is. There's another version of the IEnumerable.Aggregate() function that does not require the third parameter, so I could have used that instead, but I thought I would demonstrate it here since it's a nice feature.

Enum.reduce/2

The Enum.reduce/2 function functions just like Enum.reduce/3, except that it's missing the initial accumulator parameter. Instead of being passed the initial accumulator value, it takes the first element in the enumerable as the initial accumulator value. If the enumerable passed to Enum.reduce/2 is empty, then an Enum.EmptyError error is thrown.

Let's go back and look at the above example where I used Enum.reduce/3 to calculate the max value.

iex> data = [-3, -4, -12, -8, -1, -10]
[-3, -4, -12, -8, -1, -10]
iex> Enum.reduce(data, Enum.at(data, 0), fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end)
-1

I had to set the inital accumulator value to the first item in the list. I could have done this more easily with Enum.reduce/2.

iex> data = [-3, -4, -12, -8, -1, -10]
[-3, -4, -12, -8, -1, -10]
iex> Enum.reduce(data, fn
...>     (number, max) when number <= max -> max
...>     (number, max) when number > max -> number
...> end)
-1

The initial value of the accumulator is automatically set to the first item and the reduce function is then run starting with the second item. If I had passed it an enumerable with a single item, it would just pass me back the first item (the final accumulator value) without ever calling the reduce function.

iex> Enum.reduce([8], fn
...>     (number, max) when number <= max -> max
...>     (number, max) when number > max -> number
...> end)
8

Calls to reduce tend to be a bit more difficult to read than a call to a more specialized function, so it's best to put encapsulate these in another function that will make our code so much easier to understand. It also saves a lot of code if we ever want to reuse the functionality.

iex> max = fn enumerable -> Enum.reduce(enumerable, fn
...>        (number, max) when number <= max -> max
...>        (number, max) when number > max -> number
...>    end)
...> end
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> max.([-3, -4, -12, -8, -1, -10])
-1
iex> max.([5])
5
iex> max.([3, -4, 12, 8, 0, -1, 10])
12

I put the reduce operation in a function called max and started calling that function. That's so much easier to understand! This reduces the cognitive load to reading code when you see "max" and immediately know what it does rather than having to analyze what a reduce operation is doing every time. The fact that the call to reduce is encapsulated in a function called "max" also gives you a big clue about what it is doing when you are looking at the call to reduce.

Javascript has a reduce() function on the array, and the initial accumulator parameter is optional, so it can be equivalent to both Enum.reduce/3 and Enum.reduce/2. Here's the same example of summing an array using reduce() that I showed previously, but without the initial accumulator value.

numbers = [1, 2, 3, 4, 5];

//The result will be 15
let sum = numbers.reduce((sum, number) => sum + number);

C# also has a version of IEnumerable.Aggregate() that does not require an initial accumulator value. Here's the same example that I showed previously, but without the initial accumulator value and without the third transformation parameter.

Here's the same example of summing a list of numbers using IEnumerable.Aggregate().

List<int> numbers = new List<int> {1, 2, 3, 4, 5};

//The result will be 15
int sumOfNumbers = numbers.Aggregate((sum, number) => sum + number);

Enum.reduce_while/3

The Enum.reduce_while/3 function is like Enum.reduce/3, except that it is interruptable. You can stop the reduce operation somewhere in the middle of iterating over the enumerable if the reduce function returns a particular value. The value of the accumulator at that point becomes the final result.

When the reduce function wants the reduce operation to continue, it must return {:cont, acc}, where acc is the accumulator. When the reduce function wants to stop the reduce operation, it must return {:halt, acc}.

I created an example. It's the same "max" example from previous reduce examples, but this time I wrote it using Enum.reduce_while/3. It finds the maximum number, but stops when it encounters an item in the enumerable that is not an integer.

iex> max = fn enumerable -> Enum.reduce_while(enumerable, nil, fn
...>            (number, nil) when is_integer(number) -> {:cont, number}
...>            (number, max) when is_integer(number) and number <= max -> {:cont, max}
...>            (number, max) when is_integer(number) and number > max -> {:cont, number}
...>            (_, max) when is_integer(max) -> {:halt, max}
...>            (_, nil) -> {:halt, nil}
...>    end)
...> end
#Function<6.99386804/1 in :erl_eval.expr/5>
iex> max.([3, -4, 12, 8, 0, -1, 10])
12
iex> max.([])
nil
iex> max.([2])
2
iex> max.([2, 5, -4, "Bob", :ok, 18, 6])
5
iex> max.(["Bob", :ok, 4, 8, 31])
nil

This example has a lot more function clauses. I'm going to go over each clause.

Clause 1: Matches when the first item in the enumerable is an integer and the initial accumulator value is nil. It always returns the first item as the max value.
Clause 2: Matches when the previous max value is still the max value
Clause 3: Matches when the current item is greater than the previous max value
Clause 4: Matches when the current item is not an integer (I know that because it would have matched a previous clause if that were the case), but the previous max value is an integer. This is the case when an enumerable has at least one integer at the beginning and then it hits a non-integer value later on. In this case, we return the previous max value and pass back a :halt, telling the reduce operation to stop.
Clause 5: Matches when the current item is not an integer (it would have matched a previous clause) and the previous max value is a nil. This tells us that the very first item in the enumerable is not an integer, since that's the only time when the previous max value would be nil. In this case, we tell the reduce operation to stop and return a nil value.

I ran some tests with various scenarios, and it looks as if the reduce operation behaves as intended.

Neither Javascript nor C# have an equivalent to this function.

Enum.map_reduce/3

The Enum.map_reduce/3 function combines the functionality of map with the functionality of reduce. It works just like Enum.reduce/3 except that the parameter function returns two values in a tuple: a mapped value and an accumulator. The accumulator is passed to the next call to the function as normal, and the mapped value is stored in a list that is returned at the very end.

This allows us to do map and reduce operations with a single pass over the enumerable.

Let's look at two examples that I created for the reduce examples: a sum function and a max function, except that I've altered them to also return a mapped value.

Here's the example of creating a sum using Enum.reduce/3

iex> Enum.reduce(1..10, 0, fn (number, sum) -> sum + number end)
55

Here's the same thing using Enum.map_reduce/3. In addition to computing the sum, I also build a mapped list of doubled values from the original list.

iex> Enum.map_reduce(1..10, 0, fn (number, sum) -> {number * 2, sum + number} end)
{[2, 4, 6, 8, 10, 12, 14, 16, 18, 20], 55}

Here's an example of finding the max using Enum.reduce/3.

iex> Enum.reduce([3, -4, 12, 8, 0, -1, 10], 0, fn
...>    (number, max) when number <= max -> max
...>    (number, max) when number > max -> number
...> end)
12

Here's the same thing using Enum.map_reduce/3. In addition to computing the max, I also build a list of the maximum value as we move through the list. This is something couldn't be done with a simple map operation, but there's no need to behave strictly like a map operation. We can put anything we like as the mapped value.

iex> Enum.map_reduce([3, -4, 12, 8, 0, -1, 10], 0, fn
...>    (number, max) when number <= max -> {max, max}
...>    (number, max) when number > max -> {number, number}
...> end)
{[3, 3, 12, 12, 12, 12, 12], 12}
iex> Enum.map_reduce([3, -4, 6, 8, 0, -1, 10], 0, fn
...>    (number, max) when number <= max -> {max, max}
...>    (number, max) when number > max -> {number, number}
...> end)
{[3, 3, 6, 8, 8, 8, 10], 10}

The first value in the tuple is a list of the current max values at each iteration through the enumerable and the second value is the max value at the end. This is an interesting way of looking at what all the accumulator values are at each step in the reduce function. This isn't the primary purpose of Enum.map_reduce/3, but it can be used that way.

Neither Javascript nor C# have any functions that combine map and reduce functionality, although you could accomplish the same thing with reduce functionality. It would just take extra code.

Enum.flat_map_reduce/3

The Enum.flat_map_reduce/3 function works just like Enum.map_reduce/3 except that it flattens the results, like Enum.flat_map/2 does. That's all there is too it.

Here's an example with Enum.map_reduce/3 that produces a list of enumerables (ranges in this case) and sums the enumerable at the same time.

iex> Enum.map_reduce(1..10, 0, fn (number, acc) -> {1..number, number + acc} end)
{[1..1, 1..2, 1..3, 1..4, 1..5, 1..6, 1..7, 1..8, 1..9, 1..10], 55}

Here's the same thing, but using Enum.flat_map_reduce/3.

iex> Enum.flat_map_reduce(1..10, 0, fn (number, acc) -> {1..number, number + acc} end)
{[1, 1, 2, 1, 2, 3, 1, 2, 3, 4, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5,
  6, 7, 1, 2, 3, 4, 5, 6, 7, 8, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 2, 3, 4, ...], 55}

The result represents the same elements as the previous example, but the ranges were flattened so that all the elements are in a single list.

Neither Javascript nor C# have any functions that combine map and reduce functionality, although you could accomplish the same thing with reduce functionality. It would just take extra code.

Enum.group_by/3

The [Enum.group_by/3] function groups data in an enumerable into a map where each group is a key-value pair. The key is whatever value the items in that group have in common, and the value is a list of the items in that group.

Enum.group_by/3 takes three parameters:

An enumerable
A function that determines the group key of an item. The key determines which group the item is placed in.
An optional function that transforms an item before it is added to a group

The best way to understand this function is through example. Here's an example of grouping a range of numbers by whether they are even or odd.

iex> Enum.group_by(1..10, fn
...>    number when rem(number, 2) == 0 -> :even
...>    _ -> :odd
...> end)
%{even: [2, 4, 6, 8, 10], odd: [1, 3, 5, 7, 9]}

When the number is even, the function returns :even. Otherwise, the function return :odd. Since there are only two possible keys, there are only two possible groups. This nicely separates the numbers into two groups: one for even numbers and one for odd numbers.

I did not use the value transformation function in that example, so here's the same example with the numbers transformed into strings.

iex> Enum.group_by(1..10, fn
...>    number when rem(number, 2) == 0 -> :even
...>    _ -> :odd
...> end,
...> fn number -> Integer.to_string(number) end)
%{even: ["2", "4", "6", "8", "10"], odd: ["1", "3", "5", "7", "9"]}

Here's another example where I group strings by their length. This is a variant of the example shown in the Elixir documentation.

iex> words = ["platypus", "cat", "Zeitreisenverbrechensverfahren", "monkey", "dog", "a", "eat", "cattle", "snoep"]
["platypus", "cat", "Zeitreisenverbrechensverfahren", "monkey", "dog", "a",
 "eat", "cattle", "snoep"]
iex> Enum.group_by(words, &String.length/1)
%{
  1 => ["a"],
  3 => ["cat", "dog", "eat"],
  5 => ["snoep"],
  6 => ["monkey", "cattle"],
  8 => ["platypus"],
  30 => ["Zeitreisenverbrechensverfahren"]
}

I didn't even have to define my own group key function. There was a named function available in the String module I could reuse.

Now I'm going to have some fun (as I weren't already!), and group numbers by their Fizzbuzz category. In Fizzbuzz there are four groups: normal numbers, fizz numbers, buzz numbers, and fizzbuzz numbers. I'm going to used Enum.group_by/3 to group them.

iex> Enum.group_by(1..20, fn
...>    number when rem(number, 3) == 0 and rem(number, 5) == 0 -> "fizzbuzz"
...>    number when rem(number, 3) == 0 -> "fizz"
...>    number when rem(number, 5) == 0 -> "buzz"
...>    _ -> :normal
...> end)
%{
  :normal => [1, 2, 4, 7, 8, 11, 13, 14, 16, 17, 19],
  "buzz" => [5, 10, 20],
  "fizz" => [3, 6, 9, 12, 18],
  "fizzbuzz" => [15]
}

Each function clause corresponds to a different group key and pattern matching does the rest. Yay! That was fun!

Enum.intersperse/2

The Enum.intersperse/2 function intersperses an element between each element in the enumeration. I honestly can't think of a use case for this function at the moment, but it certainly looks like it would be useful in situations where you have to intersperse the same element throughout some data.

Here's an example.

iex> data = [3, -4, 12, 8, 0, -1, 10]
[3, -4, 12, 8, 0, -1, 10]
iex> Enum.intersperse(data, nil)
[3, nil, -4, nil, 12, nil, 8, nil, 0, nil, -1, nil, 10]
iex> Enum.intersperse(data, "*")
[3, "*", -4, "*", 12, "*", 8, "*", 0, "*", -1, "*", 10]
iex> Enum.intersperse(data, 1)
[3, 1, -4, 1, 12, 1, 8, 1, 0, 1, -1, 1, 10]
iex> Enum.intersperse(["tick", "tick", "tick"], "tock")
["tick", "tock", "tick", "tock", "tick"]

I'm unaware of any equivalent function in Javascript or C#.

Once again, we've gone over a lot of functions in the Enum module. We're halfway through now. This module as a lot of functions! I'm going to take a break from going over the Enum module and go over something else next time.

Learn With Me: Elixir - The Enum Module Part 2 (#31)

Kevin Peter

Kevin Peter

Enum.fetch!/2

Enum.fetch/2

Enum.filter/2

Enum.find/3

Enum.find_index/2

Enum.find_value/3

Enum.map/2

Enum.map_every/3

Enum.flat_map/2

Enum.join/2

Enum.map_join/3

Enum.reduce/3

Enum.reduce/2

Enum.reduce_while/3

Enum.map_reduce/3

Enum.flat_map_reduce/3

Enum.group_by/3

Enum.intersperse/2

Learn With Me: Elixir - ElixirLargeSort IntSort Project Part 4 (#82)

Learn With Me: Elixir - ElixirLargeSort IntSort Project Part 3 (#81)

Learn With Me: Elixir - ElixirLargeSort IntSort Project Part 2 (#80)

Learn With Me: Elixir - Currying and Partial Application (#32)

Learn With Me: Elixir - FizzBuzz (#30)