Functional languages work best when you create small, simple, reusable functions that can be combined together do accomplish something. This is called function composition.

A good example of this is the higher-order functions in the Enum module: Enum.map/2, Enum.filter/2, Enum.reduce/2, etc. Those functions are heavily reusable, and you just have to combine those with your own function, usually a fairly simple one. The result is two simple functions combined to achieve something greater.

Functional Composition Workflow

A common workflow when composing functions is to pass data to one function, take the result and pass it to another function, pass that result to yet another function, and so forth.

Let's say that we want to transform, filter, and sort a list of data. That would look something like this:

def process_list(list) do
	Enum.reverse(Enum.sort(Enum.filter(Enum.map(list, fn number -> number * 2 end), fn number -> number < 100 end))) end
end

This is really difficult to read. Here's what it's actually doing.

  1. Takes the collection and multiplies each element by 2
  2. Filters out any numbers >= 100
  3. Sorts the collection
  4. Reverses the order of the collection

You can find the recursive implementation in the "composition_examples.exs" file in the "lwm 29 - Composing Functions" folder under the examples folder in the Learn With Me: Elixir repository on Github.

Let's see an example where I call this function in IEx.

iex> c "examples/lwm 29/composition_examples.exs"
[CompositionExamples]
iex> CompositionExamples.process_list([2, 6, 12, 32, 64, 128, 43, 82, -8])
[86, 64, 24, 12, 4, -16]

This is not an easy operation to understand when reading the source code. All the nested function calls are hard to read. Fortunately, since this sort of composition workflow is so common in Elixir, it offers us a much more readable way of doing the same thing.

Functional Composition Using the Pipe Operator

The pipe operator allows the result of one function to be passed to the next function in a readable way. Multiple function calls separated by pipes forms a pipeline, where a product of one stage is piped to the next stage until all stages are complete. A pipe operator always passes the result of the previous function as the first argument of the next function in the pipeline. This is typically data that is being operated on by all functions in the pipeline.

You can think of this like an assembly line, where a product is passed to the next stage in the assembly process, continuing from stage to stage until it is finished.

The pipe operator is |>. A pipe operator "|>" is not the same thing as a pipe character "|", although the pipe operator does contain a pipe character.

Let's see how we can implement the same process_list/1 function shown above using a pipeline set up using the pipe operator

def process_list_pipelined(list) do
	list
	|> Enum.map(fn number -> number * 2 end)
	|> Enum.filter(fn number -> number < 100 end)
	|> Enum.sort()
	|> Enum.reverse()
end

The variable list is passed as the first argument to Enum.map/2, with the second parameter being defined with the call to Enum.map/2. The result of the mapping is then passed as the first parameter to Enum.filter/2. The result of that is passed to Enum.sort/1 and the result of that is passed to Enum.reverse/1. The final result is returned from the function.

This function is so much neater and easier to read than the previous implementation. The first element in the pipeline is the data to be transformed. It is passed as the first parameter to the next stage in the pipeline, which is passed as the first parameter to the next stage, and so forth until it comes out the other end.

Having the data being automatically passed as the first parameter allows us to focus on what we are doing to that data rather than the mechanics of passing the data. The Enum.map/2 and Enum.filter/2 functions have a second parameter that describes how to transform the data, so we just need to specify that data transformation function. Enum.sort/1 and Enum.reverse/1 have a single parameter, so we just need to give a set of empty parentheses.

I've read that you should always use parentheses when calling functions in a pipeline, since things can behave unexpectedly otherwise. I haven't seen the details on why that is, but I've seen that warning enough times to heed it. It's probably some operator precedence thing. Not all the Elixir code I've read so far follows that convention, but it seems like good advice to me.

Let's take a look at calling this pipelined function from IEx.

iex> CompositionExamples.process_list_pipelined([2, 6, 12, 32, 64, 128, 43, 82, -8])
[86, 64, 24, 12, 4, -16]

We get the same result, but this function is a lot easier to maintain. It's also a great example of composing several smaller functions to achieve something.

Similarities in Javascript and C#

To me, the Elixir pipeline example is eerily similar to creating a series of data transformation functions on a Javascript array or for an Enumerable using LINQ in C#.

The Elixir Example:

def process_list(list) do
	list
	|> Enum.map(fn number -> number * 2 end)
	|> Enum.filter(fn number -> number < 100 end)
	|> Enum.sort()
	|> Enum.reverse()
end

The equivalent in Javascript:

function processList(list) {
	return list
		.map(number => number * 2)
		.filter(number = number < 100)
		.sort()
		.reverse();
}

In Javascript, the list is an array. The Javascript sort() and reverse() functions for Javascript arrays annoyingly modify the array in-place rather than return a new one, an inconsistency in Javascript that annoys me. I suppose that this is for the sake of efficiency for large arrays. The real Javascript code would be a bit more complex to account for the in-place sorting and reversing. However, I'm pretending for now that these functions return a new array. Lodash has a much more rational set of functions that will return the transformed data as another array.

The Ramda Javascript library also implements these things in a functional way, so you can write functional Javascript as well. It even has compose and pipe functions that create pipelines. Ramda revolves around the concept of function composition, and in hindsight, learning it really helped me to understand Elixir.

Here's the same thing from above using Ramda. It uses a concept called currying (which I will discuss in future posts, since it's a common functional language feature) to create a function that creates a pipeline. We just have to apply data to the function that's created.

//Create a function to process the list in the same way
const processList = R.pipe(
	R.map(number => number * 2),
	R.filter(number => number < 100),
	R.sort((x, y) => x - y),
	R.reverse);
	
let processedList = processList([1, 20, 80]);

The equivalent in C#:

public List<int> ProcessList(List<int> list)
{
	return list
		.Select(number => number * 2)
		.Where(number => number < 100)
		.OrderBy(number => number)
		.Reverse();
}

Unlike the Javascript native array functions, the LINQ methods in C# are consistent in that they always return a copy of the transformed data rather than modifying the collection in place.

So even though neither C# nor Javascript are functional languages, many functional concepts have been integrated into those langauges.