We're going to continue investigating the Kernel module, picking up from where we left off in Part 1. Whereas Part 1 gave brief summaries of groups of functions, we're going to start to look at functions in-depth. I've still grouped them, since that really helps make sense of the huge mishmash of functions that comprises the Kernel module.
Nested Structure Functions
You may remember back in lwm 18 where we covered structs that we covered some functions for updating values in nested structures. That's what these functions are. They're all located in the Kernel module and updating a deeply-nested value is a real pain without the help of these functions. See the section in lwm 18 to see what it takes to update a nested structure without a helpful function and how it compares to using a helpful function.
All of these functions work on structures with key-value pairs. As far as I know of, that includes maps, structs, and keyword lists. These functions need to know the path of the value to update, and that is specified using keys. The value that is being changed can be of any type, but it must be contained in a keyed data structure.
Keys in the property path that is passed to these functions can be of any type. Structs and keyword lists have atom keys, but maps can have keys of any type. So the keys in the path can be of any type.
From what I've gathered from reading the documentation, these functions can be used on anything that implements the Access behavior. I haven't heard of a "behavior" before, but it sounds a lot like a protocol, in that they both sounds like interfaces to some functionality that is implemented in the module associated with the data type. I don't really know, but I imagine that I'll find out more about behaviors and protocols later on.
We're going to go over each of these nested structure functions to examine in detail what each one does.
Kernel.pop_in/1
The Kernel.pop_in/1 function pops a key from a nested structure and returns a value. The path to the key is defined by an expression. From what I understand, this function uses some fancy macro magic thingy to derive the path from the expression, so I'll just treat it as a mysterious black box for now. Understanding the details behind how this works can come later.
Here's an example of using the function. I create some nested maps that describe a data structure. There's a person named "Bob" who has a pet. The pet is a dog, and it's :type property contains dog metadata. The pet also has a name and an age. In real code, it would be better to use %Person, %Pet, and %Dog structs so that we're working with well-defined data structures and make the code easier to read, but this is good enough for this example.
iex> data_struct = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> pop_in(data_struct[:pet][:type][:species])
{"Canis lupus familiaris",
%{
name: "Bob",
pet: %{age: 5, name: "Ruff Ruff", type: %{legs: 4, name: "dog"}}
}}
iex> pop_in(data_struct.pet.type.species)
** (ArgumentError) cannot use pop_in when the last segment is a map/struct field. This would effectively remove the field :species from the map/struct
(elixir) lib/kernel.ex:2575: Kernel.nest_pop_in/3
(elixir) lib/kernel.ex:2561: Kernel.nest_pop_in/2
(elixir) lib/kernel.ex:2582: Kernel.nest_pop_in/3
(elixir) lib/kernel.ex:2561: Kernel.nest_pop_in/2
(elixir) lib/kernel.ex:2582: Kernel.nest_pop_in/3
(elixir) expanding macro: Kernel.pop_in/1
iex:3: (file)
iex> pop_in(data_struct.name)
** (ArgumentError) cannot use pop_in when the last segment is a map/struct field. This would effectively remove the field :name from the map/struct
(elixir) lib/kernel.ex:2575: Kernel.nest_pop_in/3
(elixir) expanding macro: Kernel.pop_in/1
iex:3: (file)
iex> keyword_list = [name: "Bob", pet: [ name: "Ruff Ruff", age: 5]]
[name: "Bob", pet: [name: "Ruff Ruff", age: 5]]
iex> pop_in(keyword_list[:pet][:name])
{"Ruff Ruff", [name: "Bob", pet: [age: 5]]}
I can pop the data_struct.pet.type.species property just fine using the bracket notation. The value of that property is returned along with an updated data structure where that property has been removed. Curiously, the dot notation does not work. I get an error message that doesn't make sense to me. I do want to remove :species from the map, so I don't understand why I can't do this. I wonder if it's just this particular function of if this is the case for the entire family of nested structure functions.
I also verify that I can use this function with keyword lists. It succeeds as expected.
Kernel.pop_in/2
The Kernel.pop_in/2 function is very similar to Kernel.pop_in/1, but instead of magically parsing an expression using a macro, it takes in the data structure as the first parameter and a list of keys as the second parameter. The list of keys creates the path to the value that is to be popped. Whereas the path in Kernel.pop_in/1 has to be defined at compile time, the path in Kernel.pop_in/2 can either be specified at compile time or built dynamically at runtime. That's a nice thing to have if you need to dynamically compute the path before using it.
Here's an example.
iex> data_struct = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> pop_in(data_struct, [:pet, :type, :species])
{"Canis lupus familiaris",
%{
name: "Bob",
pet: %{age: 5, name: "Ruff Ruff", type: %{legs: 4, name: "dog"}}
}}
Kernel.get_and_update_in/2
The Kernel.get_and_update_in/2 function updates a property in a nested data structure using a function to do the updating, and returns the old value.
The update function returns a tuple, the first value of which is returned as part of the "get" and the second value that is used to update the property as part of the "update". Although you would typically pass in the current value as part of the "get", you can give it whatever value you want to.
If the key does not exist, nil is passed in as a value and the whatever the "update" value is that is returned from the function is added to the nested data structure using the path. In other words, if the property doesn't exist, your function will be given the chance to create it.
The update function can also choose to not update the property and remove it instead by returning :pop from the update function.
If all this sounds familiar (and it did to me), it's because this is very similar to the Map.get_and_update/3. It's almost like the same function, but for nested data structures.
#Create a person data structure
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
#Double the number of legs on a dog and return the previous value
iex> get_and_update_in(person[:pet][:type][:legs], &{&1, &1 * 2})
{4,
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}}
#Double the number of legs on a dog using dot notation
iex> get_and_update_in(person.pet.type.legs, &{&1, &1 * 2})
{4,
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}}
#Set the number of ears on a dog to 2. This will add the property, since it doesn't already exist.
#A nil is returned as the "get" value
iex> get_and_update_in(person[:pet][:type][:ears], &{&1, 2})
{nil,
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{ears: 2, legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}}
#Return :pop to remove the property instead of updating it
iex> get_and_update_in(person[:pet][:type][:legs], fn _ -> :pop end)
{4,
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{name: "dog", species: "Canis lupus familiaris"}
}
}}
It's good to see that dot notation works for this function. I'm not sure why Kernel.pop_in/1 had an issue with a path specified using dot notation.
Kernel.get_and_update_in/3
The Kernel.get_and_update_in/3 function is the same as Kernel.get_and_update_in/2, but it accepts a data structure and a path list instead of just using a path expression.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> get_and_update_in(person, [:pet, :type, :legs], &{&1, &1 * 2})
{4,
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}}
Kernel.get_in/2
The Kernel.get_in/2 function retrieves a value from a nested data structure. Compared to some of the other nested structure functions, this one is very simple. It appears that there is only one version of this function: the version that accepts a list of keys. Unlike the other functions, we can just use the dot path syntax as-is to achieve the same thing, so I imagine that's why there's only one version of get_in.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> get_in(person, [:pet, :type, :species])
"Canis lupus familiaris"
iex> person.pet.type.species
"Canis lupus familiaris"
As you can see in the last example, it's easy to retrieve the value in a nested data structure by specifying the path using dot notation without anything else. Kernel.get_in/2 provides a way to specify a path using a keyword list at runtime and the ability to use Access functions (see section discussing Access functions below).
Kernel.put_in/2
The Kernel.put_in/2 function simply updates a property in a nested data structure with a value. It differs from the update family of functions in that it passes a value rather than a function that computes the value. This version of the function uses a path expression.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> put_in(person.pet.type.species, "Doggis droolis")
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Doggis droolis"}
}
}
iex> put_in(person[:pet][:name], "Barky")
%{
name: "Bob",
pet: %{
age: 5,
name: "Barky",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
Kernel.put_in/3
The Kernel.put_in/3 function works just like Kernel.put_in/2, except that it accepts a list that defines the path.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> put_in(person, [:pet, :type, :species], "Doggis droolis")
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Doggis droolis"}
}
}
iex> put_in(person, [:pet, :name], "Barky")
%{
name: "Bob",
pet: %{
age: 5,
name: "Barky",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
Kernel.update_in/2
The Kernel.update_in/2 function updates a property in a nested data structure with a property path expression and a function that computes the updated value. It's similar to the Kernel.get_and_update_in/2 function we read about earlier, but there are some differences. Kernel.update_in/2 returns just the updated data structure. The update function that is passed is different in that it just returns the updated value.
If the property being updated does not already exist, it will be added. Unlike, Kernel.get_and_update_in/2, there doesn't appear to be any option to remove an existing value. If I pass an update function that returns :pop the property will get set to :pop instead of being deleted.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
#Double the number of legs on a dog
iex> update_in(person[:pet][:type][:legs], &(&1 * 2))
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}
#Double the number of legs on a dog using dot notation
iex> update_in(person.pet.type.legs, &(&1 * 2))
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}
#Set the number of ears to 2. This will add the property, since it currently doesn't exist
iex> update_in(person[:pet][:type][:ears], fn _ -> 2 end)
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{ears: 2, legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
#Attempt to delete the property by returning :pop. This does not work.
iex> update_in(person[:pet][:type][:legs], fn _ -> :pop end)
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: :pop, name: "dog", species: "Canis lupus familiaris"}
}
}
As the documentation warns, if an earlier part of the path is missing, the function will throw an error instead of creating the entire path. This does not apply the final property value, which will be created if it's missing, but to anything prior to that.
iex> person = %{name: "Bob"}
%{name: "Bob"}
iex> update_in(person[:pet][:type][:ears], fn _ -> 2 end)
** (ArgumentError) could not put/update key :type on a nil value
(elixir) lib/access.ex:393: Access.get_and_update/3
(elixir) lib/map.ex:791: Map.get_and_update/3
Kernel.update_in/3
The Kernel.update_in/3 function is like Kernel.update_in/2 except that the property path is specified using a list instead of a path expression.
iex> person = %{name: "Bob", pet: %{ type: %{ name: "dog", species: "Canis lupus familiaris", legs: 4}, name: "Ruff Ruff", age: 5}}
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> update_in(person, [:pet, :type, :legs], &(&1 * 2))
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{legs: 8, name: "dog", species: "Canis lupus familiaris"}
}
}
iex> update_in(person, [:pet, :type, :ears], fn _ -> 2 end)
%{
name: "Bob",
pet: %{
age: 5,
name: "Ruff Ruff",
type: %{ears: 2, legs: 4, name: "dog", species: "Canis lupus familiaris"}
}
}
Using Access Functions
Here's where it just got really interesting. The documentation isn't very clear about this, but you can use functions from the Access module in place of keys to be able to get at nested key-value structures that are inside other collection types such as lists and tuples. This means that if I have maps inside a list, I can update their properties as well using nested structure functions. A simple path can only consist of keyed data structures, but using the Access functions, you can drill into non-keyed data structures as well.
In this example, my person structure has a list of pets. I want to do a get_and_update on the types of pets by making them were-pets. Here's me trying with just a key path.
iex> person = %{name: "Martin", pets: [%{type: "dog", name: "Ruff Ruff"}, %{type: "cat", name: "Meowth"}, %{type: "wombat", name: "Bitey"}]}
%{
name: "Martin",
pets: [
%{name: "Ruff Ruff", type: "dog"},
%{name: "Meowth", type: "cat"},
%{name: "Bitey", type: "wombat"}
]
}
iex> get_and_update_in(person, [:pets, :type], fn type -> {type, "were-" <> type} end)
** (FunctionClauseError) no function clause matching in Keyword.get_and_update/4
The following arguments were given to Keyword.get_and_update/4:
# 1
[
%{name: "Ruff Ruff", type: "dog"},
%{name: "Meowth", type: "cat"},
%{name: "Bitey", type: "wombat"}
]
# 2
[]
# 3
:type
# 4
#Function<6.128620087/1 in :erl_eval.expr/5>
Attempted function clauses (showing 3 out of 3):
defp get_and_update([{key, current} | t], acc, key, fun)
defp get_and_update([{_, _} = h | t], acc, key, fun)
defp get_and_update([], acc, key, fun)
(elixir) lib/keyword.ex:268: Keyword.get_and_update/4
(elixir) lib/map.ex:791: Map.get_and_update/3
That didn't work. The person.pets property is not a keyed structure, but a list. So I can't updated the :type property on a list. Fortunately, I can solve this using the Access.all/0 function, which will allow me to update all the elements in a list.
iex> get_and_update_in(person, [:pets, Access.all(), :type], fn type -> {type, "were-" <> type} end)
{["dog", "cat", "wombat"],
%{
name: "Martin",
pets: [
%{name: "Ruff Ruff", type: "were-dog"},
%{name: "Meowth", type: "were-cat"},
%{name: "Bitey", type: "were-wombat"}
]
}}
Ah, that works!. I put a call to Access.all() in the path list and it gave me access to all the elements in the list. So I was able to update all the pets with a single statement. This can only be done on the versions of the nested structure functions that take a list that defines a path. The versions that take a path expression cannot do this.
Let's play around with this a little and have some fun.
This one updates the second pet only.
iex> get_and_update_in(person, [:pets, Access.at(1), :type], fn type -> {type, "were-" <> type} end)
{"cat",
%{
name: "Martin",
pets: [
%{name: "Ruff Ruff", type: "dog"},
%{name: "Meowth", type: "were-cat"},
%{name: "Bitey", type: "wombat"}
]
}}
This one updates all the pets except for the cat.
iex> get_and_update_in(person, [:pets, Access.filter(fn pet -> pet.type != "cat" end), :type], fn type -> {type, "were-" <> type} end)
{["dog", "wombat"],
%{
name: "Martin",
pets: [
%{name: "Ruff Ruff", type: "were-dog"},
%{name: "Meowth", type: "cat"},
%{name: "Bitey", type: "were-wombat"}
]
}}
At first I thought that the value to be updated needed to be in a keyed data structure, as I stated at the very beginning of the nested structure functions section, but that turns out to be not quite correct. I can update other types of data like lists and tuples if I use only Access functions in the key path.
This example uses Access.all/0 to get and update all the values in a list.
iex> get_and_update_in([1, 2, 3, 4, 5], [Access.all()], &{&1, &1 * 2})
{[1, 2, 3, 4, 5], [2, 4, 6, 8, 10]}
This example updates the first item in every tuple in a list using a combination of Access.all/0 and Access.elem/1. Not a key in sight. I can't imagine that this is commonly done because Enum.map/2 does the same thing in an easier-to-understand manner.
iex> get_and_update_in([{1, 2}, {3, 4}, {5, 6}], [Access.all(), Access.elem(0)], &{&1, &1 * 2})
{[1, 3, 5], [{2, 2}, {6, 4}, {10, 6}]}
None of the access functions will let me update all the values inside a list of tuples through. Access.all/0 just works on lists.
iex> get_and_update_in([{1, 2}, {3, 4}, {5, 6}], [Access.all(), Access.all()], &{&1, &1 * 2})
** (RuntimeError) Access.all/0 expected a list, got: {1, 2}
(elixir) lib/access.ex:643: Access.all/3
(elixir) lib/access.ex:647: Access.all/4
I would have to convert the tuples to a lists first to make that work. In fact, I may be able to do that my creating my own accessor function. It's possible to create our own accessor functions and use them. There's a brief example of one in the Kernel.get_and_update_in/2 documentation, but it doesn't go into detail regarding exactly how it works.
I'm not going to go any further into these custom functions right now, but if I really wanted to create one, I'd look at the Elixir source code for the Access module. I have no doubt those examples would help me figure it out.
These nested structure functions sure turned out to be more involved than I thought when I started looking into them!
Continuing with the Kernel Module
I think that's enough for today. I'll go into some other types of functions that are to be found in the Kernel module the next time I cover it.