Today I made improvements to some R code in my Destiny 2 hobby-coding-verse after learning how to much more cleanly deal with nested lists. I had previously used a solution using map()
to apply a selector to each item in the list, but this was clunky, hard to remember, and became really hard to read with several levels of a nested list.
The far better solution is the unnest_auto
function from {tidyr}, which I came upon when tinkering with the last.fm API data recently. Once I understood how it works, it’s so easy and satisfying! The key is to first make a named tibble.
> tibble(my_tibble = instanced)
# A tibble: 906 × 1
my_tibble
<named list>
1 <named list [10]>
2 <named list [10]>
3 <named list [10]>
4 <named list [10]>
5 <named list [12]>
6 <named list [10]>
7 <named list [10]>
8 <named list [10]>
9 <named list [9]>
10 <named list [9]>
# … with 896 more rows
That nice tibble can be operated on by unnest_auto()
:
> tibble(my_tibble = instanced) %>% unnest_auto(my_tibble) %>%
select(itemLevel, breakerType)
Using `unnest_wider(my_tibble)`; elements have 8 names in common
# A tibble: 906 × 2
itemLevel breakerType
<int> <int>
1 132 NA
2 133 NA
3 133 NA
4 132 NA
5 133 3
6 132 NA
7 133 NA
8 133 NA
9 0 NA
10 0 NA
# … with 896 more rows
(I selected just a couple of columns for readability there; if you don’t do that, you’ll receive all fields at the current list level, including additional nested lists if they exist). After figuring this out, I realized that I also needed to keep the names of each list element, because they constitute a unique ID for the element returned from the API query, and I banged my head a bit on trying to do that as a part of the unnest operation, before I backed up, recentered on the outcome I wanted to produce, and realized I could do it really cleanly using mutate()
! The final code looks like this:
> tibble(my_tibble = instanced) %>% unnest_auto(my_tibble) %>%
select(itemLevel, breakerType) %>% mutate(id = names(instanced))
Using `unnest_wider(my_tibble)`; elements have 8 names in common
# A tibble: 906 × 3
itemLevel breakerType id
<int> <int> <chr>
1 132 NA 6917529338105913753
2 133 NA 6917529550016812142
3 133 NA 6917529281178546429
4 132 NA 6917529231127188610
5 133 3 6917529301504642848
6 132 NA 6917529234625200021
7 133 NA 6917529193313832065
8 133 NA 6917529182955737017
9 0 NA 6917529417667489181
10 0 NA 6917529490520758715
# … with 896 more rows
The D2 API returns a ton of nested lists, so this simplified, accessible and effective tool is 100% getting a featured spot in my toolbox.