Accessing multiple (nth) elements/lists from nested lists

Question

I have a nested list to record the results of my program:

• Each result has 2 vectors with varying lengths:
  • list(metric1=c(1, 2), metric2=c(12, 14, 0, 42))
• Each experiment consists of 100 iterations and therefore produces 100 of these results which I combine in another list together with the used settings (e.g. size, method, etc.).
  • list(size=25, method="random", list(metric1=c(...), metric2=c(...)), list(metric1=c(...), metric2=c(...), ...)
• To have a decent sample size, each experiment runs multiple times (e.g. 10 runs) and all of these results are collected in one big list. So the list with size=25 and `method="random" exists 10 times in that outer list.
  • list(list(size=25, method="random", list(metric1=c(...), metric2=c(...)), list(size=25, method="random", list(metric1=c(...), metric2=c(...)), ..., list(size=100, method="min", list(metric1=c(...), metric2=c(...)))

Now my question: How do I actually access certain list across within that big list? For example:

• How do I select all lists where size=25?
• How do I get the first (nth) iteration of each list where size=25 and method="random"?
• How do combine the 100 metric1 vectors (1 for each iteration) where size=25 and method="random"?
• How would I plot the average length (of the 10 runs) of the method2 vector over the 100 iterations, where size=25 and method="random"?

TL;DR: How do I access multiple elements from lists. With matrices, I use m[,3], but I just can't figure out how to do it with lists.

Thank you so much for your time and I would really appreciate any help on this!

Answer 1

Reproducible data:

set.seed(42)
quux <-
  list(
    list(size=4, method="random",
         replicate(4, list(metric1=sample(10, 2), metric2=sample(50, 4)), simplify = FALSE)),
    list(size=4, method="random",
         replicate(4, list(metric1=sample(10, 2), metric2=sample(50, 4)), simplify = FALSE)),
    list(size=7, method="min", # but not really ...
         replicate(7, list(metric1=sample(10, 2), metric2=sample(50, 4)), simplify = FALSE))
  )
str(quux)
# List of 3
#  $ :List of 3
#   ..$ size  : num 4
#   ..$ method: chr "random"
#   ..$       :List of 4
#   .. ..$ :List of 2
#   .. .. ..$ metric1: int [1:2] 1 5
#   .. .. ..$ metric2: int [1:4] 1 25 10 36
#   .. ..$ :List of 2
#   .. .. ..$ metric1: int [1:2] 2 1
#   .. .. ..$ metric2: int [1:4] 47 24 7 36
# ...

1. How do I select all lists where size=25?

   Q1 <- Filter(function(z) z$size == 4, quux)
   str(Q1, max.level=2)
   # List of 2
   #  $ :List of 3
   #   ..$ size  : num 4
   #   ..$ method: chr "random"
   #   ..$       :List of 4
   #  $ :List of 3
   #   ..$ size  : num 4
   #   ..$ method: chr "random"
   #   ..$       :List of 4
   

2. How do I get the first (nth) iteration ...

   nth <- 3
   Q2 <- lapply(Filter(function(z) z$size == 4 && z$method == "random", quux),
                `[[`, nth)
   str(Q2)
   # List of 2
   #  $ :List of 4
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 1 5
   #   .. ..$ metric2: int [1:4] 1 25 10 36
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 2 1
   #   .. ..$ metric2: int [1:4] 47 24 7 36
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 9 5
   #   .. ..$ metric2: int [1:4] 46 20 26 47
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 3 9
   #   .. ..$ metric2: int [1:4] 25 27 36 37
   #  $ :List of 4
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 5 4
   #   .. ..$ metric2: int [1:4] 34 28 40 3
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 10 1
   #   .. ..$ metric2: int [1:4] 42 24 30 43
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 6 8
   #   .. ..$ metric2: int [1:4] 36 4 22 18
   #   ..$ :List of 2
   #   .. ..$ metric1: int [1:2] 5 4
   #   .. ..$ metric2: int [1:4] 34 35 24 23
   

3. How do combine ...

   Q3 <- lapply(Filter(function(z) z$size == 4 && z$method == "random", quux),
                function(z) unlist(lapply(z[[3]], function(y) y$metric1)))
   str(Q3)
   # List of 2
   #  $ : int [1:8] 1 5 2 1 9 5 3 9
   #  $ : int [1:8] 5 4 10 1 6 8 5 4
   

4. ... average length (of the 10 runs) of the metric2 ...

   I don't know what you mean by "plot the average length" (just a scatterplot?), but it doesn't matter: extract in the same way you exacted Q3 and do what you need.