How can we upload multiple files in a DRY way?

library(tidyverse)
library(here)

Option 1: downloading files to a folder, and uploading from there.

This is the option you’re familiar with from the Week 7 lab (assessment starter). The downloaded files have been put into the consecutive_data folder, within the data folder.

files <- list.files(here("data", "consecutive_data"), pattern = "csv")

consecutive_data <- files %>% 
  map_dfr(~read_csv(here("data", "consecutive_data", .)))

Option 2: Downloading from the website and uploading as R objects

I’ll use this script to show how we can load in multiple data files using their URLs, all listed within a single csv file.

We’ll start by loading a csv file with URLs of the datasets, and turning it into a vector:

links <- read_csv(here("file_links.csv")) %>% 
  select(link) %>% 
  pull()

We’ll then use map to load all files from the list. This uses map and the list_rbind function, as per https://r4ds.hadley.nz/iteration.html#purrrmap-and-list_rbind (this is the current recommended way)

joined_data <- links %>% 
  map(read_csv) %>% 
  list_rbind()

Alternatively, we could use map_dfr, like this (but, map_dfr has been superseded, which means that it’s no longer the most recommended option:

joined_data_with_just_map <- links %>% 
  map_dfr(read_csv)