Data frames

What is it ?

• Table mixing different types of columns (an Excel table...)
• However within a column all values are similar, e.g. numeric, logical, character

  df <- data.frame(label = letters[1:6], 
                   id = 1:6, 
                   value = rnorm(6, mean = 0, sd = 1), 
                   flag=c(TRUE, FALSE),                   # recycling rule
                   stringsAsFactors = FALSE)
  df

  label id      value  flag
1     a  1  1.2682572  TRUE
2     b  2  1.8706980 FALSE
3     c  3 -1.0296620  TRUE
4     d  4  0.8709814 FALSE
5     e  5 -1.1437683  TRUE
6     f  6  0.6079145 FALSE

Data frames

Useful functions

dim(df)   # returns the dimensions of data frame

[1] 6 4

nrow(df)  # number of rows

[1] 6

ncol(df)  # number of columns

[1] 4

Data frames

Access specific value

• Use the df[i,j] notation, first index corresponds to row, second index to column

  df[5,3]

[1] -1.143768

• Specify the name of the column

  df[5,"value"]

[1] -1.143768

Data frames

Access specific column

• Use the df[i,j] notation

  df[,3]

[1]  1.2682572  1.8706980 -1.0296620  0.8709814 -1.1437683  0.6079145

  df[,"value"]

[1]  1.2682572  1.8706980 -1.0296620  0.8709814 -1.1437683  0.6079145

Data frames

This can be used to access a specific value

• $ for the column, [i] for the row

  df$value[5]

[1] -1.143768

Data frames

Access row

• Use the df[i,j] notation

  df[1,]

  label id    value flag
1     a  1 1.268257 TRUE

Data frames

Access specific rows

• Rows for which the value of id <= 3

   df[df$id <= 3,]

  label id     value  flag
1     a  1  1.268257  TRUE
2     b  2  1.870698 FALSE
3     c  3 -1.029662  TRUE

Select lines for which the label is c

   df[df$label ==  "c",]

  label id     value flag
3     c  3 -1.029662 TRUE

This syntax is complicated - tidyverse packages make it much more easy to manipulate and remember

Tidy data

Installation and Resources

Packages

• unzip R-session-02.zip
• Open in R data_wrangling.R

Resources

Tidy data

Basic concepts

1. Each variable must have its own column.
2. Each observation must have its own row.
3. Each value must have its own cell.

Tidy data

Load necessary libraries

library("readxl") # Import the data from Excel file
library("readr")  # Import the data from Excel file
library("dplyr")  # filter and reformat data frames
library("tidyr")  # make data tidy
library("stringr") # manipulate strings
library("lubridate") # manipulate date 
library("ggplot2") # graphics

Oceanographic data

CARBOM cruise off Brazil

Oceanographic data

Microbial populations

Read data

Text file - TAB delimited

Read data

Reading a text file

  samples <- readr::read_tsv("data/CARBOM data.txt")

• readr::read_tsv() : read tab delimited files

• readr::read_csv() : read comma delimited files

• readr::write_tsv() : write tab delimited files

Read data

Excel sheet

Read data

Read the data - read_excel

  samples <- readxl::read_excel("data/CARBOM data.xlsx", 
                           sheet = "Samples_boat")

• Can also select a range : e.g. A1:Q26
• Can skip lines

Write data

Text file

• readr::write_tsv() : write tab delimited files

  readr::write_tsv(samples, "data/CARBOM data fixed.tsv")

Excel file

• openxlsx::write.xlsx : write tab delimited files

• Many options: specific sheet, formatting etc...

  openxlsx::write.xlsx(samples, "data/CARBOM data fixed.xlsx")

dplyr - Manipulate tables

@allison_horst

Manipulate columns

List columns

  colnames(samples)

 [1] "sample number" "transect"      "station"       "date"          "time"          "depth"         "level"         "latitude"      "longitude"     "picoeuks"      "nanoeuks"     
[12] "phosphates"    "nitrates"      "temperature"   "salinity"

Manipulate columns

Summarize columns

  summary(samples$depth)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
    5.0     5.0    50.0    56.6   100.0   140.0       3

Manipulate columns

Select specific columns - select

samples_select <- dplyr::select(samples, transect, `sample number`,
                                 station, depth, latitude, longitude, 
                                 picoeuks, nanoeuks)

Manipulate columns

Select a range of columns - select

  samples_select <- dplyr::select(samples, transect:nanoeuks)

Manipulate columns

Unselect columns - select

  samples_select <- dplyr::select (samples, -nitrates, -phosphates)

Manipulate columns

Using the pipe operator - %>%

  samples_select <- samples %>% dplyr::select(transect:nanoeuks)

• It is cleaner to write on 2 lines

  samples_select <- samples %>% 
    dplyr::select(transect:nanoeuks)

Manipulate columns

Renaming variables - rename

  samples <- samples %>% 
    dplyr::rename(sample_number = `sample number`)

Manipulate columns

Creating new variables - mutate

  samples <- samples %>% 
    dplyr::mutate(pico_pct = picoeuks/(picoeuks+nanoeuks)*100)

• You can also use transmute() but then it will drop all the other columns.
• It is much much better to compute new variables in R than in Excel, because you can easily track and correct errors.

Manipulate columns

Using the pipe operator you can chain operations

  samples_select <- samples  %>% 
    dplyr::select(sample_number:nanoeuks, level) %>% 
    dplyr::mutate(pico_pct = picoeuks/(picoeuks+nanoeuks)*100)

Manipulate columns

Creating labels with mutate and stringr functions

  samples <- samples %>% 
    dplyr::mutate(sample_label = str_c("TR",transect,"St",station, sep="_"))

Manipulating rows

Order rows - arrange

  samples <- samples %>% 
    dplyr::arrange(transect, station)

• Station 6 is not ordered numerically. It is because station is a character column.

Manipulating rows

Summarize rows - count

• Compute number of stations per transect

  stations_count <- samples %>% 
    dplyr::count(transect)

Manipulating rows

Summarize rows - group_by / summarize

• Group by transect and station
• Compute mean of the percent picoplankton

  samples_mean <- samples %>% 
    dplyr::group_by(transect, station) %>% 
    dplyr::summarise(n_samples = n(), 
              mean_pico_percent = mean(pico_pct, na.rm=TRUE))

Manipulating rows

Filtering rows - filter

• Get only the surface samples

  samples_surf <- samples %>% 
    dplyr::filter(level == "Surf" )

• ! Use the logical operators == != > >= < <= is.na()

Next time: Data visualization (ggplot2)

What you will learn :

• Understand the "grammar" of graphics
• Create exploratory graphics

Reading list

• Chapter 28 of R for data science
• Fundamental of data visualization
• Data visualization: practical introduction