Summary

Often data sets we deal include date and time variables. Dealing with dates and time variables accurately can be a complicated task due to the variety in formats, time-zone differences and leap years. In this section you will be introduced to the basics of date manipulations (i.e., getting current date & time, converting strings to dates, extracting and manipulating dates, and date arithmetic) using Base R and lubridate functions.

String/character manipulations are often overlooked in data analysis because the focus typically remains on numeric values. However, the growth in text mining resulted in greater emphasis on handling, cleaning and processing character strings. In the second part of this module I will give the foundation of working with characters by covering string manipulation with Base R and stringr and the set operations for character strings.

In preparation of this section, I heavily used our recommended textbooks (Boehmke (2016) and Wickham and Grolemund (2016)), lubridate and stringr reference manuals.

Learning Objectives

The learning objectives of this module are as follows:

• Apply basic date-time manipulations using Base R functions.
• Apply basic date-time manipulations using lubridate functions.
• Learn basic string manipulations using Base R functions.
• Learn basic string manipulations using stringr functions.

Creating Strings

The most basic way to create strings is to use quotation marks and assign a string to an object similar to creating number sequences like this:

a <- "MATH2349"    # create string a
b <- "is awesome"     # create string b

The paste() function under Base R is used for creating and building strings. It takes one or more R objects, converts them to character, and then it concatenates (pastes) them to form one or several character strings.

Here are some examples of paste() function:

# paste together string a & b

paste(a, b)

## [1] "MATH2349 is awesome"

# paste character and number strings (converts numbers to character class)

paste("The life of", pi)           

## [1] "The life of 3.14159265358979"

# paste multiple strings

paste("I", "love", "Data Preprocessing")            

## [1] "I love Data Preprocessing"

# paste multiple strings with a separating character

paste("I", "love", "Data", "Preprocessing", sep = "-")  

## [1] "I-love-Data-Preprocessing"

# use paste0() to paste without spaces between characters

paste0("I", "love",  "Data", "Preprocessing")  

## [1] "IloveDataPreprocessing"

# paste objects with different lengths

paste("R", 1:5, sep = " v1.")       

## [1] "R v1.1" "R v1.2" "R v1.3" "R v1.4" "R v1.5"

## [1] "R v1.1" "R v1.2" "R v1.3" "R v1.4" "R v1.5"

Sorting character strings is very simple using sort() function:

a <- c("MATH2349", "MATH1324")   
sort(a)

## [1] "MATH1324" "MATH2349"

Converting to Strings

Similar to the numerics, strings and characters can be tested with is.character() and any other data format can be converted into string/character with as.character() or with toString().

a <- "The life of"    
b <- pi

is.character(a)

## [1] TRUE

is.character(b)

## [1] FALSE

c <- as.character(b)
is.character(c)

## [1] TRUE

toString(c("Jul", 25, 2017))

## [1] "Jul, 25, 2017"

Printing Strings

Printing strings/characters can be done with the following functions:

The primary printing function in R is print().

# basic printing

a <- "MATH2349 is awesome"    
print(a)

## [1] "MATH2349 is awesome"

# print without quotes

print(a, quote = FALSE)  

## [1] MATH2349 is awesome

# alternative to print without quotes

noquote(a)

## [1] MATH2349 is awesome

Concatenating strings

The cat() function allows us to concatenate objects and print them either on screen or to a file. The output result is very similar to noquote(); however, cat() does not print the numeric line indicator. As a result, cat() can be useful for printing nicely formatted responses to users.

# basic printing (similar to noquote)

cat(a)                   

## MATH2349 is awesome

# combining character string

cat(a, "and I love R")           

## MATH2349 is awesome and I love R

# basic printing of alphabet

cat(letters)             

## a b c d e f g h i j k l m n o p q r s t u v w x y z

# specify a separator between the combined characters

cat(letters, sep = "-")  

## a-b-c-d-e-f-g-h-i-j-k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z

# collapse the space between the combine characters

cat(letters, sep = "")   

## abcdefghijklmnopqrstuvwxyz

You can also format the line width for printing long strings using the fill argument:

x <- "Today I am learning how to manipulate strings."
y <- "Tomorrow I plan to work on my assignment."
z <- "The day after I will take a break and drink a beer :)"

# No breaks between lines

cat(x, y, z, fill = FALSE)

## Today I am learning how to manipulate strings. Tomorrow I plan to work on my assignment. The day after I will take a break and drink a beer :)

# Breaks between lines

cat(x, y, z, fill = TRUE)

## Today I am learning how to manipulate strings. 
## Tomorrow I plan to work on my assignment. 
## The day after I will take a break and drink a beer :)

Counting string elements and characters

To count the number of elements in a string use length():

length("How many elements are in this string?")

## [1] 1

length(c("How", "many", "elements", "are", "in", "this", "string?"))

## [1] 7

To count the number of characters in a string use nchar():

nchar("How many characters are in this string?")

## [1] 39

nchar(c("How", "many", "characters", "are", "in", "this", "string?"))

## [1]  3  4 10  3  2  4  7

Upper/lower case conversion

To convert all upper case characters to lower case we will use tolower():

a <- "MATH2349 is AWesomE"
tolower(a)

## [1] "math2349 is awesome"

To convert all lower case characters to upper case we will use toupper():

toupper(x)

## [1] "TODAY I AM LEARNING HOW TO MANIPULATE STRINGS."

Simple Character Replacement

To replace a character (or multiple characters) in a string we can use chartr():

# replace 'A' with 'a'

x <- "This is A string."
chartr(old = "A", new = "a", x)

## [1] "This is a string."

# multiple character replacements
# replace any 'd' with 't' and any 'z' with 'a'

y <- "Tomorrow I plzn do lezrn zbout dexduzl znzlysis."
chartr(old = "dz", new = "ta", y)

## [1] "Tomorrow I plan to learn about textual analysis."

Note that chartr() replaces every identified letter for replacement so you need to use it when you are certain that you want to change every possible occurrence of that letter(s).

String Abbreviations

To abbreviate strings we can use abbreviate():

streets <- c("Victoria", "Yarra", "Russell", "Williams", "Swanston")

# default abbreviations
abbreviate(streets)

## Victoria    Yarra  Russell Williams Swanston 
##   "Vctr"   "Yarr"   "Rssl"   "Wllm"   "Swns"

# set minimum length of abbreviation
abbreviate(streets, minlength = 2)

## Victoria    Yarra  Russell Williams Swanston 
##     "Vc"     "Yr"     "Rs"     "Wl"     "Sw"

Extract/Replace Substrings

To extract or replace substrings in a character vector there are two primary base R functions to use: substr() and strsplit(). 

The purpose of substr() is to extract and replace substrings with specified starting and stopping characters. Here are some examples on substr() usage:

alphabet <- paste(LETTERS, collapse = "")
alphabet

## [1] "ABCDEFGHIJKLMNOPQRSTUVWXYZ"

# extract 18th character in alphabet
substr(alphabet, start = 18, stop = 18)

## [1] "R"

# extract 18-24th characters in alphabet
substr(alphabet, start = 18, stop = 24)

## [1] "RSTUVWX"

# replace 19-24th characters with `R`
substr(alphabet, start = 19, stop = 24) <- "RRRRRR"
alphabet

## [1] "ABCDEFGHIJKLMNOPQRRRRRRRYZ"

To split the elements of a character string we can use strsplit(). Here are some examples:

z <- "The day after I will take a break and drink a beer :)"
strsplit(z, split = " ")

## [[1]]
##  [1] "The"   "day"   "after" "I"     "will"  "take"  "a"     "break" "and"  
## [10] "drink" "a"     "beer"  ":)"

a <- "Victoria-Yarra-Russell-Williams-Swanston"
strsplit(a, split = "-") 

## [[1]]
## [1] "Victoria" "Yarra"    "Russell"  "Williams" "Swanston"

Note that the output of strsplit() is a list. To convert the output to a simple atomic vector simply wrap in unlist():

unlist(strsplit(a, split = "-"))

## [1] "Victoria" "Yarra"    "Russell"  "Williams" "Swanston"

Set operations for character strings

There are also base R functions that allows for assessing the set union, intersection, difference, equality, and membership of two vectors.

To obtain the elements of the union between two character vectors we can use union():

set_1 <- c("lagunitas", "bells", "dogfish", "summit", "odell")
set_2 <- c("sierra", "bells", "harpoon", "lagunitas", "founders")
union(set_1, set_2)

## [1] "lagunitas" "bells"     "dogfish"   "summit"    "odell"     "sierra"   
## [7] "harpoon"   "founders"

To obtain the common elements of two character vectors we can use intersect().

intersect(set_1, set_2)

## [1] "lagunitas" "bells"

In order to obtain the non-common elements, or the difference, of two character vectors we can use setdiff().

# returns elements in set_1 not in set_2

setdiff(set_1, set_2)

## [1] "dogfish" "summit"  "odell"

# returns elements in set_2 not in set_1
setdiff(set_2, set_1)

## [1] "sierra"   "harpoon"  "founders"

In order to test if two vectors contain the same elements regardless of order we can use setequal()

set_3 <- c("VIC", "NSW", "TAS")
set_4 <- c("WA", "SA", "NSW")
set_5 <- c("NSW", "SA", "WA")

setequal(set_3, set_4)

## [1] FALSE

setequal(set_4, set_5)

## [1] TRUE

We can use identical() to test if two character vectors are equal in content and order.

set_6 <- c("VIC", "NSW", "TAS")
set_7 <- c("NSW", "VIC", "TAS")
set_8 <- c("VIC", "NSW", "TAS")

identical(set_6, set_7)

## [1] FALSE

identical(set_6, set_8)

## [1] TRUE

In order to test if an element is contained within a character vector use is.element() or %in%. Here are some examples:

set_6 <- c("VIC", "NSW", "TAS")
set_7 <- c("NSW", "VIC", "TAS")
set_8 <- c("VIC", "NSW", "TAS")

is.element("VIC", set_8)

## [1] TRUE

"VIC" %in% set_8

## [1] TRUE

"WA" %in% set_8

## [1] FALSE

Basic operations

There are three string functions that are closely related to their base R equivalents, but with a few enhancements. They are:

• Concatenate with str_c()
• Number of characters with str_length()
• Substring with str_sub()

str_c() is equivalent to the paste() function in Base R.

# same as paste0()

str_c("Learning", "to", "use", "the", "stringr", "package")

## [1] "Learningtousethestringrpackage"

# same as paste()

str_c("Learning", "to", "use", "the", "stringr", "package", sep = " ")

## [1] "Learning to use the stringr package"

str_length() is similiar to the nchar() function in BaseR:

# some text

text = c("Learning", "to", NA, "use", "the", NA, "stringr", "package")

# `nchar()`

nchar(text)

## [1]  8  2 NA  3  3 NA  7  7

# `str_length()` same as above

str_length(text)

## [1]  8  2 NA  3  3 NA  7  7

As seen above, both str_length() and nchar() functions count the number of characters for non-missing values. For the missing values, they return NA as a value.

You can access individual character using str_sub(). The str_sub() function is equivalent to the substr() function in BaseR and it takes three arguments: a character vector, a start position and an end position. Either position can either be a positive integer, which counts from the left, or a negative integer which counts from the right. Note that the positions are inclusive, and if longer than the string, will be silently truncated.

# Some text

y <- c("abcdef", "ghifjk")

# The 3rd letter

str_sub(y, 3, 3)

## [1] "c" "i"

# The 2nd to 2nd-to-last character

str_sub(y, 2, -2)

## [1] "bcde" "hifj"

You can also use str_sub() to modify strings:

# Change the third letter with X

str_sub(y, 3, 3) <- "X"
y

## [1] "abXdef" "ghXfjk"

Duplicate Characters within a String

The stringr provides a new functionality using str_dup() in which base R does not have a specific function for character duplication.

str_dup("apples", times = 4)

## [1] "applesapplesapplesapples"

str_dup("apples", times = 1:4)

## [1] "apples"                   "applesapples"            
## [3] "applesapplesapples"       "applesapplesapplesapples"

Remove Leading and Trailing White space

In string processing, a common task is parsing text into individual words. Often, this results in words having blank spaces (white spaces) on either end of the word. The str_trim() can be used to remove these spaces. Here are some examples:

text <- c("Text ", "  with", " whitespace ", " on", "both ", " sides ")
text

## [1] "Text "        "  with"       " whitespace " " on"          "both "       
## [6] " sides "

# remove white spaces on the left side
str_trim(text, side = "left")

## [1] "Text "       "with"        "whitespace " "on"          "both "      
## [6] "sides "

# remove white spaces on the right side
str_trim(text, side = "right")

## [1] "Text"        "  with"      " whitespace" " on"         "both"       
## [6] " sides"

# remove white spaces on both sides
str_trim(text, side = "both")

## [1] "Text"       "with"       "whitespace" "on"         "both"      
## [6] "sides"

Pad a String with White space

To add white space, or to pad a string, we will use str_pad(). We can also use str_pad() to pad a string with specified characters. The width argument will give width of padded strings and the pad argument will specify the padding characters. Here are some examples:

str_pad("apples", width = 10, side = "left")

## [1] "    apples"

str_pad("apples", width = 10, side = "both")

## [1] "  apples  "

str_pad("apples", width = 10, side = "right", pad = "!")

## [1] "apples!!!!"

Upper/lower case conversion

Similar to tolower() and toupper() functions in BaseR, stringr also has upper/lower case transformation functions:

# Some text

x <- "I like VeGGies."

# convert all upper-case characters to lower case

str_to_lower(x)

## [1] "i like veggies."

#convert all lower-case characters to upper case

str_to_upper(x)

## [1] "I LIKE VEGGIES."

# convert only first letters of words to upper case

str_to_title(x)

## [1] "I Like Veggies."

String ordering and sorting

String ordering and sorting can be done using str_order and str_sort functions.

# some text

x <- c("y", "i", "k")

# return the index of ordered values

str_order(x)

## [1] 2 3 1

Note that str_order() returns the index of ordered values, not the order.

# sort characters

str_sort(x)

## [1] "i" "k" "y"

Pattern matching (Pattern detect, subset, locate, count, extract, and replace)

Most string manipulations require pattern matching for a given text. Good news is, stringr package has many specialized pattern matching functions to detect, subset, locate, count, extract, and replace strings. Here, I will demonstrate the commonly used pattern matching functions under stringr. For more information on other useful pattern matching functions, please refer to the [stringr vignette] (https://stringr.tidyverse.org/articles/stringr.html).

Note that, each pattern matching function described below has the same first two arguments, a character vector of strings to process and a single pattern to match specified by the pattern = argument.

• str_detect() detects the presence or absence of a pattern and returns a logical vector. Here is an example of its usage:
  

# detects pattern "ea"

x <- c("apple", "banana", "pear")
str_detect(x, pattern ="ea")

## [1] FALSE FALSE  TRUE

#same as above
str_detect(x, "ea")

## [1] FALSE FALSE  TRUE

While matching patterns, one can also use the regular expressions. Regular expressions (a.k.a. regex’s) are a language that allow you to describe patterns in strings. They take a little while to get your head around, but once you understand them, you’ll find them extremely useful.

# Same as above using regex

x <- c("apple", "banana", "pear")
str_detect(x, regex("ea"))

## [1] FALSE FALSE  TRUE

You can also perform a case-insensitive match using ignore.cases = TRUE:

bananas <- c("banana", "Banana", "BANANA")

#case sensitive match

str_detect(bananas, "banana") 

## [1]  TRUE FALSE FALSE

#case insensitive match

str_detect(bananas, regex("banana", ignore_case = TRUE)) 

## [1] TRUE TRUE TRUE

With regex, you can create your own character classes using [ ]. For example:

• [abc]: matches a, b, or c.
• [a-z]: matches every character between a and z (in Unicode code point order).
• [^abc]: matches anything except a, b, or c.
• [\^\-]: matches ^ or -.

There are a number of pre-built classes that you can use inside [ ]:

• [:punct:]: punctuation.
• [:alpha:]: letters.
• [:lower:]: lowercase letters.
• [:upper:]: upperclass letters.
• [:digit:]: digits.
• [:xdigit:]: hex digits.
• [:alnum:]: letters and numbers.
• [:cntrl:]: control characters.
• [:graph:]: letters, numbers, and punctuation.
• [:print:]: letters, numbers, punctuation, and white space.
• [:space:]: space characters (basically equivalent to ).
• [:blank:]: space and tab.

Here are some examples:

emails <- c("s123546@student.rmit.edu.au", "sona.taheri@rmit.edu.au", "s2342565@rmit.edu.vn")

#detect the emails containing numbers

str_detect(emails, "[:digit:]")

## [1]  TRUE FALSE  TRUE

#detect the emails containing lowercase letters

str_detect(emails, "[:lower:]")

## [1] TRUE TRUE TRUE

For more information on the regex capabilities, please refer to [regular expressions vignette] (https://stringr.tidyverse.org/articles/regular-expressions.html) under stringr package.

• str_subset() returns the elements of a character vector that match a regular expression.

#subset emails that contains numbers

str_subset(emails, "[:digit:]")

## [1] "s123546@student.rmit.edu.au" "s2342565@rmit.edu.vn"

• str_extract() extracts text corresponding to the first match, returning a character vector.

# extract the digits 1234

str_extract("Let's extract the digits 1234", pattern = "1234")

## [1] "1234"

# extract the digits 23 in emails

str_extract(emails, pattern = "23")

## [1] "23" NA   "23"

• str_locate() locates the first position of a pattern and returns a numeric matrix with columns start and end whereas str_locate_all()

# locate the first i in the string

str_locate("Locate the first i in this string", "i")

##      start end
## [1,]    13  13

# locate all the i's in this string

str_locate_all("Locate all the i's in this string", "i")

## [[1]]
##      start end
## [1,]    16  16
## [2,]    20  20
## [3,]    25  25
## [4,]    31  31

# locate all the full stops (\\.) in this string

str_locate_all("Full stop separates two sentences. Now I will locate all full stops.", "\\.")

## [[1]]
##      start end
## [1,]    34  34
## [2,]    68  68

• str_count() counts the number of matches for a given string.

# Counts the digits

str_count("90 Dollars", "[:digit:]") 

## [1] 2

# Counts the letters

str_count("90 Dollars", "[:alpha:]") 

## [1] 7

• str_replace() replaces a string with another one. The pattern argument will give the string that is going to be replaced and replacement argument will specify the replacement string.

# replace Dollars with AUD

str_replace("90 Dollars", pattern = "Dollars", replacement = "AUD")

## [1] "90 AUD"

• str_replace_all() replaces all matches.

# replace all l's with "" (delete l's)

str_replace_all("Hello world", pattern = "l", replacement = "")

## [1] "Heo word"