Category: R (programming language)

Working with XML Files in detail

XML, short for Extensible Markup Language, is composed of markup tags where each tag represents specific data within an XML file. To manipulate XML files in R, we need to use the XML package, which must be installed explicitly using the following command:

install.packages("XML")

Creating an XML File

An XML file is structured using hierarchical tags that contain information about data. It must be saved with a .xml extension.

Consider the following XML file named students.xml:

<STUDENTS>
  <STUDENT>
      <ID>101</ID>
      <NAME>Rahul</NAME>
      <SCORE>750</SCORE>
      <DEPARTMENT>Science</DEPARTMENT>
  </STUDENT>
  <STUDENT>
      <ID>102</ID>
      <NAME>Sneha</NAME>
      <SCORE>540</SCORE>
      <DEPARTMENT>Arts</DEPARTMENT>
  </STUDENT>
  <STUDENT>
      <ID>103</ID>
      <NAME>Amit</NAME>
      <SCORE>680</SCORE>
      <DEPARTMENT>Commerce</DEPARTMENT>
  </STUDENT>
  <STUDENT>
      <ID>104</ID>
      <NAME>Priya</NAME>
      <SCORE>720</SCORE>
      <DEPARTMENT>Science</DEPARTMENT>
  </STUDENT>
  <STUDENT>
      <ID>105</ID>
      <NAME>Varun</NAME>
      <SCORE>590</SCORE>
      <DEPARTMENT>Science</DEPARTMENT>
  </STUDENT>
</STUDENTS>

Reading an XML File in R

After installing the required package, we can read and parse an XML file using the xmlParse() function. This function takes the filename as an argument and returns the content as a structured list.

# Load necessary libraries
library("XML")
library("methods")

# Parse the XML file
student_data <- xmlParse(file = "students.xml")

print(student_data)

Output:

101
Rahul
750
Science
102
Sneha
540
Arts
103
Amit
680
Commerce
104
Priya
720
Science
105
Varun
590
Science

Extracting Information from an XML File

Using R, we can extract specific details from the XML structure, such as the number of nodes, specific elements, or attributes.

# Load required libraries
library("XML")
library("methods")

# Parse the XML file
parsed_data <- xmlParse(file = "students.xml")

# Extract the root node
root_node <- xmlRoot(parsed_data)

# Count the number of nodes
total_nodes <- xmlSize(root_node)

# Retrieve a specific record (2nd student)
second_student <- root_node[2]

# Extract a particular attribute (Score of 4th student)
specific_score <- root_node[[4]][[3]]

cat('Total number of students:', total_nodes, '\n')
print('Details of the 2nd student:')
print(second_student)

print('Score of the 4th student:', specific_score)

Output:

Total number of students: 5
Details of the 2nd student:
$STUDENT
    102
    Sneha
    540
    Arts

Score of the 4th student: 720

Converting XML to a Data Frame

To improve readability and ease of analysis, XML data can be converted into a structured data frame using the xmlToDataFrame() function in R.

# Load required libraries
library("XML")
library("methods")

# Convert XML to a data frame
student_df <- xmlToDataFrame("students.xml")
print(student_df)

Output:

ID    NAME   SCORE   DEPARTMENT
1   101   Rahul   750   Science
2   102   Sneha   540   Arts
3   103   Amit    680   Commerce
4   104   Priya   720   Science
5   105   Varun   590   Science

Working with CSV files in detail

In this article, we will explore how to handle CSV files in the R programming language.

Understanding CSV Files in R

CSV (Comma-Separated Values) files are plain text files where data is stored in tabular form, with values in each row separated by a delimiter such as a comma or tab. We will use a sample CSV file for demonstration purposes.

Managing the Working Directory in R

Before working with a CSV file, it is essential to check and set the working directory where the file is located.

# Display the current working directory
print(getwd())

# Change the working directory
setwd("/data/analysis")

# Confirm the new working directory
print(getwd())

Output:

[1] "C:/Users/DataScience/Documents"
[1] "C:/Users/DataScience/Documents"

Using the getwd() function, we can retrieve the current working directory, and with setwd(), we can modify it as needed.

Sample CSV File for Input

id, name, department, salary, projects
1,   Alex,   IT,        75000,   4
2,   Brian,  HR,        67000,   3
3,   Clara,  Marketing, 72000,   5
4,   Daniel, Sales,     58000,   2
5,   Emma,   Tech,      65000,   3
6,   Frank,  IT,        70000,   6
7,   Grace,  HR,        69000,   4

Save this file as employees.csv to use it in R.

Reading a CSV File in R

The read.csv() function allows us to read the contents of a CSV file into a data frame.

Example:

# Load the CSV file as a data frame
csv_data <- read.csv(file = 'C:\\Users\\DataScience\\Documents\\employees.csv')
print(csv_data)

# Display the number of columns
print(ncol(csv_data))

# Display the number of rows
print(nrow(csv_data))

Output:

id   name  department  salary  projects
1  1   Alex        IT   75000        4
2  2  Brian        HR   67000        3
3  3  Clara Marketing   72000        5
4  4 Daniel    Sales   58000        2
5  5   Emma      Tech   65000        3
6  6  Frank        IT   70000        6
7  7  Grace        HR   69000        4
[1] 5
[1] 7

The read.csv() function reads the file and stores it as a data frame in R. The ncol() and nrow() functions return the number of columns and rows, respectively.

Filtering Data from a CSV File

We can perform queries on the data using functions like subset() and logical conditions.

Finding the Minimum Value

# Find the minimum number of projects
min_projects <- min(csv_data$projects)
print(min_projects)

Output:

2

Filtering Employees with Salary Above 65000

# Select 'name' and 'salary' columns for employees with salary greater than 65000
result <- csv_data[csv_data$salary > 65000, c("name", "salary")]

# Display the filtered result
print(result)

Output:

name salary
1  Alex  75000
2 Brian  67000
3 Clara  72000
7 Grace  69000

The subset of employees meeting the condition is stored as a new data frame.

Writing Data to a CSV File

R allows exporting data frames to CSV files using write.csv().

# Calculate the average salary for each department
avg_salary <- tapply(csv_data$salary, csv_data$department, mean)

# Display the results
print(avg_salary)

Output:

HR        IT  Marketing   Sales    Tech
68000.0  72500.0  72000.0  58000.0  65000.0

Exporting Data in detail

When a program terminates, all data held in the program is lost. To ensure data persistence, we store the fetched information in files. This enables transferring data across systems and prevents re-entering large datasets. Files can be stored in formats such as .txt, .csv, or even in online/cloud storage. R provides straightforward methods to export data to these file types.

Exporting Data to a Text File

Text files are a common format for data storage. R provides methods like write.table() to export data frames or matrices to text files.

1. write.table(): The write.table() function writes a data frame or matrix to a text file.

Syntax:

write.table(x, file, append = FALSE, sep = " ", dec = ".", row.names = TRUE, col.names = TRUE)

Parameters:

• x: Data frame or matrix to be written.
• file: File name as a string.
• sep: Field separator (e.g., \t for tab-separated values).
• dec: Decimal separator (default is .).
• row.names: Logical or character vector for row names.
• col.names: Logical or character vector for column names.

Example:

# Creating a data frame
employee_data <- data.frame(
  "Employee" = c("John", "Emma", "Liam"),
  "Department" = c("HR", "IT", "Finance"),
  "Age" = c(29, 34, 41)
)

# Exporting the data frame to a text file
write.table(employee_data,
            file = "employee_data.txt",
            sep = "\t",
            row.names = TRUE,
            col.names = NA)

Output:

""    "Employee"    "Department"    "Age"
"1"    "John"        "HR"             29
"2"    "Emma"        "IT"             34
"3"    "Liam"        "Finance"        41

write_tsv(): The write_tsv() method from the readr package exports tab-separated values.

Syntax:

write_tsv(file, path)

Example:

# Creating a data frame
employee_data <- data.frame(
  "Employee" = c("John", "Emma", "Liam"),
  "Department" = c("HR", "IT", "Finance"),
  "Age" = c(29, 34, 41)
)

# Exporting the data frame to a text file
write.table(employee_data,
            file = "employee_data.txt",
            sep = "\t",
            row.names = TRUE,
            col.names = NA)

Output:

""    "Employee"    "Department"    "Age"
"1"    "John"        "HR"             29
"2"    "Emma"        "IT"             34
"3"    "Liam"        "Finance"        41

write_tsv(): The write_tsv() method from the readr package exports tab-separated values.

Syntax:

write_tsv(file, path)

Example:

# Importing the readr package
library(readr)

# Creating a data frame
student_data <- data.frame(
  "Name" = c("Alice", "Bob", "Charlie"),
  "Grade" = c("A", "B", "A+"),
  "Age" = c(20, 22, 21)
)

# Exporting the data frame using write_tsv()
write_tsv(student_data, path = "student_data.txt")

Output:

Name    Grade    Age
Alice   A        20
Bob     B        22
Charlie A+       21

Exporting Data to a CSV File

CSV files are widely used for storing tabular data. R provides multiple methods for exporting data to .csv files.

write.table(): The write.table() function can also export data to CSV files by specifying sep = ",".

Example:

# Creating a data frame
product_data <- data.frame(
  "Product" = c("Laptop", "Phone", "Tablet"),
  "Price" = c(1000, 500, 300),
  "Stock" = c(50, 200, 150)
)

# Exporting the data frame to a CSV file
write.table(product_data,
            file = "product_data.csv",
            sep = ",",
            row.names = FALSE)

Output:

Product,Price,Stock
Laptop,1000,50
Phone,500,200
Tablet,300,150

write.csv()

The write.csv() function simplifies exporting data to CSV files, using a comma as the default separator.

Example:

# Creating a data frame
city_data <- data.frame(
  "City" = c("New York", "Los Angeles", "Chicago"),
  "Population" = c(8419600, 3980400, 2716000),
  "Area" = c(468.9, 503, 227.3)
)

# Exporting the data frame to a CSV file
write.csv(city_data, file = "city_data.csv")

Output:

"","City","Population","Area"
"1","New York",8419600,468.9
"2","Los Angeles",3980400,503
"3","Chicago",2716000,227.3

write.csv2():The write.csv2() function is similar to write.csv() but uses a semicolon (;) as the separator and a comma for the decimal point.

Example:

# Creating a data frame
sales_data <- data.frame(
  "Month" = c("January", "February", "March"),
  "Sales" = c(15000.50, 17000.75, 16000.30)
)

# Exporting the data frame to a CSV file
write.csv2(sales_data, file = "sales_data.csv")

Output:

";""Month"";""Sales"
"1";"January";"15000,50"
"2";"February";"17000,75"
"3";"March";"16000,30"

write_csv(): The write_csv() method from the readr package exports data to CSV files.

Example:

# Importing the readr package
library(readr)

# Creating a data frame
book_data <- data.frame(
  "Title" = c("R for Data Science", "Python Crash Course", "The Art of R Programming"),
  "Author" = c("Hadley Wickham", "Eric Matthes", "Norman Matloff"),
  "Price" = c(35.99, 29.99, 45.00)
)

# Exporting the data frame using write_csv()
write_csv(book_data, path = "book_data.csv")

Output:

Title,Author,Price
R for Data Science,Hadley Wickham,35.99
Python Crash Course,Eric Matthes,29.99
The Art of R Programming,Norman Matloff,45.00

Import Data from a File in detail

Data is a collection of facts and can exist in multiple formats. To analyze data using the R programming language, it first needs to be imported. R allows importing data from various file types such as text files, CSV, and other delimiter-separated files. Once imported, users can manipulate, analyze, and generate reports from the data.

Importing Data from Files into R

This guide demonstrates how to import different file formats into R programming.

Importing CSV Files

Method 1: Using read.csv()

The read.csv() function is a straightforward method for importing CSV files.

read.csv(file_path, header = TRUE, sep = ",")

Arguments:

• file_path: The file’s location.
• header: TRUE (default) to indicate column headings.
• sep: The separator for values in each row (default is a comma ,).

Example:

# Specify file path
file_path <- "data.csv"

# Read the CSV file
content <- read.csv(file_path)

# Print file contents
print(content)

Output:

ID Name   Role Age
1  1  Alex  Dev  30
2  2  Sam   QA   25
3  3  Emma  HR   28

Method 2: Using read.table()

Another way to import CSV files is by using read.table().

# Import CSV using read.table()
data <- read.table("C://data//records.csv", header = TRUE, sep = ",")

# Print file contents
print(data)

Output:

Col1 Col2 Col3
1  101  A1   B1
2  202  A2   B2
3  303  A3   B3

Importing Data from a Text File

read.table() can also be used for importing text files.

Syntax:

read.table("file.txt", header = TRUE/FALSE)

Example:

# Read text file
data <- read.table("C://data//records.txt", header = FALSE)

# Print file contents
print(data)

Output:

V1  V2  V3
1 200  A1  B1
2 300  A2  B2
3 400  A3  B3

Importing Data from a Delimited File

The read.delim() function is used to import delimited files, where values are separated by specific symbols such as |, $, or ,.

Syntax:

read.delim("file.txt", sep="|", header=TRUE)

Example:

# Read a delimited file
data <- read.delim("C://data//info.txt", sep="|", header=TRUE)

# Print file contents
print(data)

Output:

$ID
[1] "101" "102" "103"
$Name
[1] "John" "Lily" "Raj"
$Salary
[1] "1500" "2000" "2500"

Importing XML Files

To import XML files, use the XML package.

XML File Sample:

<RECORDS>
  <EMPLOYEE>
    <ID>1</ID>
    <NAME>Adam</NAME>
    <SALARY>5000</SALARY>
  </EMPLOYEE>
  <EMPLOYEE>
    <ID>2</ID>
    <NAME>Sophia</NAME>
    <SALARY>6000</SALARY>
  </EMPLOYEE>
</RECORDS>

Example:

# Load XML package
library("XML")

# Parse XML file
data <- xmlParse(file = "C://data//employees.xml")

# Print parsed data
print(data)

Output:

1  Adam   5000
2  Sophia 6000

Importing SPSS Files

SPSS .sav files can be imported using the haven package.

Syntax:

read_sav("file.sav")

Example:

# Load haven package
library("haven")

# Read SPSS file
data <- read_sav("C://data//survey.sav")

# Print data
print(data)

Output:

ID   Age  Response  Score
1  1   23   Agree     4.5
2  2   30   Neutral   3.0
3  3   27   Disagree  2.5

Data Handling in detail

R offers several functions to import data from various file formats into your working environment. This guide demonstrates how to import data into R using different file formats.

Importing Data in R

To illustrate, we will use a sample dataset in two formats: .csv and .txt. Let’s dive into the methods for importing data.

Reading a CSV (Comma-Separated Values) File

Method 1: Using read.csv()

The read.csv() function is a simple way to import CSV files. It includes the following parameters:

• file.choose(): Opens a dialog box to select a CSV file.
• header: Indicates if the first row contains column names. Use TRUE if it does or FALSE otherwise.

Example:

# Import and store the dataset in data1
data1 <- read.csv(file.choose(), header = TRUE)

# Display the data
print(data1)

Output:

Name    Age Department
1 John    25   IT
2 Alice   30   HR
3 Robert  28   Finance

Method 2: Using read.table()

The read.table() function requires you to specify the delimiter using the sep parameter. For CSV files, use sep=",".

Example:

# Import and store the dataset in data2
data2 <- read.table(file.choose(), header = TRUE, sep = ",")

# Display the data
print(data2)

Output:

Name    Age Department
1 John    25   IT
2 Alice   30   HR
3 Robert  28   Finance

Reading a Tab-Delimited (.txt) File

Method 1: Using read.delim()

This function is specifically for tab-delimited files. It also has parameters like:

• file.choose(): Opens a file selection dialog.
• header: Indicates whether the first row contains column names.

Example:

# Import and store the dataset in data3
data3 <- read.delim(file.choose(), header = TRUE)

# Display the data
print(data3)

Output:

Product Price Quantity
1  Apples  100       50
2 Bananas   50      120
3 Oranges   75       80

Method 2: Using read.table()

For tab-delimited files, use sep="\t" to specify the delimiter.

Example:

# Import and store the dataset in data4
data4 <- read.table(file.choose(), header = TRUE, sep = "\t")

# Display the data
print(data4)

Output:

Product Price Quantity
1  Apples  100       50
2 Bananas   50      120
3 Oranges   75       80

Using RStudio to Import Data

You can also import data interactively using RStudio. Follow these steps:

1. In the Environment tab, click Import Dataset.
2. Choose the file format (CSV, Excel, etc.).
3. Browse your computer to select the file.
4. The data will appear in the RStudio Viewer. Type the dataset name in the console to display it.

Reading JSON Files in R

To work with JSON files, install the rjson package. This package allows you to:

• Load JSON files.
• Convert JSON data into data frames for analysis.

Install the Package:

install.packages("rjson")

Example JSON File (saved as example.json):

{
  "ID": ["101", "102", "103"],
  "Name": ["Alice", "Bob", "Charlie"],
  "Salary": ["5000", "6000", "5500"],
  "Department": ["IT", "HR", "Finance"]
}

Code to Read JSON:

# Load the rjson library
library(rjson)

# Provide the path to the JSON file
result <- fromJSON(file = "C:\\example.json")

# Print the result
print(result)

Output:

$ID
[1] "101" "102" "103"

$Name
[1] "Alice"   "Bob"     "Charlie"

$Salary
[1] "5000"  "6000"  "5500"

$Department
[1] "IT"      "HR"      "Finance"

Converting JSON to a Data Frame:

# Convert JSON to a data frame
data <- as.data.frame(result)
print(data)

Output:

ID    Name Salary Department
1    101   Alice   5000         IT
2    102     Bob   6000         HR
3    103 Charlie   5500    Finance

Data Handling in detail

The R programming language is extensively used for statistical analysis and data visualization. Handling data involves importing and exporting files, and R simplifies this process by supporting various file types such as CSV, text files, Excel spreadsheets, SPSS, SAS, and more.

R provides several predefined functions to navigate and interact with system directories. These functions allow users to either retrieve the current directory path or change it as needed.

Directory Functions in R

• getwd(): Retrieves the current working directory.
• setwd(): Changes the working directory. The directory path is passed as an argument to this function.

Example:

# Change working directory
setwd("D:/RProjects/")

# Alternative way using double backslashes
setwd("D:\\RProjects\\")

• list.files(): Displays all files and folders in the current working directory.

fluidPage(…, title = NULL, theme = NULL)

Importing Files in R

Importing Text Files: Text files can be read into R using the read.table() function.

Syntax:

read.table(filename, header = FALSE, sep = "")

Parameters:

• header: Indicates whether the file contains a header row.
• sep: Specifies the delimiter used in the file.

For more details, use the command:

help("read.table")

Example:
Suppose the file “SampleText.txt” in the current working directory contains the following data:

101 X p
202 Y q
303 Z r
404 W s
505 V t
606 U u

Code:

# Get the current working directory
getwd()

# Read the text file into a data frame
data <- read.table("SampleText.txt", header = FALSE, sep = " ")

# Print the data frame
print(data)

# Print the class of the object
print(class(data))

Output:

[1] "D:/RProjects"
   V1 V2 V3
1 101  X  p
2 202  Y  q
3 303  Z  r
4 404  W  s
5 505  V  t
6 606  U  u
[1] "data.frame"

Importing CSV Files: CSV files can be imported using the read.csv() function.

Syntax:

read.csv(filename, header = FALSE, sep = "")

Parameters:

• header: Specifies if the file contains a header row.
• sep: Indicates the delimiter used.

For details, run:

help("read.csv")

Example:
Assume the file “SampleCSV.csv” contains the following data:

101,XA,pa
202,YB,qb
303,ZC,rc
404,WD,sd
505,VE,te

Code:

# Read the CSV file
data <- read.csv("SampleCSV.csv", header = FALSE)

# Print the data frame
print(data)

# Print the class of the object
print(class(data))

Output:

V1  V2  V3
1 101  XA  pa
2 202  YB  qb
3 303  ZC  rc
4 404  WD  sd
5 505  VE  te
[1] "data.frame"

Importing Excel Files: To read Excel files, install the openxlsx package and use the read.xlsx() function.

Syntax:

read.xlsx(filename, sheet = 1)

Parameters:

• sheet: Specifies the sheet name or index.

For help:

help("read.xlsx")

Example:
Suppose the Excel file “SampleExcel.xlsx” contains the following data:

A	B	C
1001	XYA	xyz
2002	YZB	yqw
3003	ZWC	wuv

Code:

# Install and load the openxlsx package
install.packages("openxlsx")
library(openxlsx)

# Read the Excel file
data <- read.xlsx("SampleExcel.xlsx", sheet = 1)

# Print the data frame
print(data)

# Print the class of the object
print(class(data))

Output:

A    B   C
1 1001  XYA xyz
2 2002  YZB yqw
3 3003  ZWC wuv
[1] "data.frame"

Exporting Files in R

Redirecting Output with cat(): The cat() function outputs objects to the console or redirects them to a file.

Syntax:

cat(..., file)

Example:

# Redirect output to a file
cat("Greetings from R!", file = "OutputText.txt")

Output:

Greetings from R!

Redirecting Output with sink(): The sink() function captures output and redirects it to a file.

Syntax:

sink(filename)
...
sink()

Example:

# Redirect output to a file
sink("OutputSink.txt")

x <- c(2, 4, 6, 8, 12)
print(mean(x))
print(class(x))
print(max(x))

# End redirection
sink()

Output (file content):

[1] 6.4
[1] "numeric"
[1] 12

Writing CSV Files: The write.csv() function writes data to a CSV file.

Syntax:

write.csv(x, file)

Example:

# Create a data frame
df <- data.frame(A = c(11, 22, 33), B = c("X", "Y", "Z"), C = c(TRUE, FALSE, TRUE))

# Write the data frame to a CSV file
write.csv(df, file = "OutputCSV.csv", row.names = FALSE)

Output:

A,B,C
11,X,TRUE
22,Y,FALSE
33,Z,TRUE

Data Munging in detail

Data Munging refers to the process of transforming raw or erroneous data into a clean and usable format. Without data munging—whether done manually by a user or through an automated system—the data is often unsuitable for downstream analysis or consumption. Essentially, data munging involves cleansing and reformatting data manually or using automated tools.

In R Programming, the following methods are commonly used for the data munging process:

• apply() Family
• aggregate()
• dplyr package
• plyr package

Using the apply() Family for Data Munging

The apply() function is one of the foundational functions in R for performing operations on matrices or arrays. Other functions in the same family include lapply(), sapply(), and tapply(). These functions often serve as an alternative to loops, providing a cleaner and more efficient approach to repetitive tasks.

The apply() function is particularly suited for operations on matrices or arrays with homogeneous elements. When applied to other data structures, such as data frames, the function first converts them into a matrix before processing.

Syntax:

apply(X, margin, function)

Parameters:

• X: An array or matrix.
• margin: A value (1 for rows, 2 for columns) indicating where to apply the function.
• function: The operation or function to perform.

Example:

# Example of apply()
matrix_data <- matrix(1:12,
                      nrow = 3,
                      ncol = 4)
matrix_data

result <- apply(matrix_data, 2, sum)
result

Output:

[,1] [,2] [,3] [,4]
[1,]    1    4    7   10
[2,]    2    5    8   11
[3,]    3    6    9   12

[1]  6 15 24 33

The lapply() Function: The lapply() function operates on lists and returns a list of the same length. Unlike apply(), it does not require a margin parameter. The “l” in lapply() signifies that the output is always a list.

Syntax:

lapply(X, func)

Parameters:

• X: A list, vector, or object.
• func: The function to apply.

Example:

# Example of lapply()
fruits <- c("APPLE", "BANANA", "CHERRY", "MANGO")
fruits

lowercase_fruits <- lapply(fruits, tolower)
lowercase_fruits

Output:

[1] "APPLE"   "BANANA"  "CHERRY"  "MANGO"

[[1]]
[1] "apple"

[[2]]
[1] "banana"

[[3]]
[1] "cherry"

[[4]]
[1] "mango"

The sapply() Function: The sapply() function works similarly to lapply(). However, it tries to simplify the output into a vector or matrix if possible.

Example:

# Example of sapply()
fruits <- c("APPLE", "BANANA", "CHERRY", "MANGO")

lowercase_fruits <- sapply(fruits, tolower)
lowercase_fruits

Output:

[1] "apple"  "banana" "cherry" "mango"

The tapply() Function: The tapply() function is used to perform an operation on subsets of data grouped by a factor. It is particularly useful for aggregating data.

Syntax:

tapply(X, index, func = NULL)

Parameters:

• X: A vector or object.
• index: A factor or list of factors for grouping.
• func: The function to apply.

Example:

# Example of tapply()
data(iris)

species_median <- tapply(iris$Sepal.Length,
                         iris$Species,
                         median)
species_median

Output:

setosa versicolor  virginica
5.0        5.9        6.5

Using aggregate() in R

To summarize data by grouping variables and applying a function (e.g., sum, mean).

Syntax:

aggregate(formula, data, function)

Parameters:

• formula: Specifies the variables for grouping.
• data: The dataset for aggregation.
• function: The operation to perform on the grouped data.

Example:

exposures <- aggregate(
  x = assets[c("counterparty.a", "counterparty.b", "counterparty.c")],
  by = assets[c("asset.class", "rating")],
  FUN = function(market.values) { sum(pmax(market.values, 0)) }
)

Using the plyr Package

A versatile package for splitting, applying functions, and combining data.

Key Functions:

• ddply(): Operates on data frames.
• llply(): Operates on lists.

Advantages:

• Simplifies operations with consistent syntax.
• Offers parallel computation and progress bars.

Example with ddply():

library(plyr)
ddply(dfx, .(group, sex), summarize,
      mean = round(mean(age), 2),
      sd = round(sd(age), 2))

Using the dplyr Package

Purpose: Provides a consistent grammar for data manipulation with verbs like arrange, filter, mutate, select, and summarize.

Advantages:

• Fast and efficient backend.
• Easy-to-read pipe (%>%) syntax.

Examples:

• Arrange rows:

starwars %>% arrange(desc(mass))

• Filter rows:

starwars %>% filter(species == "Droid")

• Mutate new variables:

starwars %>% mutate(bmi = mass / ((height / 100) ^ 2)) %>%
            select(name:mass, bmi)

• Summarize grouped data:

starwars %>% group_by(species) %>%
            summarize(n = n(), avg_mass = mean(mass, na.rm = TRUE)) %>%
            filter(n > 1)

Example:

library(dplyr)

# Group by gender, summarise, and filter
starwars %>%
  group_by(gender) %>%
  summarise(
    n = n(),
    avg_height = mean(height, na.rm = TRUE)
  ) %>%
  filter(n > 3)

Output:

Assuming the starwars dataset is unmodified:

gender	n	avg_height
male	60	178.41
female	16	165.56

Tidyverse Packages in detail

When working with Data Science in R, the Tidyverse packages are your ultimate toolkit! These packages were designed specifically for Data Science and share a unified design philosophy.

The Tidyverse packages cover the entire data science workflow, from data import and tidying to transformation and visualization. For example, readr is used for data importing, tibble and tidyr for tidying, dplyr and stringr for transformation, and ggplot2 for visualization.

What Are the Tidyverse Packages in R?

Core Tidyverse Packages

There are eight core Tidyverse packages: ggplot2, dplyr, tidyr, readr, purrr, tibble, stringr, and forcats. These are automatically loaded when you use the command:

install.packages("tidyverse")

Specialized Packages

In addition to the core packages, the Tidyverse also includes specialized packages like DBI for databases, httr for web APIs, and rvest for web scraping. These need to be loaded individually.

Now, let’s explore the core Tidyverse packages and their uses.

Data Visualization and Exploration

1. ggplot2: ggplot2 is a powerful data visualization library based on the “Grammar of Graphics.” It allows you to create visualizations like bar charts, scatter plots, and histograms using a high-level API. Once you define the mapping of variables to aesthetics, ggplot2 takes care of the rest.

To install ggplot2:

install.packages("ggplot2")

Or use the development version:

devtools::install_github("tidyverse/ggplot2")

Example:

# Load the library
library(ggplot2)

# Create a dataframe with categories and values
data <- data.frame(
  Category = c('X', 'Y', 'Z', 'W'),
  Value = c(10, 20, 15, 25)
)

# Create a bar plot
ggplot(data, aes(x = Category, y = Value, fill = Category)) +
  geom_bar(stat = "identity")

Output: A bar plot with default colors for the bars based on categories.

Data Wrangling and Transformation

1. dplyr: dplyr is a widely-used library for data manipulation. Its key functions, often used with group_by(), include:

• mutate(): Adds new variables.
• select(): Selects specific columns.
• filter(): Filters rows based on conditions.
• summarise(): Aggregates data.
• arrange(): Sorts rows.

To install dplyr:

install.packages("dplyr")

Or use the development version:

devtools::install_github("tidyverse/dplyr")

Example: Filtering Rows

library(dplyr)

# Using the built-in mtcars dataset
mtcars %>% filter(cyl == 6)

Output: Displays rows of the mtcars dataset where the number of cylinders is 6.

2. tidyr: tidyr helps tidy your data, ensuring each variable has its own column and each observation its own row.

Key functions include:

• Pivoting: Reshaping data between wide and long formats.
• Nesting: Grouping data into nested structures.
• Splitting/Combining: Working with character columns.

To install tidyr:

install.packages("tidyr")

Or use the development version:

devtools::install_github("tidyverse/tidyr")

Example: Reshaping Data with pivot_longer()

library(tidyr)

# Create a data frame
data <- data.frame(
  ID = 1:5,
  Score1 = c(80, 90, 85, 88, 92),
  Score2 = c(75, 85, 82, 89, 95)
)

# Convert wide format to long format
long_data <- data %>%
  pivot_longer(cols = starts_with("Score"),
               names_to = "Score_Type",
               values_to = "Value")

print(long_data)

Output:

ID Score_Type Value
1  1    Score1    80
2  1    Score2    75
3  2    Score1    90
4  2    Score2    85
...

3. stringr: stringr simplifies string manipulation in R, offering consistent naming conventions. Functions include:

• str_detect(): Detect patterns.
• str_extract(): Extract patterns.
• str_replace(): Replace patterns.
• str_length(): Compute string length.

To install stringr:

install.packages("stringr")

Example: Calculating String Length

library(stringr)

# Calculate string length
length <- str_length("Tidyverse")
print(length)

Output:

9

4. Forcats: The forcats library in R is designed to address common challenges associated with working with categorical variables, often referred to as factors. Factors are variables with a fixed set of possible values, which are predefined. forcats helps with tasks like reordering levels, modifying the order of values, and other related operations.

Some key functions in forcats include:

• fct_relevel(): Reorders factor levels manually.
• fct_reorder(): Reorders a factor based on another variable.
• fct_infreq(): Reorders a factor by frequency of values.

To install forcats, the recommended approach is to install the tidyverse package:

install.packages("tidyverse")

Alternatively, you can install forcats directly:

install.packages("forcats")

To install the development version from GitHub, use:

devtools::install_github("tidyverse/forcats")

Example:

library(forcats)
library(dplyr)
library(ggplot2)

# Example data: species counts
print(head(starwars %>%
             filter(!is.na(species)) %>%
             count(species, sort = TRUE)))

Output:

# A tibble: 6 × 2
  species      n
  <chr>    <int>
1 Human       35
2 Droid        6
3 Gungan       3
4 Kaminoan     2
5 Mirialan     2
6 Twi'lek      2

Data Import and Management in Tidyverse in R

1. Readr: The readr library offers an efficient way to import rectangular data formats such as .csv, .tsv, .delim, and others. It automatically parses and converts columns into appropriate data types, making data import easier and faster.

Common functions include:

• read_csv(): Reads comma-separated files.
• read_tsv(): Reads tab-separated files.
• read_table(): Reads tabular data.
• read_fwf(): Reads fixed-width files.
• read_delim(): Reads delimited files.
• read_log(): Reads log files.

To install readr, use:

install.packages("tidyverse")  # Recommended
install.packages("readr")      # Alternatively

For the development version:

devtools::install_github("tidyverse/readr")

Example:

library(readr)

# Read a tab-separated file
data <- read_tsv("sample_data.txt", col_names = FALSE)
print(data)

Output:

# A tibble: 1 × 1
  X1
  <chr>
1 A platform for data enthusiasts.

2. Tibble: A tibble is an enhanced version of a data frame in R. Unlike traditional data frames, tibbles do not modify variable names or types and provide better error handling. This makes the code cleaner and more robust. Tibbles are especially useful for large datasets with complex objects.

Key functions:

• tibble(): Creates a tibble from column vectors.
• tribble(): Creates a tibble row by row.

To install tibble:

install.packages("tidyverse")  # Recommended
install.packages("tibble")     # Alternatively

Development version:

devtools::install_github("tidyverse/tibble")

Example:

library(tibble)

# Create a tibble
data <- tibble(a = 1:3, b = letters[1:3], c = Sys.Date() - 1:3)
print(data)

Output:

# A tibble: 3 × 3
      a b     c
  <int> <chr> <date>
1     1 a     2025-01-22
2     2 b     2025-01-21
3     3 c     2025-01-20

Functional Programming in Tidyverse in R

Purrr: The purrr package provides tools for functional programming in R, particularly with functions and vectors. It simplifies complex operations by replacing repetitive for loops with clean, readable, and type-stable code.

One of its most popular functions is map(), which applies a function to each element of a list or vector.

To install purrr:

install.packages("tidyverse")  # Recommended
install.packages("purrr")      # Alternatively

Development version:

devtools::install_github("tidyverse/purrr")

Example:

library(purrr)

# Example: Model fitting and extracting R-squared
mtcars %>%
  split(.$cyl) %>%
  map(~ lm(mpg ~ wt, data = .)) %>%
  map(summary) %>%
  map_dbl("r.squared")

Output:

4         6         8
0.5086326 0.4645102 0.4229655

Shiny Package in detail

Packages in the R programming language are a collection of R functions, compiled code, and sample data. They are stored under a directory called “library” in the R environment. By default, R installs a set of packages during installation. One of the most important packages in R is the Shiny package, which makes it easy to build interactive web applications directly from R.

Installing the Shiny Package in R

To use a package in R, it must be installed first. This can be done using the install.packages("packagename") command. To install the Shiny package, use the following command:

install.packages("shiny")

To install the latest development builds directly from GitHub, use this:

if (!require("remotes"))
  install.packages("remotes")
remotes::install_github("rstudio/shiny")

Important Functions in the Shiny Package

1. fluidPage():The fluidPage() function creates a page with a fluid layout. A fluid layout consists of rows that include columns. Rows ensure their elements appear on the same line, while columns define the horizontal space within a 12-unit-wide grid. Fluid pages scale their components dynamically to fit the available browser width.

Syntax:

fluidPage(…, title = NULL, theme = NULL)

Parameter	Description
…	Elements to include within the page.
title	The browser window title.
theme	An alternative Bootstrap stylesheet.

Example:

# Import shiny package
library(shiny)

# Define a page with a fluid layout
ui <- fluidPage(
  h1("Interactive App with Shiny"),
  p(style = "font-family:Arial", "This is a simple Shiny app")
)

server <- function(input, output) {}

shinyApp(ui = ui, server = server)

Output:

2. shinyApp(): The shinyApp() function creates Shiny app objects by combining UI and server components. It can also take the path of a directory containing a Shiny app.

Syntax:

shinyApp(ui, server, onStart = NULL, options = list(), uiPattern = "/", enableBookmarking = NULL)
shinyAppDir(appDir, options = list())
shinyAppFile(appFile, options = list())

Parameter	Description
ui	The UI definition of the app.
server	Server logic containing input, output, session.
onStart	Function to call before the app runs.
options	Options passed to runApp.
uiPattern	Regular expression to match GET requests.
enableBookmarking	Can be “url”, “server”, or “disable”. Default is NULL.

Example:

# Import shiny package
library(shiny)

# Define fluid page layout
ui <- fluidPage(
  sliderInput(
    inputId = "num",
    label = "Choose a number",
    value = 10,
    min = 1,
    max = 1000
  ),
  plotOutput("hist")
)

server <- function(input, output) {
  output$hist <- renderPlot({
    hist(rnorm(input$num))
  })
}

shinyApp(ui = ui, server = server)

Output:

3. reactive(): The reactive() function creates a reactive expression, which updates whenever its dependencies change.

Syntax:

reactive(x, env = parent.frame(), quoted = FALSE, label = NULL)

Parameter	Description
x	An expression.
env	Parent environment for the expression.
quoted	Whether the expression is quoted (default: FALSE).
label	A label for the reactive expression.

Example:

# Import shiny package
library(shiny)

# Define fluid page layout
ui <- fluidPage(
  numericInput("num", "Enter a number", value = 10),
  plotOutput("hist"),
  verbatimTextOutput("stats")
)

server <- function(input, output) {
  data <- reactive({
    rnorm(input$num)
  })

  output$hist <- renderPlot({
    hist(data())
  })

  output$stats <- renderPrint({
    summary(data())
  })
}

shinyApp(ui = ui, server = server)

Output:

4. observeEvent(): The observeEvent() function responds to event-like reactive inputs and triggers specific code on the server side.

Syntax:

observeEvent(eventExpr, handlerExpr,
event.env = parent.frame(), event.quoted = FALSE,
handler.env = parent.frame(), handler.quoted = FALSE,
label = NULL, suspended = FALSE, priority = 0,
domain = getDefaultReactiveDomain(), autoDestroy = TRUE,
ignoreNULL = TRUE, ignoreInit = FALSE, once = FALSE)

Parameter	Description
eventExpr	Reactive expression triggering the event.
handlerExpr	Code to execute when eventExpr is invalidated.
ignoreNULL	Ignore the action when input is NULL (default: TRUE).
once	Whether the event is triggered only once.

Example:

# Import shiny package
library(shiny)

# Define fluid page layout
ui <- fluidPage(
  numericInput("num", "Enter a number", value = 10),
  actionButton("calculate", "Show Data"),
  tableOutput("table")
)

server <- function(input, output) {
  observeEvent(input$calculate, {
    num <- as.numeric(input$num)

    if (is.na(num)) {
      cat("Invalid numeric value entered.\n")
      return(NULL)
    }
    cat("Displaying data for", num, "rows.\n")
  })

  df <- eventReactive(input$calculate, {
    num <- as.numeric(input$num)

    if (is.na(num)) {
      return(NULL)
    }

    head(mtcars, num)
  })

  output$table <- renderTable({
    df()
  })
}

shinyApp(ui = ui, server = server)

Output:

Random Rows:
    Name Age Height
1 Sanjay  30    5.9
2  Meera  24     NA

Random Fraction:
    Name Age Height
1  Anita  28    5.4
2  Rahul  25     NA

5. eventReactive() in Shiny:eventReactive() is used to create a reactive expression that triggers only when specific events occur. It listens to “event-like” reactive inputs, values, or expressions.

Syntax

eventReactive(eventExpr,
              valueExpr,
              event.env = parent.frame(),
              event.quoted = FALSE,
              value.env = parent.frame(),
              value.quoted = FALSE,
              label = NULL,
              domain = getDefaultReactiveDomain(),
              ignoreNULL = TRUE,
              ignoreInit = FALSE)

Parameters

Parameter	Description
eventExpr	The expression representing the event, which can be a simple or complex reactive expression.
valueExpr	Produces the return value of eventReactive. Executes within an isolate() scope.
event.env	Parent environment for eventExpr. Default is the calling environment.
event.quoted	Indicates if eventExpr is quoted. Default is FALSE.
value.env	Parent environment for valueExpr. Default is the calling environment.
value.quoted	Indicates if valueExpr is quoted. Default is FALSE.
ignoreNULL	Determines if action should trigger when the input is NULL.
ignoreInit	If TRUE, ignores the handler expression when first initialized. Default is FALSE.

Example: Using eventReactive

library(shiny)

ui <- fluidPage(
  sliderInput(inputId = "num",
              label = "Choose a number",
              value = 25, min = 1, max = 100),
  actionButton(inputId = "update",
               label = "Update"),
  plotOutput("histogram")
)

server <- function(input, output) {
  data <- eventReactive(input$update, {
    rnorm(input$num)
  })

  output$histogram <- renderPlot({
    hist(data())
  })
}

shinyApp(ui = ui, server = server)

Output:

6. actionButton() in Shiny: actionButton() creates a button that triggers an action when clicked.

Syntax

actionButton(inputId, label, icon = NULL, width = NULL, ...)

Parameters

Parameter	Description
inputId	ID for accessing the button value.
label	Text displayed on the button.
icon	Icon to display with the button (optional).
width	Width of the button (e.g., ‘200px’, ‘100%’).
...	Additional attributes for the button.

Example: Using actionButton

library(shiny)

ui <- fluidPage(
  sliderInput("obs", "Number of Observations", min = 1, max = 1000, value = 500),
  actionButton("goButton", "Generate Plot"),
  plotOutput("plot")
)

server <- function(input, output) {
  output$plot <- renderPlot({
    input$goButton
    isolate({
      dist <- rnorm(input$obs)
      hist(dist)
    })
  })
}

shinyApp(ui, server)

Output:

7. checkboxGroupInput() in Shiny:checkboxGroupInput() creates a group of checkboxes for selecting multiple options.

Syntax

checkboxGroupInput(inputId, label, choices = NULL, selected = NULL, inline = FALSE, width = NULL, choiceNames = NULL, choiceValues = NULL)

Parameters

Parameter	Description
inputId	ID for accessing the selected checkbox values.
label	Label displayed above the checkboxes.
choices	List of values for the checkboxes. If named, the name is displayed instead of the value.
selected	Initial selected value(s).
inline	If TRUE, renders the checkboxes horizontally.
width	Width of the input element.
choiceNames	Names displayed for the choices.
choiceValues	Values corresponding to the choices.

Example: Using checkboxGroupInput

library(shiny)

ui <- fluidPage(
  checkboxGroupInput("choices", "Select Options:",
                     choiceNames = list("Apple", "Banana", "Cherry", "Date"),
                     choiceValues = list("apple", "banana", "cherry", "date")),
  textOutput("selection")
)

server <- function(input, output) {
  output$selection <- renderText({
    paste("You selected:", paste(input$choices, collapse = ", "))
  })
}

shinyApp(ui = ui, server = server)

8. textInput(): This function creates a text input box for users to enter text.

Syntax:

textInput(inputId, label, value = "", width = NULL, placeholder = NULL)

Parameters:

Parameter	Description
inputId	The ID of the input element, used to retrieve the value in the server function.
label	The text label displayed for the input box.
value	The initial value of the input box (optional).
width	Specifies the width of the input box (e.g., ‘300px’, ‘50%’).
placeholder	Provides a hint about the expected input in the box.

Example: Simple Text Input and Display

# Load Shiny library
library(shiny)

# UI layout
ui <- fluidPage(
  textInput("userText", "Enter text here:", "Type something"),
  verbatimTextOutput("displayText")
)

# Server logic
server <- function(input, output) {
  output$displayText <- renderText({ input$userText })
}

# Create Shiny app
shinyApp(ui = ui, server = server)

Output:
A text input box appears, where users can type text. The entered text is displayed below the input box.

9. textOutput():
This function creates an output text element to display reactive text in your Shiny app.

Syntax:

textOutput(outputId, container = if (inline) span else div, inline = FALSE)

Parameters:

Parameter	Description
outputId	The ID used to access the output text in the server.
container	A function (e.g., div, span) that wraps the output HTML element.
inline	Boolean value indicating if the output should be displayed inline or block.

Example: Welcome Message

# Load Shiny library
library(shiny)

# UI layout
ui <- fluidPage(
  textInput("userName", "Enter your name:"),
  textOutput("welcomeText")
)

# Server logic
server <- function(input, output, session) {
  output$welcomeText <- renderText({
    paste("Hello,", input$userName, "! Welcome to the Shiny app.")
  })
}

# Create Shiny app
shinyApp(ui = ui, server = server)

10. wellPanel():
This function creates a bordered box with a gray background to highlight specific elements in your app.

Syntax:

wellPanel(...)

Output:
A text input box appears, where users can type text. The entered text is displayed below the input box.

Parameters:

Parameter	Description
...	UI elements to be placed inside the panel.

Example: Histogram Inside a Panel

# Load Shiny library
library(shiny)

# UI layout
ui <- fluidPage(
  sliderInput("numValues", "Choose a number:", min = 10, max = 100, value = 50),
  wellPanel(
    plotOutput("histPlot")
  )
)

# Server logic
server <- function(input, output) {
  output$histPlot <- renderPlot({
    hist(rnorm(input$numValues), col = "lightblue", main = "Sample Histogram")
  })
}

# Create Shiny app
shinyApp(ui = ui, server = server)

Output:
A histogram is displayed inside a gray-bordered well panel. The number of data points is controlled by a slider.

Enhanced Example: Interactive Scatter Plot

# Load Shiny library
library(shiny)

# UI layout
ui <- fluidPage(
  titlePanel("Interactive Scatter Plot"),
  sidebarLayout(
    sidebarPanel(
      numericInput("numPoints", "Number of Points:", value = 50, min = 10, max = 100),
      br(),
      actionButton("updateBtn", "Generate Plot")
    ),
    mainPanel(
      plotOutput("scatterPlot", height = "400px")
    )
  )
)

# Server logic
server <- function(input, output) {
  # Reactive function to generate data
  scatterData <- reactive({
    data.frame(
      x = rnorm(input$numPoints),
      y = rnorm(input$numPoints)
    )
  })

  # Render scatter plot
  observeEvent(input$updateBtn, {
    output$scatterPlot <- renderPlot({
      plot(
        scatterData()$x, scatterData()$y,
        main = "Scatter Plot",
        xlab = "X-axis", ylab = "Y-axis",
        col = "blue", pch = 19, xlim = c(-3, 3), ylim = c(-3, 3)
      )
    })
  })
}

# Create Shiny app
shinyApp(ui = ui, server = server)

Output:
An interactive scatter plot is displayed. Users can control the number of points with a numeric input and update the plot using a button.

Grid and Lattice Packages in detail

Every programming language offers packages to implement various functions. In R programming, packages bundle related functions to streamline development. To utilize these functions, installing and loading the respective packages is necessary. The CRAN repository hosts over 10,000 R packages. Notable packages like Grid and Lattice in R are used to implement graphical functions to create visual outputs such as rectangles, circles, histograms, bar plots, etc.

Grid Package in R

The Grid package, previously part of the CRAN repository, is now included as a base package in R. It serves as the foundation for advanced graphical functions in other packages like lattice and ggplot2. Moreover, it can modify lattice-generated outputs. Being a base package, it doesn’t require separate installation as it comes pre-installed with R.

To load the Grid package, use the following command in the console and select “grid” when prompted:

local({pkg <- select.list(sort(.packages(all.available = TRUE)), graphics = TRUE)
if(nchar(pkg)) library(pkg, character.only = TRUE)})

The Grid package provides several functions to create graphical objects, also known as “grobs.” Some of the functions include:

• circleGrob
• linesGrob
• polygonGrob
• rasterGrob
• rectGrob
• segmentsGrob
• legendGrob
• xaxisGrob
• yaxisGrob

To see the complete list of functions in the Grid package, use the following command:

library(help = "grid")

Example: Using the Grid Package

The following example demonstrates how to create and save graphical objects using the Grid package:

library(grid)

# Save output as a PNG file
png(file = "grid_example.png")

# Create a circular grob
circle <- circleGrob(name = "circle", x = 0.4, y = 0.4, r = 0.3,
                     gp = gpar(col = "blue", lty = 2))

# Draw the circle grob
grid.draw(circle)

# Create a rectangular grob
rectangle <- rectGrob(name = "rectangle", x = 0.6, y = 0.6,
                      width = 0.4, height = 0.3,
                      gp = gpar(fill = "lightgreen", col = "darkgreen"))

# Draw the rectangle grob
grid.draw(rectangle)

# Save the file
dev.off()

Output

Lattice Package in R

The Lattice package builds upon the Grid package to create Trellis graphics. These graphics are particularly useful for visualizing relationships between multiple variables under different conditions.

Installing the Lattice Package

The Lattice package can be installed using the following command:

install.packages("lattice")

Lattice provides various graph types, including:

• barchart
• contourplot
• densityplot
• histogram

The general syntax for using these graphs is:

graph_type(formula, data)

• graph_type: Specifies the type of graph to generate.
• formula: Defines the variables or conditional relationships.

To view all functions in the Lattice package, use:

library(help = "lattice")

Example 1: Density Plot

library(lattice)

# Use the built-in mtcars dataset

# Save output as a PNG file
png(file = "density_plot_example.png")

# Create a density plot for the variable 'mpg'
densityplot(~mpg, data = mtcars,
            main = "Density Plot of MPG",
            xlab = "Miles per Gallon")

# Save the file
dev.off()

Output:

Example 2: Histogram

library(lattice)

# Use the built-in ToothGrowth dataset

# Save output as a PNG file
png(file = "histogram_example.png")

# Create a histogram for the variable 'len'
histogram(~len, data = ToothGrowth,
          main = "Histogram of Length",
          xlab = "Length")

# Save the file
dev.off()

Output:

Both the Grid and Lattice packages offer powerful tools for graphical representations in R, making it easier to visualize and analyze data effectively.