---
output:
  html_document:
      toc: true
      fig_width: 4.5
      fig_height: 3.25
      css: http://web.pdx.edu/~gerbing/styles/stylesRmd.css
title: "Visualize the Distribution of a Categorical Variable"
subtitle: |
          |  Basic Visualizations
          |
          |  ISQA 521
author: |
        | David W. Gerbing
        | The School of Business
        | Portland State University
date: "`r format(Sys.time(), '%A %B %d, %Y at %H:%M')`"
---

Access the needed functions by retrieving from the R library, here **lessR**, **ggplot2**, and **waffle**, which, of course, requires that all three packages are first installed. Usually access **ggplot2** from the *tidyverse* functions, which include **ggplot2**.

```{r }
library(lessR)
library(tidyverse)
library(waffle)
```


In this chapter we construct a bar chart from a single categorical variable. Much of the **lessR** perspective is described in my recent  article: <a href="https://towardsdatascience.com/easy-bar-charts-from-simple-to-sophisticated-7270c03eced8?source=friends_link&sk=0a724c1bb0317fc494ae80fa0ae7b58b" target="_blank" rel="noopener noreferrer">medium.com</a> article. This link provides free access. The two-variable bar charts are covered next week in this course.

A *bar chart* visualization associates a numerical value, plotted as the height of a bar, with each value of a categorical variable, plotted on the other axis. A bar chart of a single variable, then, involves two variables, a categorical variable $x$ and a numerical variable $y$. The values of the numerical variable can be provided as almost any numerical variable. The usual default application is to tabulate the counts of the number of occurrences of each level of the categorical variable.


## Bar Chart from the Measurements


**lessR** can assign variable labels, a more informative description of the meaning of the variable than the variable name.  **lessR** uses the labels to enhance visualizations and the text output to the console. For example, by default an axis is labeled with the variable name. If variable labels are present, the corresponding **lessR** function labels an axis with the variable name followed by the variable label.

The file of variable labels, such as constructed in Excel, consistes of two columns: the variable name in the first column, and then, if present, the corresponding variable label in the second column. Here read such a variable label file into R with the `var_labels` parameter set to `TRUE` to inform the `Read()` function that labels are to be read, not regular data. Then display the data frame, *l*, of variable names and labels.


```{r }
#  l <- vl("http://lessRstats.com/data/employee_lbl.csv")
l <- rd("Employee_lbl", var_labels=TRUE)
l
```


One common application of a bar chart applies to counts.  That is, the $y$-variable in these bar charts is the tabulated frequency for each category in a data table.

These bar charts can be generated from two different forms. Some bar chart functions require the tabulation be done separately, and only the results passed to the function, which are then plotted, as with the previous examples. Other bar chart functions, from **lessR** and **ggplot2**, require only the input of only a single categorical variable, from which the tabulation is automatically provided by the function.


```{r }
d <- Read("Employee")  # get some data
```

Both **lessR** and **ggplot2** automatically provide the tabulation from the variable specification. Here is the **lessR** 1-variable bar chart specification, using the abbreviation `bc()` and the default data table name of *d*.  
For both **lessR** and **ggplot2**, specify tabulation to compute the values of the $y$-variable by the absence of a $y$-variable in the function call. No given $y$-variable informs the function to compute it, the corresponding height of each bar. Note that the $x$-axis is labeled with the variable label *Department Employed* instead of just the variable name.

```{r }
bc(Dept)
```

Now that we have seen that **lessR** console output for the accompanying statistical analysis, with the `style()` function turn off console output for the remaining **lessR** output for a more compact presentation. Do not need to see the frequency distributions again.

```{r }
style(quiet=TRUE)
```

The default **ggplot2** bar chart with automatic tabulation to define the numerical variable to plot involves three different functions: `ggplot()` to set up the plot, `aes()`to define the variable(s), and, in this situation, `geom_bar()` to indicate the object plotted, bars.

```{r dataTable, echo=FALSE, out.width='42%', fig.asp=.7, fig.align='center', fig.cap="ggplot2 structure for a default bar chart."}
knitr::include_graphics(rep("images/geom_bar.png"))
```

The function call follows. By default, the plot is in grayscale.

```{r }
ggplot(d, aes(Dept)) + geom_bar()
```

The base R bar chart function, `barplot()` requires that the tabulations first be computed, and then passed to the bar chart function, which directly plots the values passed to it. First get the counts with the `table()` function, and then plot the counts with `barplot()`. Must reference the data frame that contains the variable as well as there is no `data` parameter. Do so by prefix the variable name with the data frame name that contains the variable, followed by a $.

```{r }
barplot(table(d$Dept))
```

Or, to improve readability, separate the tabulation from the generation of the bar chart. Store the table of counts in the object we name *mycounts*. 
```{r }
mycounts <- table(d$Dept)
mycounts  # not necessary, but shows the tabulated counts
barplot(mycounts)
```

There are reasons why most no longer use base R functions to plot. The output is not so appealing without doing more work, and also more work to just get the default plot.


## Bar Chart from Summary Data

Any bar chart from any software system is constructed from a summary (pivot) table, such as the counts for each level of a categorical variable. Summary tables can also be calculated for other statistics such as the mean. When applied to counts, the process is called *tabulation*. When applied to a particular statistic in general, the process is called *aggregation*. Within Excel, the output is called a *pivot table*.

As we saw with tabulation, the **lessR** and **ggplot2** bar chart functions can provide this tabulation automatically. The summary table, such as tabulated values, however, may be all that is available, not the data file of measurements from which the tabulation or other aggregation occurred. 

To illustrate, although we have the original measurements, to illustrate how to input the summary table into the bar chart functions we will get an example summary table by constructing it. The following explains how to derive the corresponding summary table outside of the bar chart function itself. One approach comes from the tidyverse including **ggplot2**, the other from **lessR**.

### Aggregate the Measurements Data

Many statistics can be aggregated by groups. Here get average Salaries for each *Dept*. In Excel, this aggregation results in what is called a *pivot table*. Same result, just different name.

The first approach here is with the tidyverse functions, of which **ggplot2** is a part. Chain the operations together in a sequence from the initial *d* data frame to the aggregated *avgd* data frame with the pipe operator, `%>%`. This operator takes the output of the preceding function and transfers that output, by default, into the first parameter of the receiving function. For these functions here, the first parameter is the `data` parameter, which now is not explicitly listed when using the chain operator because it receives that input direct from the chain operator.

The tidyverse functions for aggregation (pivoting) are `group_by()`, which identifies the groups, followed by `summarize()`, which does the computations. Within the `summarize()` function, specify the statistic for which to perform the aggregation, here `mean()`, and then name the result, here *Avg*. 

The `na.omit()` function removes any missing data, which otherwise is reported by **ggplot** functions. Not a right or wrong, but a preference, depending on the analysis. (**lessR** never plots missing data, but does report the number of missing values in its text output. **ggplot** does not provide text output, so plotting is the only way to report it.)

The following tidyverse code instructs R to take the *d* data frame and input it into the `group_by()` function to identify the groups of data, one group for each *Dept*. Then take those groups an input that information into the `summarize()` function, computing the mean Salary of each group, and name the result *Avg*. Then remove any rows with missing data. Then take that result into the newly created data frame *avg*.


```{r }
d %>% group_by(Dept) %>% summarize(Avg=mean(Salary)) %>% na.omit(d) -> avgd
```

Display the averages.

```{r }
avgd
```

Or, use the less verbose, simpler **lessR** function `pivot()` to aggregate the mean of *Salary* across values of *Dept*. The function also provides, for a single variable aggregated, the sample size of each group. Here save the result into the new data frame *avgd*. Then display the contents of *avg*.

```{r }
avgd <- pivot(d, mean, Salary, Dept)
avgd
```

To further clarify, here call the `pivot()` function with the parameters explicitly named.

```{r}
pivot(data=d, compute=mean, value=Salary, by=Dept)
```

For the **lessR** `BarChart()` function, analyze the data values for the categorical variable, assigned to the x-variable, *Dept*, in terms of the corresponding values of the numerical variable. The numerical value of the variable *Avg* define the $y$-variable. Specifying a $y$-variable for `BarChart()` indicates that the input data are in summary (pivot) table form.

Use the *data* parameter because the data table is not the default name of *d*, so must be explicitly named.

```{r }
BarChart(Dept, Salary, data=avgd)
```

Here make all declarations explicit.

```{r }
BarChart(x=Dept, y=Salary, data=avgd, values="%", quiet=TRUE)
```

ggplot2.

1. begin with the **ggplot** function.
   * first list the name of the data table
   * then invoke the `aes()` function, for aesthetics, to specify the names of the $x$-variable and, if present, the $y$-variable.
2. add at least one layer that specifies the type of geometric object to plot, of the form `geom_whatever()`.

For a bar chart, the object to plot is a bar, use the `geom_bar()` function. The `stat="identity"` argument informs **ggplot** to *not* tabulate the data, but instead directly plot the data values.

```{r }
ggplot(avgd, aes(Dept, Salary)) + geom_bar(stat="identity")
```

Or use a different function, `geom_col()`.

```{r }
ggplot(avgd, aes(Dept, Salary)) + geom_col()
```

Base R constructs the plot directly from the numerical vector. The labels for each bar are optional, but if supplied they are declared with the `names.arg` parameter. There is no `data` parameter for `barplot()`, so must specify the name of the data table with the variable name to identify its location. Follow the data table name with a $.

```{r }
barplot(avgd$Salary, names.arg=avgd$Dept)
```


### The BarChart() Function Aggregates

The **lessR** bar chart function can also do the aggregations direct from the measurements with the `stat` parameter. Possible values include `"sum"`, `"mean"`, `"sd"`, `"dev"`, `"min"`, `"median"`, and `"max"`.

```{r}
BarChart(Dept, Salary, stat="mean")
```

The value `"dev"` indicates to plot the mean deviations, which can make the comparison of values more explicit. The `sort` parameter indicates to sort, which again facilitates the comparison.

```{r}
BarChart(Dept, Salary, stat="dev", sort="+")
```

We also include a new option, fill_sort, which provides two colors for the bars, here split at 0 to differentiate between negative and positive deviations.

```{r}
BarChart(Dept, Salary, stat="dev", sort="+", fill_split=0)
```



## Horizontal plot

### lessR

**lessR** horizontal bar chart with the R parameter horiz=TRUE.

```{r }
bc(Dept, horiz=TRUE)
```


### ggplot2

Generate a horizontal **ggplot2** bar chart with `coord_flip()`.

```{r }
ggplot(d, aes(Dept)) + geom_bar() + coord_flip()
```


## Process Long Labels

### lessR

**lessR** addresses long labels with label.max parameter to abbreviate the values of a categorical variable, and also rotate.values, to not display horizontally on the x-axis, and offset to displace some distance down.

To illustrate, get some data with a categorical variable with long value names, the variable Make in the included data set Cars93.

```{r chunk-Q6_bc}
myd <- rd("Cars93")
str(myd$Make)  # R show structure function, 32 levels of Make
bc(Make, data=myd)
```


`rotate.x` is a **lessR** parameter to rotate the $x$-axis labels the specified degrees
With so many bars, better to turn the border color off and rotate the labels.
`offset` is an R parameter in conjunction with `rotate_x` to align the labels.
First the vertical bar chart.

```{r }
bc(Make, rotate_x=90, offset=1.4, data=myd)
bc(Make, horiz=TRUE, data=myd)
```


### ggplot2

Rotate values and offset a bit. No abbreviations. **ggplot2** accurately adjusts the space of the plot. The `hjust` parameter moves the labels down vertically.

```{r }
ggplot(myd, aes(Make)) + geom_bar() +
  theme(axis.text.x = element_text(angle=45, hjust=1))

# Can also add abbreviations.
ggplot(myd, aes(Make)) + geom_bar() +
  theme(axis.text.x = element_text(angle=45,  hjust=1)) +
  scale_x_discrete(labels = abbreviate)
```


## Bubble Plot Alternative

As a second alternative to the standard bar chart, consider a form of the scatter plot called a bubble plot, in which the points are plotted according to the coordinates of the relevant categorical variable. Because the variable is categorical, the coordinates are just the positive integers. To avoid the over-plotting problem in which multiple values are plotted at the same coordinate,  the size of each plotted point is a function of the frequency of occurrence for that value.

The `Plot()` function determines the type of the submitted variable, here categorical. The resulting bubble plot form of a scatterplot is more compact than the traditional bar chart.

```{r }
Plot(Dept)
```

Can also impose a color gradient. (See all color names and associated colors with the **less** `showColors()` function.)

```{r }
Plot(Dept, low="lemonchiffon2", hi="maroon3")
```

Value labels are automatically printed for formal R factors, but can also add them to integer scored variables without declaring a factor.

```{r }
Plot(Gender, value.labels=c("Female", "Male"))
```


# Part - Whole Plots

## Pie Chart

Invoke the **lessR** function `PieChart()`, abbreviation `pc()`, which by default provides a more modern version, the *ring* or *doughnut* chart. The `hole` parameter sets the size of the hole.

```{r }
PieChart(Dept)
PieChart(Dept, hole=0, fill="grays")

ggplot(na.omit(d), aes(x="", y=stat(count), fill=Dept)) +
  geom_bar() + coord_polar(theta = "y") +
  theme_void() +
  theme(legend.text=element_text(size=12)) +
  theme(legend.title=element_text(siz=15))
```


## Waffle Plot

A *waffle plot* is a square pie plot. The `waffle()` function is from the **waffle** package, so it must be installed separately, and then the library function invoked. It accepts as input the already tabulated vector of counts for each category, similar to the base R `barplot()` function, so first invoke the base R `table()` function.

```{r }
d <- rd("Employee")
myFreq <- table(d$Dept)
waffle(myFreq)
waffle(myFreq, flip=TRUE, xlab="Employees by Dept")
```

The waffle plot is not automatically adjusted for scale. So if plotting data with thousands of data values, best to first reduce the scale, such as by:

```
myFreq <- round(myfreq/100)
```

# Cleveland Dot Plot

**lessR** offers two alternatives to the standard bar chart, both presented as forms of a scatterplot, obtained with the **lessR** `Plot()` function.

First consider a scatterplot visualization as an alternative to the bar plot, a scatterplot of the counts for each category, a *Cleveland Dot Plot*. To enhance readability, the scatterplot defaults to a larger point size, and connects each point to the horizontal axis with a line segment. Because there is no over-plotting, the points are not translucent, but opaque. Specify to plot the counts instead of the original data values with the `stat` parameter, which indicates to plot a transformation of the data values, instead of the original data values.

```{r }
Plot(Dept, stat="count")
```

Can also request proportions instead of frequencies. Here use the standard R parameter, `xlim`, to specify a custom range for the y-axis.

```{r }
Plot(Dept, stat="prop")
Plot(Dept, stat="prop", xlim=c(0,0.5))
```



# Plot Multiple Distributions

## Stack Many Bar Charts

One useful application is to view the responses to standard survey Likert items, that is, with responses Strongly Disagree, Disagree, ... Strongly Agree. The **lessR** function `factors()` converts the numerical item responses, 0 to 5, to the actual response categories.

```{r }
d <- rd("Mach4", quiet=TRUE)
d[1:4,]
l <- vl("http://lessRstats.com/data/Mach4_lbl.xlsx")

LikertCats <- c("Strongly Disagree", "Disagree", "Slightly Disagree", "Slightly Agree", "Agree", "Strongly Agree")
d <- factors(m01:m20, levels=0:5, labels=LikertCats)
```

To specify all bar charts on a single bar chart, just reference a vector of multiple variables.

```{r Mach20, fig.height=5}

BarChart(m01:m20)
```


## Bubble Plot Frequency Matrix

**lessR** generalizes the form of the bubble plot from that for a single variable to a plot of multiple variables, specified as a vector, where, to be meaningful, a vector of categorical variables. There are several ways in R to form a vector, but the most general invokes the `c()`, or concatenate function, to group any specified, valid entities of the same type. Here the vector of two categorical variables in the Employee data set is c(Dept, Gender).

The resulting plot is called here a Bubble Plot Frequency Matrix (BPFM), basically a set of stacked frequency bubble charts.

```{r }
d <- Read("Employee", quiet=TRUE)  # get some data
Plot(c(Dept, Gender))
```

The preceding plot is relatively compact, but grows wider with additional response alternatives. The BPFM is more meaningfully applied to a vector of multiple categorical variables all sharing the same response scale. A prime example of this configuration is standard Likert scale data from survey research in which respondents provide responses along a continuum of agreement with various items. One such response scale and corresponding coding scheme follows.

0. Strongly Disagree
1. Disagree
2. Slightly Disagree
3. Slightly Agree
4. Agree
5. Strongly Agree

To illustrate, consider Likert data from 351 respondents who completed a Machiavellianism scale called the Mach IV. The data set is included with lessR.

```{r }
d <- Read("Mach4", in.lessR=TRUE)
```

Items m06, m07, m09 and m10 form what is called the Deceit subscale. Here is their BPFM. The `n.cat` parameter specifies that for a variable with unique values of integers from 0 to 5, treat any variable with six or less evenly spaced integer values as a categorical variable.

```{r }
Plot(c(m06, m07, m09, m10), n.cat=6)
```

For all the items.

```{r Mach20BP, fig.height=9}
Plot(m01:m20, n.cat=6, sort.yx="+")
```


## Multiple Plots


```{r }
d <- Read("Employee", quiet=TRUE)  # get some data
```

First analysis in any project is to count the values of the variables. **lessR** provides for counting the values of all categorical variables with one function call. Make the first argument passed to the function a data frame instead of a variable. The default data frame of *d* still applied.


```{r }
bc()
```

