Factor Analysis

class: center, middle, inverse, title-slide

.title[
# Factor Analysis
]
.date[
### 2022/05/10
]

---

# Psychometrics

Many of the things we want to measure are *constructs* that are not *directly* measurable. e.g. IQ, anxiety, risk

.pull-left[
.center[
![](https://upload.wikimedia.org/wikipedia/commons/thumb/d/d5/Brooklyn_Museum_-_Blind_Men_Appraising_an_Elephant_-_Ohara_Donshu.jpg/220px-Brooklyn_Museum_-_Blind_Men_Appraising_an_Elephant_-_Ohara_Donshu.jpg)
]
]
.pull-right[

"A group of blind men heard that a strange animal, called an elephant, had been brought to the town, but none of them were aware of its shape and form. Out of curiosity, they said: "We must inspect and know it by touch, of which we are capable"."

]

???

Blind Men Appraising an Elephant by Ohara Donshu, Edo Period (early 19th century), Brooklyn Museum. From [Wikipedia](https://en.wikipedia.org/wiki/Blind_men_and_an_elephant)

---
# Psychometrics

Many of the things we want to measure are *constructs* that are not *directly* measurable. e.g. IQ, anxiety, risk

We can try to capture different *aspects* of latent variables.

For example, we might ask a variety of different questions as with standard scales and questionnaires like
- HEXACO
- Historical Clinical Risk Management-20 (HCR-20)
- Patient Health Questionnaire 9 (PHQ-9)
]

???

Blind Men Appraising an Elephant by Ohara Donshu, Edo Period (early 19th century), Brooklyn Museum. From [Wikipedia](https://en.wikipedia.org/wiki/Blind_men_and_an_elephant)

---
# The HEXACO personality measures

The HEXACO scale measures personality using 60 or 100 item questionnaires.

These questionnaires supposedly breaks personality down into six different factors:

-  Honesty-Humility
-  Emotionality
-  eXtraversion [sic]
-  Agreeableness
-  Conscientiousness
-  Openness to Experience

---
#Example HEXACO items

---
class: inverse, center, middle
# Performing factor and component analysis

---
# Graphical representation of factor analysis

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-2-1.png)
]
.pull-right[

Each axis is a *dimension* relating to an underlying construct.

In this example, based on the HEXACO scale, the x-axis represents the *Honesty-Humility* dimension, while the y-axis represents the *Emotionality* dimension.

]

---
# Graphical representation of factor analysis

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-3-1.png)
]
.pull-right[
Each dot represents the score on an individual item.

The points that cluster together are correlated and are measuring part of the same underlying dimension.

We can shift the *axes* to pass through these points. 
]

---
# Factor loadings

.pull-left[

Items that measure the *Emotionality* factor cluster - or **load** - high on the y-axis.

Items that measure the *Honesty-Humility* factor load high on the x-axis.

]
.pull-right[
![](6-Factor-Analysis_files/figure-html/class-dim-1.png)
]

---
# Factor loadings

.pull-left[
The distance of an item from zero on a particular dimension indicates how heavily the item *loads* on that dimension.

Items that measure the *Emotionality* factor cluster - or **load** - high on the y-axis.

Items that measure the *Honesty-Humility* factor load high on the x-axis.

]
.pull-right[
![](6-Factor-Analysis_files/figure-html/both-facs-1.png)
]

---
# Factor loadings

.pull-left[
The items that load on the Honesty-Humility axis are close to the centre of the *y-axis*, but distant from zero on the *x-axis*.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/fac-load-emo-1.png)
]

---
# Factor loadings

.pull-left[
The items that load on the Honesty-Humility axis are close to the centre of the *y-axis*, but distant from zero on the *x-axis*.

The items that load on the Emotionality factor are close to the centre of the *x-axis*, but distant from zero on the *y-axis*.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/fac-dim-hum-1.png)
]

---
# Factor loadings

.pull-left[
Let's add a third set of items, a set of items that correlate with each other but not with either existing cluster.

These clearly load negatively on both our existing factors, but we may need another factor to characterise them properly.
]

.pull-right[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-4-1.png)
]

---
# Factor loadings

.pull-left[
Let's add a third set of items, a set of items that correlate with each other but not with either existing cluster.

These clearly load negatively on both our existing factors, but we may need another factor to characterise them properly.

For each distinct *factor*, we need an additional *dimension*.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-5-1.png)
]

---
class: inverse, center, middle
# Preparing for factor analysis

---
# The *R-matrix*

.pull-left[
*Correlations* are at the heart of how we understand which of our questionnaire items measure the same *factors*.

There are 60-item and 100-item versions of the HEXACO.

Here we take a look a small subset of those items.

We have *two clusters* of items that correlate with *with each other* but not with the items in the *other* cluster.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/corr-dims-1.png)
]

---
# The identity matrix

The matrix on the right is the *identity* matrix - this is what the correlation matrix would be like without structure.

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-6-1.png)
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-7-1.png)
]

---
# Checking the *R-matrix*

We need to know whether there is sufficient correlative structure in the data!

The *Bartlett* test - run using the `check_sphericity_bartlett()` function from `parameters` - is used to check whether the correlation matrix significantly differs from the *identity* matrix.
<style type="text/css">
pre code {
  white-space: pre-wrap;
}
</style>

```r
check_sphericity_bartlett(hexaco_subset)
```

```
## # Test of Sphericity
## 
## Bartlett's test of sphericity suggests that there is sufficient significant correlation in the data for factor analysis (Chisq(15) = 187.45, p < .001).
```

---
# Checking sampling adequacy

We also need to know if there is enough variability in the data.

The Kaiser-Meyer-Olkin statistic measures the degree to which each variable in the data can be predicted from the other variables.

KMO ranges from 0 to 1; values above .7 are generally considered acceptable.

```r
check_kmo(hexaco_only)
```

```
## # KMO Measure of Sampling Adequacy
## 
*## The Kaiser, Meyer, Olkin (KMO) measure of sampling adequacy suggests that data seems appropriate for factor analysis (KMO = 0.77).
```

---
# Checking for sufficient factor structure

The `check_factorstructure()` function from `parameters` does both of these at once!

```r
check_factorstructure(hexaco_only)
```

```
## # Is the data suitable for Factor Analysis?
## 
##   - KMO: The Kaiser, Meyer, Olkin (KMO) measure of sampling adequacy suggests that data seems appropriate for factor analysis (KMO = 0.77).
##   - Sphericity: Bartlett's test of sphericity suggests that there is sufficient significant correlation in the data for factor analysis (Chisq(1770) = 7153.57, p < .001).
```

---
class: inverse, middle, center
# How many factors do we need?

---
# How many factors do we need?

.pull-left[
We need to figure out how many factors we need to break down our data.

In theory, we could have one per item.

... but that would be a lot of factors.

Here, it looks like there are at least five different groups.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-12-1.png)
]

---
# Scree plots

.pull-left[
Catell (1966) proposed the scree plot as a way to choose how many factors to keep.

The y-axis shows the eigenvalue of each potential factor, up to the maximum number possible.

```r
scree(hexaco_only, factors = FALSE)
```
]

.pull_right[
![](6-Factor-Analysis_files/figure-html/scree-plot1-1.png)
]

---
# Eigenvalues

Eigenvalues tell us how much variance a particular factor explains.

Higher values mean more variance explained, and the more variance a factor explains, the more important it is.

They help us determine whether a factor is worth *extracting* for further analysis.

```r
eigen(cor(hexaco_only))$values
```

```
##  [1] 6.6794022 4.9331351 4.1642486 3.6237441 3.1055095 2.4175969
##  [7] 1.8303788 1.7109257 1.6276844 1.4397559 1.3272585 1.2604329
## [13] 1.1922544 1.1163815 1.0921884 1.0615500 0.9873230 0.9548026
## [19] 0.9119362 0.8633100 0.8313116 0.7990644 0.7715206 0.7308254
## [25] 0.7084408 0.6727497 0.6619408 0.6319305 0.6145891 0.5917170
## [31] 0.5852278 0.5682678 0.5418668 0.5317604 0.5112533 0.4972359
## [37] 0.4697304 0.4557374 0.4375651 0.4280798 0.4018966 0.3945424
## [43] 0.3844344 0.3547073 0.3484113 0.3357983 0.3289566 0.3065944
## [49] 0.2977506 0.2890174 0.2725412 0.2599978 0.2595737 0.2436230
## [55] 0.2312924 0.2221693 0.2179546 0.1937423 0.1609424 0.1554209
```

---
# Scree plots

.pull-left[
We look for the *point of inflexion* - the point at which the eigenvalues have (more or less) stopped decreasing much.

It's probably around 9 components here!
]

.pull_right[
![](6-Factor-Analysis_files/figure-html/screeplotmark-1.png)
]

---
# Kaiser's criterion

.pull-left[
An alternative to looking for the point of inflexion is to keep any factor where the eigenvalue is higher than 1 - this is called *Kaiser's criterion*.

This would pick out around *16* factors here.

Kaiser's criterion tends to keep too many factors.
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-14-1.png)
]

---
# Parallel analysis

.pull-left[
Arguably the best method is **Parallel Analysis**, using the `fa.parallel()` function.

In parallel analysis, *random* data is generated and compared to the *true* data.

Factors above the *red* line should be kept. Here, it's 9, just like our "point of inflexion" rule would suggest.

```r
fa.parallel(hexaco_only,
            fa = "pc")
```

```
## Parallel analysis suggests that the number of factors =  NA  and the number of components =  9
```
]

.pull-right[
![](6-Factor-Analysis_files/figure-html/parallel-fa-1.png)
]

---
class: center, middle, inverse
# Principal Component Analysis

---
# Principal Component Analysis

There are a number of differenct factor analysis methods available. We'll look at PCA. PCA is a *dimension reduction* method.

It produces a *simplified model* of the data that captures the inter-relationships between variables.

To run PCA on this kind of data, we can use the **principal()** function from the **psych** package.

```r
?principal
```

---
# Principal Component Analysis

Having decided we need nine factors, we use the **principal()** function to extract them from the data.

```r
pca_hexa <- principal(hexaco_only,
                      nfactors = 9,
                      rotate = "varimax")
```

(Output is on the next slide!)

---

```r
pca_hexa
```

```
## Principal Components Analysis
## Call: principal(r = hexaco_only, nfactors = 9, rotate = "varimax")
## Standardized loadings (pattern matrix) based upon correlation matrix
##            RC1   RC2   RC3   RC6   RC9   RC5   RC4   RC7   RC8   h2
## hexaco1  -0.17  0.00  0.74  0.03 -0.01 -0.03  0.09  0.04  0.05 0.59
## hexaco2  -0.01  0.21  0.02  0.10 -0.05 -0.58 -0.03  0.24  0.17 0.48
## hexaco3   0.59 -0.06 -0.09  0.19 -0.15  0.06  0.02  0.01  0.15 0.44
## hexaco4   0.15 -0.12  0.08  0.17 -0.59 -0.25  0.09  0.23  0.09 0.56
## hexaco5   0.10  0.50  0.03 -0.14 -0.06 -0.06 -0.08  0.03  0.01 0.30
## hexaco6   0.14  0.02  0.06  0.64  0.06 -0.03  0.00  0.06 -0.24 0.49
## hexaco7  -0.01 -0.06 -0.59  0.04  0.06 -0.14  0.07 -0.03 -0.08 0.39
## hexaco8  -0.01  0.10 -0.03  0.13 -0.23 -0.19  0.20  0.61  0.06 0.53
## hexaco9  -0.58 -0.10 -0.12 -0.12  0.08  0.00 -0.03  0.07  0.19 0.42
## hexaco10 -0.11 -0.27  0.09 -0.66  0.15  0.21  0.02  0.08 -0.03 0.60
## hexaco11  0.15  0.16  0.32 -0.12  0.02  0.14 -0.64  0.12  0.01 0.60
## hexaco12 -0.05  0.53  0.00  0.02  0.37 -0.05 -0.07  0.26  0.07 0.50
## hexaco13  0.00  0.07 -0.65  0.05 -0.01  0.36 -0.14  0.21 -0.08 0.63
## hexaco14 -0.11 -0.12  0.19 -0.06 -0.05  0.30  0.01 -0.48  0.20 0.43
## hexaco15 -0.65 -0.01  0.01  0.00  0.10  0.14 -0.05  0.24  0.07 0.52
## hexaco16  0.09  0.16  0.21  0.01 -0.21  0.14  0.65  0.06 -0.13 0.58
## hexaco17  0.14  0.67  0.04 -0.03  0.09  0.03  0.15  0.18  0.00 0.54
## hexaco18  0.22  0.03 -0.05  0.46  0.11  0.08  0.08  0.17  0.08 0.33
## hexaco19 -0.11  0.01  0.33  0.11 -0.19  0.05 -0.21 -0.11  0.47 0.45
## hexaco20  0.05 -0.12  0.27 -0.16  0.08  0.64  0.05 -0.15  0.13 0.57
## hexaco21 -0.70 -0.01  0.16 -0.06  0.07  0.13  0.07 -0.04  0.20 0.58
## hexaco22  0.24  0.06  0.16  0.10 -0.67 -0.08  0.21  0.22  0.04 0.64
## hexaco23  0.18  0.58  0.06  0.21 -0.01  0.07  0.13  0.10  0.05 0.45
## hexaco24 -0.02 -0.05 -0.01 -0.25  0.05  0.05  0.05  0.05  0.74 0.62
## hexaco25  0.04 -0.07 -0.64  0.15  0.04  0.03  0.06  0.21  0.15 0.51
## hexaco26 -0.14 -0.02 -0.14  0.06  0.12  0.63 -0.01 -0.28  0.06 0.53
## hexaco27  0.62  0.04 -0.07  0.27 -0.05  0.09  0.15 -0.04  0.21 0.54
## hexaco28 -0.05  0.13 -0.04  0.05  0.65  0.07 -0.17  0.00  0.09 0.49
## hexaco29  0.14  0.53  0.02  0.00  0.11 -0.12 -0.17  0.12  0.14 0.39
## hexaco30 -0.03  0.14  0.02 -0.55 -0.06  0.11  0.01 -0.05  0.30 0.43
## hexaco31  0.03  0.04  0.62 -0.02 -0.02  0.26 -0.03  0.09 -0.14 0.48
## hexaco32 -0.02 -0.13  0.18 -0.16  0.36  0.28 -0.13 -0.40 -0.01 0.46
## hexaco33  0.62  0.14  0.00  0.15  0.04 -0.01  0.05  0.03  0.04 0.43
## hexaco34  0.01 -0.05 -0.06 -0.05 -0.19  0.10  0.75  0.05  0.09 0.63
## hexaco35  0.14 -0.61 -0.03 -0.04 -0.36  0.10  0.07 -0.26  0.08 0.61
## hexaco36  0.04  0.09 -0.14  0.67 -0.10 -0.09  0.08  0.14  0.06 0.53
## hexaco37 -0.14  0.07 -0.36  0.09 -0.23  0.39 -0.04  0.42  0.05 0.55
## hexaco38  0.11  0.11 -0.11  0.04 -0.11 -0.12  0.07  0.66 -0.03 0.51
## hexaco39  0.59  0.03  0.05  0.00 -0.10 -0.09  0.13  0.05 -0.01 0.39
## hexaco40  0.21  0.09  0.16 -0.02 -0.26  0.12  0.56  0.13 -0.03 0.49
## hexaco41  0.03 -0.70  0.00 -0.01 -0.06  0.01  0.03  0.11 -0.05 0.51
## hexaco42 -0.13  0.11  0.18 -0.44  0.07  0.02 -0.06  0.06  0.16 0.30
## hexaco43 -0.02 -0.20 -0.25  0.02  0.22  0.01  0.30  0.37 -0.20 0.41
## hexaco44 -0.13  0.01  0.12 -0.11  0.22  0.65  0.07 -0.21  0.12 0.58
## hexaco45  0.68 -0.06 -0.04  0.05  0.00  0.01 -0.07  0.09 -0.02 0.49
## hexaco46 -0.27 -0.10  0.06  0.06  0.52 -0.07 -0.38 -0.11 -0.01 0.53
## hexaco47  0.03  0.57  0.10  0.02  0.01 -0.05  0.07  0.09 -0.09 0.36
## hexaco48 -0.08 -0.03 -0.08 -0.30  0.18 -0.14  0.17  0.01  0.65 0.61
## hexaco49 -0.07  0.00  0.46  0.03  0.32 -0.36  0.09 -0.29 -0.08 0.55
## hexaco50 -0.17  0.16  0.05  0.01  0.01 -0.17 -0.04  0.63  0.09 0.50
## hexaco51  0.58  0.07  0.05  0.17  0.05  0.18 -0.13  0.12  0.01 0.44
## hexaco52 -0.07  0.27 -0.09 -0.03  0.66  0.21 -0.14  0.03  0.10 0.60
## hexaco53  0.06 -0.71  0.00  0.01  0.00  0.08  0.14  0.10  0.05 0.55
## hexaco54  0.16 -0.10  0.10  0.57  0.07  0.07 -0.09  0.10 -0.02 0.40
## hexaco55  0.12  0.08  0.56 -0.03 -0.14 -0.01 -0.18 -0.10  0.12 0.41
## hexaco56  0.10 -0.14 -0.03 -0.10  0.07  0.61  0.14  0.03 -0.12 0.45
## hexaco57 -0.52 -0.10 -0.02 -0.01  0.21  0.06  0.16  0.07  0.12 0.37
## hexaco58 -0.17 -0.17 -0.16  0.05 -0.12 -0.02  0.71  0.09  0.16 0.64
## hexaco59 -0.02 -0.71  0.06 -0.17  0.15  0.09 -0.01  0.00  0.12 0.58
## hexaco60 -0.09 -0.19  0.02 -0.69  0.16  0.17  0.02  0.07  0.03 0.57
##            u2 com
## hexaco1  0.41 1.2
## hexaco2  0.52 1.9
## hexaco3  0.56 1.6
## hexaco4  0.44 2.4
## hexaco5  0.70 1.4
## hexaco6  0.51 1.5
## hexaco7  0.61 1.2
## hexaco8  0.47 2.0
## hexaco9  0.58 1.6
## hexaco10 0.40 1.8
## hexaco11 0.40 2.1
## hexaco12 0.50 2.5
## hexaco13 0.37 2.0
## hexaco14 0.57 2.9
## hexaco15 0.48 1.5
## hexaco16 0.42 1.9
## hexaco17 0.46 1.4
## hexaco18 0.67 2.2
## hexaco19 0.55 3.1
## hexaco20 0.43 1.9
## hexaco21 0.42 1.4
## hexaco22 0.36 2.0
## hexaco23 0.55 1.8
## hexaco24 0.38 1.3
## hexaco25 0.49 1.5
## hexaco26 0.47 1.8
## hexaco27 0.46 1.9
## hexaco28 0.51 1.3
## hexaco29 0.61 1.8
## hexaco30 0.57 1.9
## hexaco31 0.52 1.5
## hexaco32 0.54 4.2
## hexaco33 0.57 1.3
## hexaco34 0.37 1.2
## hexaco35 0.39 2.3
## hexaco36 0.47 1.4
## hexaco37 0.45 4.0
## hexaco38 0.49 1.4
## hexaco39 0.61 1.2
## hexaco40 0.51 2.2
## hexaco41 0.49 1.1
## hexaco42 0.70 2.2
## hexaco43 0.59 4.9
## hexaco44 0.42 1.8
## hexaco45 0.51 1.1
## hexaco46 0.47 2.7
## hexaco47 0.64 1.2
## hexaco48 0.39 2.0
## hexaco49 0.45 3.8
## hexaco50 0.50 1.5
## hexaco51 0.56 1.7
## hexaco52 0.40 1.8
## hexaco53 0.45 1.2
## hexaco54 0.60 1.5
## hexaco55 0.59 1.7
## hexaco56 0.55 1.5
## hexaco57 0.63 1.9
## hexaco58 0.36 1.6
## hexaco59 0.42 1.3
## hexaco60 0.43 1.5
## 
##                        RC1  RC2  RC3  RC6  RC9  RC5  RC4  RC7  RC8
## SS loadings           4.47 4.46 3.54 3.46 3.18 3.15 3.02 2.87 1.94
## Proportion Var        0.07 0.07 0.06 0.06 0.05 0.05 0.05 0.05 0.03
## Cumulative Var        0.07 0.15 0.21 0.27 0.32 0.37 0.42 0.47 0.50
## Proportion Explained  0.15 0.15 0.12 0.12 0.11 0.10 0.10 0.10 0.06
## Cumulative Proportion 0.15 0.30 0.41 0.53 0.64 0.74 0.84 0.94 1.00
## 
## Mean item complexity =  1.9
## Test of the hypothesis that 9 components are sufficient.
## 
## The root mean square of the residuals (RMSR) is  0.05 
##  with the empirical chi square  2333.16  with prob <  4.1e-66 
## 
## Fit based upon off diagonal values = 0.91
```

---

.pull-left[

```r
fa.diagram(pca_hexa)
```

![](6-Factor-Analysis_files/figure-html/unnamed-chunk-17-1.png)
]

---
class: inverse, center, middle
# Factor Rotation

---
# Factor rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/class-dim-1.png)
]
.pull-right[
Remember that each factor (or component!) adds an additional axis to this plot. Here, the items load highly on one particular component each.

But when there are many items and many components, the items tend to load on multiple components.
]

---
# Factor rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/class-dim-1.png)
]
.pull-right[
Items tend to load largely on the most important (highest eigenvalue) components, and then a little bit on the smaller components.

We can alter how we place the axes, rotating them such that each individual item loads on fewer components!
]
---
# Factor rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/class-dim-1.png)
]
.pull-right[
There are two types of rotation:
- Orthogonal 
    - Factors are forced to be *uncorrelated*

- Oblique
    - Factors are allowed to be *correlated*
]

---
# Orthogonal rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-18-1.png)
]
.pull-right[
In this case, the items are slightly off the original axes.
]

---
# Orthogonal rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-19-1.png)
]
.pull-right[
In this case, the items are slightly off the original axes.

If we rotate the axes slightly clockwise, the items are now back on the axes.

Note that the angles of the axes stay *orthogonal* (i.e. 90 degrees).
]

---
# Orthogonal rotation

The typical method of orthogonal rotation is called *varimax*.

```r
principal(hexaco_only,
          nfactors = 9,
          rotate = "varimax")
```

---
# Oblique rotation

.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-21-1.png)
]
.pull-right[
In this case, the items are displaced from the axes slightly differently.
]

---
# Oblique rotation
.pull-left[
![](6-Factor-Analysis_files/figure-html/unnamed-chunk-22-1.png)
]
.pull-right[
In this case, the items are displaced from the axes slightly differently.

Here, we allow the axes to be non-orthogonal (i.e. oblique - not 90 degrees), which means the axes correlate with each other.
]
---
# Oblique rotation
The typical method of oblique rotation is called *oblimin*.

```r
principal(hexaco_only,
          nfactors = 9,
          rotate = "oblimin")
```

```
## Principal Components Analysis
## Call: principal(r = hexaco_only, nfactors = 9, rotate = "oblimin")
## Standardized loadings (pattern matrix) based upon correlation matrix
##            TC1   TC2   TC3   TC6   TC5   TC4   TC7   TC9   TC8   h2
## hexaco1  -0.18 -0.03  0.74  0.07  0.10 -0.05  0.02  0.15  0.03 0.59
## hexaco2  -0.01  0.16  0.10  0.12 -0.55 -0.03 -0.04  0.21  0.20 0.48
## hexaco3   0.57 -0.08 -0.10  0.17  0.07 -0.12 -0.01 -0.02  0.15 0.44
## hexaco4   0.08 -0.13  0.06  0.17 -0.21 -0.58  0.03  0.20  0.08 0.56
## hexaco5   0.09  0.50 -0.02 -0.15 -0.05 -0.08 -0.09  0.01  0.02 0.30
## hexaco6   0.10 -0.05  0.10  0.59  0.03  0.11  0.01  0.06 -0.28 0.49
## hexaco7   0.01 -0.04 -0.52  0.02 -0.24  0.11  0.14 -0.11 -0.06 0.39
## hexaco8  -0.04  0.03 -0.01  0.10 -0.17 -0.19  0.16  0.59  0.05 0.53
## hexaco9  -0.56 -0.09 -0.12 -0.04 -0.03  0.05  0.00  0.06  0.18 0.42
## hexaco10 -0.05 -0.23  0.05 -0.67  0.15  0.09  0.00  0.13  0.01 0.60
## hexaco11  0.13  0.10  0.16 -0.12  0.16 -0.06 -0.69  0.18  0.00 0.60
## hexaco12 -0.02  0.47  0.02  0.03 -0.04  0.38 -0.05  0.27  0.09 0.50
## hexaco13 -0.01  0.07 -0.73  0.01  0.24  0.03 -0.09  0.13 -0.10 0.63
## hexaco14 -0.11 -0.05  0.13  0.01  0.34 -0.10  0.01 -0.45  0.18 0.43
## hexaco15 -0.66 -0.03 -0.03  0.05  0.14  0.07 -0.04  0.26  0.03 0.52
## hexaco16  0.07  0.21  0.24 -0.03  0.20 -0.16  0.62  0.07 -0.14 0.58
## hexaco17  0.14  0.66  0.03 -0.06  0.06  0.11  0.15  0.17  0.01 0.54
## hexaco18  0.21 -0.04 -0.01  0.45  0.12  0.17  0.08  0.16  0.06 0.33
## hexaco19 -0.14 -0.01  0.25  0.23  0.14 -0.25 -0.26 -0.07  0.44 0.45
## hexaco20  0.07 -0.09  0.16 -0.12  0.67  0.04  0.02 -0.08  0.10 0.57
## hexaco21 -0.69  0.01  0.15  0.03  0.16  0.02  0.08  0.00  0.17 0.58
## hexaco22  0.15  0.07  0.10  0.07 -0.03 -0.66  0.13  0.19  0.02 0.64
## hexaco23  0.15  0.55  0.05  0.20  0.13  0.02  0.12  0.09  0.04 0.45
## hexaco24  0.03 -0.06 -0.02 -0.12  0.05  0.01  0.02  0.05  0.76 0.62
## hexaco25  0.05 -0.09 -0.61  0.15 -0.06  0.11  0.11  0.12  0.15 0.51
## hexaco26 -0.14  0.02 -0.23  0.10  0.60  0.11  0.02 -0.27  0.02 0.53
## hexaco27  0.62  0.01 -0.06  0.26  0.12  0.00  0.13 -0.06  0.22 0.54
## hexaco28  0.03  0.07  0.01  0.08  0.06  0.66 -0.13  0.04  0.11 0.49
## hexaco29  0.15  0.49  0.00  0.01 -0.10  0.10 -0.18  0.10  0.16 0.39
## hexaco30  0.01  0.19 -0.04 -0.50  0.08 -0.12 -0.02 -0.05  0.33 0.43
## hexaco31  0.01  0.02  0.54 -0.04  0.35 -0.07 -0.10  0.19 -0.17 0.48
## hexaco32  0.03 -0.10  0.16 -0.14  0.28  0.32 -0.11 -0.33 -0.01 0.46
## hexaco33  0.62  0.10  0.02  0.10  0.01  0.08  0.03  0.03  0.06 0.43
## hexaco34  0.02  0.01  0.01 -0.05  0.12 -0.13  0.74  0.03  0.09 0.63
## hexaco35  0.12 -0.55 -0.05 -0.02  0.08 -0.37  0.05 -0.27  0.07 0.61
## hexaco36 -0.01  0.04 -0.10  0.68 -0.03 -0.03  0.09  0.10  0.02 0.53
## hexaco37 -0.18  0.05 -0.48  0.08  0.33 -0.22 -0.05  0.37  0.01 0.55
## hexaco38  0.09  0.03 -0.12 -0.01 -0.14 -0.07  0.04  0.64 -0.02 0.51
## hexaco39  0.60  0.01  0.07 -0.05 -0.07 -0.07  0.09  0.04  0.02 0.39
## hexaco40  0.19  0.12  0.17 -0.06  0.17 -0.21  0.52  0.13 -0.04 0.49
## hexaco41  0.05 -0.72  0.03 -0.02 -0.01 -0.05  0.01  0.14 -0.05 0.51
## hexaco42 -0.10  0.13  0.14 -0.41  0.01  0.01 -0.09  0.10  0.19 0.30
## hexaco43  0.01 -0.23 -0.17 -0.04 -0.04  0.29  0.33  0.36 -0.18 0.41
## hexaco44 -0.10  0.05  0.04 -0.06  0.67  0.19  0.08 -0.16  0.08 0.58
## hexaco45  0.69 -0.11 -0.05 -0.01  0.00  0.03 -0.11  0.09  0.01 0.49
## hexaco46 -0.21 -0.15  0.10  0.10 -0.08  0.49 -0.34 -0.06  0.00 0.53
## hexaco47  0.01  0.57  0.10 -0.01 -0.02  0.02  0.07  0.08 -0.08 0.36
## hexaco48  0.00 -0.02 -0.02 -0.18 -0.15  0.17  0.17  0.00  0.70 0.61
## hexaco49 -0.03  0.00  0.59  0.05 -0.28  0.31  0.10 -0.22 -0.06 0.55
## hexaco50 -0.18  0.08  0.04  0.00 -0.16  0.01 -0.07  0.64  0.09 0.50
## hexaco51  0.58  0.00  0.00  0.12  0.20  0.07 -0.17  0.14  0.01 0.44
## hexaco52  0.00  0.22 -0.07 -0.01  0.18  0.67 -0.09  0.06  0.11 0.60
## hexaco53  0.08 -0.73  0.04  0.01  0.07  0.01  0.12  0.13  0.04 0.55
## hexaco54  0.13 -0.18  0.11  0.56  0.13  0.10 -0.10  0.12 -0.06 0.40
## hexaco55  0.09  0.07  0.49  0.00  0.09 -0.20 -0.25 -0.03  0.11 0.41
## hexaco56  0.11 -0.12 -0.11 -0.14  0.58  0.07  0.13  0.06 -0.15 0.45
## hexaco57 -0.49 -0.09  0.02  0.04  0.06  0.21  0.19  0.09  0.11 0.37
## hexaco58 -0.16 -0.12 -0.05  0.08 -0.01 -0.05  0.72  0.05  0.16 0.64
## hexaco59  0.03 -0.72  0.09 -0.14  0.06  0.13 -0.02  0.05  0.12 0.58
## hexaco60 -0.02 -0.15 -0.01 -0.69  0.10  0.11  0.01  0.10  0.07 0.57
##            u2 com
## hexaco1  0.41 1.3
## hexaco2  0.52 2.0
## hexaco3  0.56 1.6
## hexaco4  0.44 2.0
## hexaco5  0.70 1.4
## hexaco6  0.51 1.7
## hexaco7  0.61 1.9
## hexaco8  0.47 1.7
## hexaco9  0.58 1.4
## hexaco10 0.40 1.5
## hexaco11 0.40 1.6
## hexaco12 0.50 2.7
## hexaco13 0.37 1.4
## hexaco14 0.57 2.7
## hexaco15 0.48 1.5
## hexaco16 0.42 2.2
## hexaco17 0.46 1.4
## hexaco18 0.67 2.4
## hexaco19 0.55 4.4
## hexaco20 0.43 1.3
## hexaco21 0.42 1.4
## hexaco22 0.36 1.5
## hexaco23 0.55 1.8
## hexaco24 0.38 1.1
## hexaco25 0.49 1.6
## hexaco26 0.47 2.0
## hexaco27 0.46 1.9
## hexaco28 0.51 1.2
## hexaco29 0.61 2.0
## hexaco30 0.57 2.4
## hexaco31 0.52 2.4
## hexaco32 0.54 4.3
## hexaco33 0.57 1.2
## hexaco34 0.37 1.2
## hexaco35 0.39 2.5
## hexaco36 0.47 1.1
## hexaco37 0.45 3.7
## hexaco38 0.49 1.3
## hexaco39 0.61 1.2
## hexaco40 0.51 2.5
## hexaco41 0.49 1.1
## hexaco42 0.70 2.4
## hexaco43 0.59 4.8
## hexaco44 0.42 1.4
## hexaco45 0.51 1.2
## hexaco46 0.47 2.8
## hexaco47 0.64 1.2
## hexaco48 0.39 1.5
## hexaco49 0.45 2.4
## hexaco50 0.50 1.4
## hexaco51 0.56 1.7
## hexaco52 0.40 1.5
## hexaco53 0.45 1.2
## hexaco54 0.60 1.8
## hexaco55 0.59 2.2
## hexaco56 0.55 1.7
## hexaco57 0.63 2.1
## hexaco58 0.36 1.3
## hexaco59 0.42 1.3
## hexaco60 0.43 1.3
## 
##                        TC1  TC2  TC3  TC6  TC5  TC4  TC7  TC9  TC8
## SS loadings           4.44 4.31 3.46 3.39 3.18 3.28 3.13 2.81 2.10
## Proportion Var        0.07 0.07 0.06 0.06 0.05 0.05 0.05 0.05 0.03
## Cumulative Var        0.07 0.15 0.20 0.26 0.31 0.37 0.42 0.47 0.50
## Proportion Explained  0.15 0.14 0.12 0.11 0.11 0.11 0.10 0.09 0.07
## Cumulative Proportion 0.15 0.29 0.41 0.52 0.62 0.73 0.84 0.93 1.00
## 
##  With component correlations of 
##       TC1   TC2   TC3   TC6   TC5   TC4   TC7   TC9   TC8
## TC1  1.00  0.07 -0.01  0.16 -0.01 -0.17  0.03  0.04 -0.11
## TC2  0.07  1.00  0.06  0.10 -0.05  0.05 -0.08  0.16  0.01
## TC3 -0.01  0.06  1.00 -0.06  0.02 -0.07 -0.08 -0.10  0.02
## TC6  0.16  0.10 -0.06  1.00 -0.12 -0.08  0.01  0.10 -0.12
## TC5 -0.01 -0.05  0.02 -0.12  1.00  0.04 -0.01 -0.07  0.04
## TC4 -0.17  0.05 -0.07 -0.08  0.04  1.00 -0.17 -0.09  0.01
## TC7  0.03 -0.08 -0.08  0.01 -0.01 -0.17  1.00  0.09  0.02
## TC9  0.04  0.16 -0.10  0.10 -0.07 -0.09  0.09  1.00  0.00
## TC8 -0.11  0.01  0.02 -0.12  0.04  0.01  0.02  0.00  1.00
## 
## Mean item complexity =  1.9
## Test of the hypothesis that 9 components are sufficient.
## 
## The root mean square of the residuals (RMSR) is  0.05 
##  with the empirical chi square  2333.16  with prob <  4.1e-66 
## 
## Fit based upon off diagonal values = 0.91
```

---
# Which rotation to use?

For the most part, use *orthogonal* rotation (i.e. Varimax).

*Oblique* rotation is defensible when there are *a priori*, *theoretical* reasons to believe there will be correlations between dimensions.

.pull-left[

```r
principal(hexaco_only,
          nfactors = 9,
          rotate = "varimax")
```
]
.pull-right[

```r
principal(hexaco_only,
          nfactors = 9,
          rotate = "oblimin")
```
]

---
class: inverse, middle, center
# Factor interpretation

---
# Final PCA

Let's finish off by looking closely at the PCA solution with nine factors and *varimax* rotation.

```r
pca_hexa
```

---

We can hide all the smaller factor loadings (using the *cut* parameter), and sort the factor loadings in order of size.

```r
print(pca_hexa,
      cut = 0.3,
      sort = TRUE)
```

```
## Principal Components Analysis
## Call: principal(r = hexaco_only, nfactors = 9, rotate = "varimax")
## Standardized loadings (pattern matrix) based upon correlation matrix
##          item   RC1   RC2   RC3   RC6   RC9   RC5   RC4   RC7   RC8
## hexaco21   21 -0.70                                                
## hexaco45   45  0.68                                                
## hexaco15   15 -0.65                                                
## hexaco27   27  0.62                                                
## hexaco33   33  0.62                                                
## hexaco39   39  0.59                                                
## hexaco3     3  0.59                                                
## hexaco51   51  0.58                                                
## hexaco9     9 -0.58                                                
## hexaco57   57 -0.52                                                
## hexaco59   59       -0.71                                          
## hexaco53   53       -0.71                                          
## hexaco41   41       -0.70                                          
## hexaco17   17        0.67                                          
## hexaco35   35       -0.61             -0.36                        
## hexaco23   23        0.58                                          
## hexaco47   47        0.57                                          
## hexaco29   29        0.53                                          
## hexaco12   12        0.53              0.37                        
## hexaco5     5        0.50                                          
## hexaco1     1              0.74                                    
## hexaco13   13             -0.65              0.36                  
## hexaco25   25             -0.64                                    
## hexaco31   31              0.62                                    
## hexaco7     7             -0.59                                    
## hexaco55   55              0.56                                    
## hexaco49   49              0.46        0.32 -0.36                  
## hexaco60   60                   -0.69                              
## hexaco36   36                    0.67                              
## hexaco10   10                   -0.66                              
## hexaco6     6                    0.64                              
## hexaco54   54                    0.57                              
## hexaco30   30                   -0.55                          0.30
## hexaco18   18                    0.46                              
## hexaco42   42                   -0.44                              
## hexaco22   22                         -0.67                        
## hexaco52   52                          0.66                        
## hexaco28   28                          0.65                        
## hexaco4     4                         -0.59                        
## hexaco46   46                          0.52       -0.38            
## hexaco44   44                                0.65                  
## hexaco20   20                                0.64                  
## hexaco26   26                                0.63                  
## hexaco56   56                                0.61                  
## hexaco2     2                               -0.58                  
## hexaco34   34                                      0.75            
## hexaco58   58                                      0.71            
## hexaco16   16                                      0.65            
## hexaco11   11              0.32                   -0.64            
## hexaco40   40                                      0.56            
## hexaco38   38                                            0.66      
## hexaco50   50                                            0.63      
## hexaco8     8                                            0.61      
## hexaco14   14                                0.30       -0.48      
## hexaco37   37             -0.36              0.39        0.42      
## hexaco32   32                          0.36             -0.40      
## hexaco43   43                                            0.37      
## hexaco24   24                                                  0.74
## hexaco48   48                                                  0.65
## hexaco19   19              0.33                                0.47
##            h2   u2 com
## hexaco21 0.58 0.42 1.4
## hexaco45 0.49 0.51 1.1
## hexaco15 0.52 0.48 1.5
## hexaco27 0.54 0.46 1.9
## hexaco33 0.43 0.57 1.3
## hexaco39 0.39 0.61 1.2
## hexaco3  0.44 0.56 1.6
## hexaco51 0.44 0.56 1.7
## hexaco9  0.42 0.58 1.6
## hexaco57 0.37 0.63 1.9
## hexaco59 0.58 0.42 1.3
## hexaco53 0.55 0.45 1.2
## hexaco41 0.51 0.49 1.1
## hexaco17 0.54 0.46 1.4
## hexaco35 0.61 0.39 2.3
## hexaco23 0.45 0.55 1.8
## hexaco47 0.36 0.64 1.2
## hexaco29 0.39 0.61 1.8
## hexaco12 0.50 0.50 2.5
## hexaco5  0.30 0.70 1.4
## hexaco1  0.59 0.41 1.2
## hexaco13 0.63 0.37 2.0
## hexaco25 0.51 0.49 1.5
## hexaco31 0.48 0.52 1.5
## hexaco7  0.39 0.61 1.2
## hexaco55 0.41 0.59 1.7
## hexaco49 0.55 0.45 3.8
## hexaco60 0.57 0.43 1.5
## hexaco36 0.53 0.47 1.4
## hexaco10 0.60 0.40 1.8
## hexaco6  0.49 0.51 1.5
## hexaco54 0.40 0.60 1.5
## hexaco30 0.43 0.57 1.9
## hexaco18 0.33 0.67 2.2
## hexaco42 0.30 0.70 2.2
## hexaco22 0.64 0.36 2.0
## hexaco52 0.60 0.40 1.8
## hexaco28 0.49 0.51 1.3
## hexaco4  0.56 0.44 2.4
## hexaco46 0.53 0.47 2.7
## hexaco44 0.58 0.42 1.8
## hexaco20 0.57 0.43 1.9
## hexaco26 0.53 0.47 1.8
## hexaco56 0.45 0.55 1.5
## hexaco2  0.48 0.52 1.9
## hexaco34 0.63 0.37 1.2
## hexaco58 0.64 0.36 1.6
## hexaco16 0.58 0.42 1.9
## hexaco11 0.60 0.40 2.1
## hexaco40 0.49 0.51 2.2
## hexaco38 0.51 0.49 1.4
## hexaco50 0.50 0.50 1.5
## hexaco8  0.53 0.47 2.0
## hexaco14 0.43 0.57 2.9
## hexaco37 0.55 0.45 4.0
## hexaco32 0.46 0.54 4.2
## hexaco43 0.41 0.59 4.9
## hexaco24 0.62 0.38 1.3
## hexaco48 0.61 0.39 2.0
## hexaco19 0.45 0.55 3.1
## 
##                        RC1  RC2  RC3  RC6  RC9  RC5  RC4  RC7  RC8
## SS loadings           4.47 4.46 3.54 3.46 3.18 3.15 3.02 2.87 1.94
## Proportion Var        0.07 0.07 0.06 0.06 0.05 0.05 0.05 0.05 0.03
## Cumulative Var        0.07 0.15 0.21 0.27 0.32 0.37 0.42 0.47 0.50
## Proportion Explained  0.15 0.15 0.12 0.12 0.11 0.10 0.10 0.10 0.06
## Cumulative Proportion 0.15 0.30 0.41 0.53 0.64 0.74 0.84 0.94 1.00
## 
## Mean item complexity =  1.9
## Test of the hypothesis that 9 components are sufficient.
## 
## The root mean square of the residuals (RMSR) is  0.05 
##  with the empirical chi square  2333.16  with prob <  4.1e-66 
## 
## Fit based upon off diagonal values = 0.91
```

---
# Final PCA

Down at the bottom of our output are statistics about the amount of variance our factors explain.

```
##                             RC1        RC2        RC3        RC6
## SS loadings           4.4744097 4.45609263 3.54088899 3.46245860
## Proportion Var        0.0745735 0.07426821 0.05901482 0.05770764
## Cumulative Var        0.0745735 0.14884171 0.20785652 0.26556417
## Proportion Explained  0.1486879 0.14807922 0.11766634 0.11506004
## Cumulative Proportion 0.1486879 0.29676714 0.41443348 0.52949352
##                              RC9        RC5        RC4        RC7
## SS loadings           3.18062363 3.14794568 3.01616898 2.87416485
## Proportion Var        0.05301039 0.05246576 0.05026948 0.04790275
## Cumulative Var        0.31857456 0.37104032 0.42130980 0.46921255
## Proportion Explained  0.10569446 0.10460854 0.10022951 0.09551061
## Cumulative Proportion 0.63518797 0.73979651 0.84002602 0.93553663
##                              RC8
## SS loadings           1.93987219
## Proportion Var        0.03233120
## Cumulative Var        0.50154376
## Proportion Explained  0.06446337
## Cumulative Proportion 1.00000000
```

---
# Interpreting the output

It looks like there are 10 items that load on our first factor.

The top three are the following items from the HEXACO-60:

Item 21: People think of me as someone who has a quick temper.

Item 45: Most people tend to get angry more quickly than I do.

Item 15: People sometimes tell me that I'm too stubborn.

---
# Interpreting the output

In fact, the ten items are all those that correspond to *Agreeableness*:

![](images/hexaco1-agreeable.png)

Note that several should be reversed, and they have *negative* factor loadings because we didn't actually reverse them!

---
# How do individual *participants* score?

Once we know what our *factors* are, how do we convert each participant's data into something that tells us how that participant rated for each factor?

```r
head(pca_hexa$scores)
```

```
##             RC1         RC2        RC3        RC6        RC9
## [1,] -2.0370952 -0.74766949  1.0906981  2.4308437 -0.7581765
## [2,] -1.3067772  0.07377406 -0.9048852  0.6698510 -1.3186814
## [3,] -0.4223559 -0.43381267 -1.0134896 -0.6835562  1.2656719
## [4,] -1.4688597 -2.01939403 -1.6466062  2.3991087  0.1195594
## [5,]  0.8065396  0.42209968 -0.6753594  0.9384934 -1.7880986
## [6,]  0.4092487 -1.41218241 -1.5163114 -2.2545879  1.9586546
##             RC5         RC4        RC7        RC8
## [1,] -0.5199759 -0.64786941  1.4417465 -0.2530221
## [2,] -0.4670667 -0.49776904 -0.2658352 -0.8120384
## [3,] -1.1032942  0.19409792  0.6168094  0.2996072
## [4,] -1.3225573 -1.10714105 -0.1981588 -0.4550324
## [5,] -0.7686581  1.06179528 -1.2344637  0.4208103
## [6,] -1.7329438  0.03392348 -1.0618261 -1.5631121
```

---
# A quick example
.pull-left[
The factor scores can be treated as if they were any other variable!
Here I combine the Factor Scores with the original data.

```r
final_data <- cbind(crime,
                    pca_hexa$scores)
ggplot(final_data,
       aes(x = RC1,
           fill = factor(sex, 
                         levels = c(1, 2),
                         labels = c("Female", "Male")))) +
  geom_density(alpha = 0.5) + 
  scale_fill_brewer(palette = "Dark2") + 
  labs(x = "Agreeableness",
       fill = "") +
  theme_classic() +
  theme(text = element_text(size = 20))
```
]
.pull-right[
![](6-Factor-Analysis_files/figure-html/final-plot-1.png)
]

---
class: center, middle, inverse
# Why would you do this?

---
# Why use factor analysis?

1) Rather than trying to analyse many, many different items as if they are each independent from each other, you can reduce the task down to a smaller set of factors

2) Factor analysis helps you *condense* the information down, while still retaining the benefit of having many different, independent measurements of the underlying constructs.

3) During the *design* of questionnaires, it helps you work out which items are measuring which thing, and which items are worth keeping!

---
# This week's background

Background reading for this week can be found in Field et al, Discovering Statistics Using R (2011), Chapter 17 - Exploratory Factor Analysis.

There is a Datacamp course, Factor Analysis in R. Note: it's a little tough in places - don't be discouraged! It's good practice and covers some topics we didn't cover today!

---
background-image: url('images/tea-studying-hard.jpeg')
background-size: contain