12 Modelling

Pre-requisite for this class: ch.22-24 (“Model”) from R for Data Science - http://r4ds.had.co.nz/model-intro.html

Let us explore how we can apply linear models with the Understanding Society data for your reports.

First we need to read in the data. This is the same data frame that I used for Data visualisation 2. I saved it then so now I can simply open the file.

library(tidyverse)
UndSoc <- readRDS("myData/all7clean.rds")

We have a variable for interest in politics (vote6) and we will take it as the outcome variable we want to study. It is a discrete variable measured on a four-point scale (from “not at all interested” to “very interested”). It is an ordinal rather than continuous variable so strictly speaking it is not statisticaly appropriate to calculate the mean of this variable and to use simple linear regression. You will see though that we can still do this and learn something interesting from this exercise.

First let us look at the distribution of the variable.

table(UndSoc$vote6)
## 
##     fairly not al all   not very       very 
##     102212      84351      86790      30981

To do the modelling we want to recode it to numeric. I will do this in such a way that larger values indicate stronger interest in politics.

UndSoc <- UndSoc %>%
  mutate(polinterest = recode(vote6, 
                              "very" = "4", "fairly" = "3",
                              "not very" = "2", "not al all" = "1")) %>%
  mutate(polinterest = as.numeric(polinterest))

table(UndSoc$polinterest)
## 
##      1      2      3      4 
##  84351  86790 102212  30981

12.1 Cross-sectional analysis

Let us start from cross-sectional analysis, i.e. the analysis of the data at one point in time. You can choose any wave to do this; I will go with wave 1.

First I create a separate data frame for wave 1 only.

wave1 <- UndSoc %>%
  filter(wave == "a") %>%
  filter(!is.na(dvage))

12.1.1 Age and political interest

We will start with the association between age and political interest. It is always a good idea to start with visualisations.

wave1 %>%
  ggplot(aes(x = dvage, y = polinterest)) +
  geom_smooth()

I fit a non-parametric smooth here with geom_smooth, and the association between age and political interest seems to be non-linear.

We can also fit a regression line and see how well it describes the data.

wave1 %>%
  ggplot(aes(x = dvage, y = polinterest)) +
  geom_smooth() +
  geom_smooth(method = "lm", colour = "red")

Up to the age of about 70 years two lines are pretty close and I’d say that the linear function adequately describes the data. However, for people older than 75 the linear assocition is inadequate and provides a really poor fit.

12.1.2 Sex and political interest

Let us look at a bar chart showing the association between sex and political interest.

wave1 %>%
  ggplot(aes(x = sex, y = polinterest)) +
  geom_bar(stat = "summary", fun.y = "mean")

We see that mean political interest is somewhat higher for men compared to women.

12.1.3 Linear model

As I said above, the outcome variable is ordinal so fitting a linear model will have some limitations. For our purposes this is fine though.

For your reports you should fit a linear model for ordered and binary outcomes. The linear model for binary outcomes is called the linear probability model. Please do not estimate logit or probit models for the purpose of this assignment. Of course, you cannot fit a linear model for nominal outcomes. If you outcome is nominal (this is unlikely), you should either stay with descriptive statistics and graphs or use a multinomial logit model. You should only do this if you are confident that you know what you are doing.

Now we can fit a simple linear regression model with two predictors: age and sex.

m1 <- lm(polinterest ~ sex + dvage, wave1)
summary(m1)
## 
## Call:
## lm(formula = polinterest ~ sex + dvage, data = wave1)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.8578 -0.9275 -0.0703  0.7293  2.1042 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 1.7688374  0.0123862  142.81   <2e-16 ***
## sexmale     0.3115085  0.0087412   35.64   <2e-16 ***
## dvage       0.0079332  0.0002393   33.16   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.946 on 47527 degrees of freedom
##   (3464 observations deleted due to missingness)
## Multiple R-squared:  0.04805,    Adjusted R-squared:  0.04801 
## F-statistic:  1199 on 2 and 47527 DF,  p-value: < 2.2e-16

Both age and sex are highly statistically significant predictors of political interest. Men on average are 0.3 points higher on the political interest scale. For age, a one-year difference is associated with about 0.01 change in political interest (older people are more interested). For two people with the age difference of about 30 years this corresponds to the difference in political interest of about 0.3.

We have seen that the association between age and political interest is non-linear. To model this, we may want to include the quadratic term for age.

m2 <- lm(polinterest ~ sex + dvage + I(dvage^2), wave1)
summary(m2)
## 
## Call:
## lm(formula = polinterest ~ sex + dvage + I(dvage^2), data = wave1)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.61209 -0.83774 -0.09245  0.72122  2.23983 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  1.390e+00  2.781e-02   49.97   <2e-16 ***
## sexmale      3.123e-01  8.720e-03   35.82   <2e-16 ***
## dvage        2.616e-02  1.221e-03   21.42   <2e-16 ***
## I(dvage^2)  -1.880e-04  1.236e-05  -15.22   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.9437 on 47526 degrees of freedom
##   (3464 observations deleted due to missingness)
## Multiple R-squared:  0.05266,    Adjusted R-squared:  0.0526 
## F-statistic: 880.7 on 3 and 47526 DF,  p-value: < 2.2e-16

The quadratic term is statistically significanty indicating a non-linear fit. Note, however, that the coefficients are now more difficult to interpret. We can visualise the association between age and political interest to get some idea of the effect size.

wave1 %>%
  ggplot(aes(x = dvage, y = polinterest)) +
  geom_smooth(method = "lm", formula = y ~ x + I(x^2))

Note that this visualisation does not control for sex.

12.1.4 Interaction effect between sex and age

Another question we can ask is whether the association between age and political interest is the same or different for men and women. Note that in the previous model we control for sex, and the coefficient for age represents an averaged association between age and political interest for men and women. It is possible to have a situation where the association is very different for two sexes (for example, for men political interest increases with age and for women it decreases). To check this formally, we can fit a model with an interaction effect.

m3 <- lm(polinterest ~ sex * dvage, wave1)
summary(m3)
## 
## Call:
## lm(formula = polinterest ~ sex * dvage, data = wave1)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.82832 -0.91573 -0.06676  0.71663  2.11783 
## 
## Coefficients:
##                 Estimate Std. Error t value Pr(>|t|)    
## (Intercept)    1.7479264  0.0157990 110.636   <2e-16 ***
## sexmale        0.3587056  0.0238008  15.071   <2e-16 ***
## dvage          0.0083904  0.0003213  26.115   <2e-16 ***
## sexmale:dvage -0.0010263  0.0004814  -2.132    0.033 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.946 on 47526 degrees of freedom
##   (3464 observations deleted due to missingness)
## Multiple R-squared:  0.04814,    Adjusted R-squared:  0.04808 
## F-statistic: 801.2 on 3 and 47526 DF,  p-value: < 2.2e-16

We see that the interaction effect is indeed statistically significant and negative suggesting that for men the association between age and political interest is weaker.

We can also fit a quadratic model.

m4 <- lm(polinterest ~ sex * dvage + sex * I(dvage^2), wave1)
summary(m4)
## 
## Call:
## lm(formula = polinterest ~ sex * dvage + sex * I(dvage^2), data = wave1)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.60079 -0.84767 -0.07443  0.71480  2.22023 
## 
## Coefficients:
##                      Estimate Std. Error t value Pr(>|t|)    
## (Intercept)         1.463e+00  3.689e-02  39.674  < 2e-16 ***
## sexmale             1.409e-01  5.545e-02   2.541   0.0111 *  
## dvage               2.201e-02  1.629e-03  13.517  < 2e-16 ***
## I(dvage^2)         -1.400e-04  1.641e-05  -8.532  < 2e-16 ***
## sexmale:dvage       9.684e-03  2.462e-03   3.933 8.41e-05 ***
## sexmale:I(dvage^2) -1.120e-04  2.493e-05  -4.494 7.02e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.9435 on 47524 degrees of freedom
##   (3464 observations deleted due to missingness)
## Multiple R-squared:  0.05318,    Adjusted R-squared:  0.05308 
## F-statistic: 533.9 on 5 and 47524 DF,  p-value: < 2.2e-16

The interaction effect is highly statistically significant. To make sense of the association between age and political interest for men and women it is best to visualise this model.

wave1 %>%
  ggplot(aes(x = dvage, y = polinterest, colour = sex)) +
  geom_smooth(method = "lm", formula = y ~ x + I(x^2))

We can see that the lines for men and women are indeed different, although the non-linear pattern of the association is similar for both sexes.

12.1.5 Checking the assumptions

In Data Analysis 2 your task was to check the assumptions underlying linear regression models. This is an important step, but it is less relevant for our work in this module. Many of these assumptions are about the distribution of residuals and affect standard errors of regression coefficients. This is very important for small samples, but less important for large samples like the one we have in the Understanding Society where standard errors are usually small compared to regression coefficients and moderate violation of the regression assumpions (such as not normal distribution of the residuals and heteroskedasticity) will not affect the results much. You do not have to show how you have checked the assumptions in your reports (although if you want to do it for yourself this would not hurt).

Another common question is the interpretation of the R-squared coefficient. In Model 4 the R-squared coefficients was about 0.05 suggesting that age and sex can jointly account for about 5% of the variance of political interest. Is this an indication that the model is really poor? Yes, if your task is to build a model that predicts political interest well. This shows that just knowning a person’s age and sex you can only make a wild guess about their level of political interest (this isn’t really surprising).

However, if our task is simply to explore the association between age, sex and political interest rather than to build a model that predicts political interest well, then we should not pay too much attention to R-squared.

12.1.6 Presenting the output for multiple regression models

How to present the regression output in your reports? You can of course simply fo summary(m1) as we did above, but this does not look really nice. Another way is to use the package stargazer. Note that stargazer will only work if you knit your reports as pdf rather than Word. To knit as pdf you will need to install LaTeX on your computers (see https://www.latex-project.org). This is how the table produced by stargazer will look like.

library(stargazer)
stargazer(m1, m2, m3, m4, type = "html")
Dependent variable:
polinterest
(1) (2) (3) (4)
sexmale 0.312*** 0.312*** 0.359*** 0.141**
(0.009) (0.009) (0.024) (0.055)
dvage 0.008*** 0.026*** 0.008*** 0.022***
(0.0002) (0.001) (0.0003) (0.002)
I(dvage2) -0.0002*** -0.0001***
(0.00001) (0.00002)
sexmale:dvage -0.001** 0.010***
(0.0005) (0.002)
sexmale:I(dvage2) -0.0001***
(0.00002)
Constant 1.769*** 1.390*** 1.748*** 1.463***
(0.012) (0.028) (0.016) (0.037)
Observations 47,530 47,530 47,530 47,530
R2 0.048 0.053 0.048 0.053
Adjusted R2 0.048 0.053 0.048 0.053
Residual Std. Error 0.946 (df = 47527) 0.944 (df = 47526) 0.946 (df = 47526) 0.943 (df = 47524)
F Statistic 1,199.426*** (df = 2; 47527) 880.675*** (df = 3; 47526) 801.192*** (df = 3; 47526) 533.896*** (df = 5; 47524)
Note: p<0.1; p<0.05; p<0.01

In your reports you will knit as LaTeX rather than html so you should do something like:

```{r results = 'asis'}
stargazer(m1, m2, m3, m4, type = "latex")
```

The results argument shoud be set to asis so that the results are displayed correctly. Note that type is latex.

stargazer has many options to customise the tables. Please do experiment with them.

12.2 Longitudinal analysis

Now we may want to model how things change over time (i.e. do longitudinal modelling).

12.2.1 Simple model

We will start from simply plotting mean political interest over time.

# First let us code the variable for year.
UndSoc <- UndSoc %>%
  mutate(year = dplyr::recode(wave, "a" = "2009",
                       "b" = "2010",
                       "c" = "2011",
                       "d" = "2012",
                       "e" = "2013",
                       "f" = "2014",
                       "g" = "2015")) %>%
  mutate(year = as.numeric(year))

UndSoc %>%
    group_by(year) %>%
    summarise(
      meanPI = mean(polinterest, na.rm = TRUE)
    ) %>%
    ggplot(aes(x = year, y = meanPI)) +
    geom_point() +
    geom_line()

The model that describes this chart will be the following.

m5 <- lm(polinterest ~ as.factor(year), UndSoc)
summary(m5)
## 
## Call:
## lm(formula = polinterest ~ as.factor(year), data = UndSoc)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -1.3387 -1.2137 -0.2253  0.7295  1.7863 
## 
## Coefficients:
##                       Estimate Std. Error t value Pr(>|t|)    
## (Intercept)          2.2705028  0.0044708 507.848  < 2e-16 ***
## as.factor(year)2010  0.0009813  0.0062259   0.158    0.875    
## as.factor(year)2011 -0.0451822  0.0063862  -7.075 1.50e-12 ***
## as.factor(year)2012 -0.0567904  0.0064826  -8.760  < 2e-16 ***
## as.factor(year)2013 -0.0262915  0.0065703  -4.002 6.29e-05 ***
## as.factor(year)2014  0.0099673  0.0067440   1.478    0.139    
## as.factor(year)2015  0.0681780  0.0066526  10.248  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.9747 on 304327 degrees of freedom
##   (280369 observations deleted due to missingness)
## Multiple R-squared:  0.001471,   Adjusted R-squared:  0.001451 
## F-statistic:  74.7 on 6 and 304327 DF,  p-value: < 2.2e-16

We can see from this model that in 2011 to 2013 political interest was statistically significantly lower than in 2009, and in 2015 it was statistically significantly higher.

12.2.2 Adding a time-constant variable

We may want to check if the change in political interest depends on a time-constant variable such as sex. Was the change in mean political interest similar for men and women?

UndSoc %>%
    filter(!is.na(sex)) %>%
    group_by(year, sex) %>%
    summarise(
      meanPI = mean(polinterest, na.rm = TRUE)
    ) %>%
    ggplot(aes(x = year, y = meanPI, colour = sex)) +
    geom_point() +
    geom_line()

It does not look like there was an interaction between sex and year, but we can check it formally.

m6 <- lm(polinterest ~ as.factor(year)*sex, UndSoc)
summary(m6)
## 
## Call:
## lm(formula = polinterest ~ as.factor(year) * sex, data = UndSoc)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.51330 -1.06358 -0.08411  0.80109  1.93642 
## 
## Coefficients:
##                               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                  2.1316987  0.0058923 361.777  < 2e-16 ***
## as.factor(year)2010          0.0034852  0.0082076   0.425 0.671108    
## as.factor(year)2011         -0.0542942  0.0084160  -6.451 1.11e-10 ***
## as.factor(year)2012         -0.0681161  0.0085534  -7.964 1.68e-15 ***
## as.factor(year)2013         -0.0475894  0.0086757  -5.485 4.13e-08 ***
## as.factor(year)2014         -0.0020655  0.0089103  -0.232 0.816686    
## as.factor(year)2015          0.0672156  0.0087858   7.651 2.01e-14 ***
## sexmale                      0.3150753  0.0088775  35.491  < 2e-16 ***
## as.factor(year)2010:sexmale -0.0061332  0.0123615  -0.496 0.619786    
## as.factor(year)2011:sexmale  0.0208112  0.0126810   1.641 0.100772    
## as.factor(year)2012:sexmale  0.0238450  0.0128683   1.853 0.063883 .  
## as.factor(year)2013:sexmale  0.0451257  0.0130399   3.461 0.000539 ***
## as.factor(year)2014:sexmale  0.0235125  0.0133829   1.757 0.078935 .  
## as.factor(year)2015:sexmale -0.0006904  0.0132029  -0.052 0.958299    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.9608 on 304318 degrees of freedom
##   (280371 observations deleted due to missingness)
## Multiple R-squared:  0.02969,    Adjusted R-squared:  0.02965 
## F-statistic: 716.2 on 13 and 304318 DF,  p-value: < 2.2e-16

This suggest that the gap in political interest between men and women was slighty hgher in 2013, but the effect size is really small.

12.2.3 Adding a time-varying variable

Things get more tricky (and more interesting) if we want to add a time-varying variable such as age to the analysis. We have already modelled the association between age and political interest cross-sectionally. This answers the question of whether there is any difference in political interest between people of different age.

Another question that we may want to ask is whether political interest changes when people get older. Note that this is a different question, and answering it requires the use of longitudinal data.

To answer this question we need to apply regression models with fixed effects. You have not covered these models in Data Analysis 2. The main idea is to look at the changes for the same individuals. We want to see if for each person in the data getting older is associated with the changes in political interest, and then we can average the effects across different people. In other words, instead of fitting the model that compares between individuals we want to compare within individuals.

Technically, we can achieve this by simply controlling for individual id.

With the data of our size the model will be very difficult to estimate, so for the demonstration purposes I will select 500 random individuals from the data and run the model for them.

# creating a data frame with data for 500 random people
set.seed(1)
random500 <- sample(unique(UndSoc$pidp), 500)
UndSoc500 <- UndSoc %>%
  filter(pidp %in% random500)

m7 <- lm(polinterest ~ as.factor(year) + as.factor(pidp) + dvage, UndSoc500)

If we run summary(m7) now we will get a really long output with 499 coefficients for pidp. I will use stargazer to present the results and will omit these coefficients (called individual fixed effects) and the coefficients for year.

stargazer(m7, omit = c("pidp", "year"), type = "html")
Dependent variable:
polinterest
dvage 0.019
(0.071)
Constant 2.605
(4.254)
Observations 1,800
R2 0.760
Adjusted R2 0.677
Residual Std. Error 0.551 (df = 1333)
F Statistic 9.079*** (df = 466; 1333)
Note: p<0.1; p<0.05; p<0.01

Note that the coefficient for age is not statisticaly significant. This indicates that as people get older their interest in politics does not change much, at least in our sub-sample of 500 people.

A more efficient way to estimate fixed effects model is to use the package plm.

library(plm)
m8 <- plm(polinterest ~ dvage, data = UndSoc500, model = "within", index = c("pidp", "year"),
          effect = "twoways")
summary(m8)
## Twoways effects Within Model
## 
## Call:
## plm(formula = polinterest ~ dvage, data = UndSoc500, effect = "twoways", 
##     model = "within", index = c("pidp", "year"))
## 
## Unbalanced Panel: n = 460, T = 1-7, N = 1800
## 
## Residuals:
##     Min.  1st Qu.   Median  3rd Qu.     Max. 
## -1.72200 -0.26171  0.00000  0.20677  2.19176 
## 
## Coefficients:
##       Estimate Std. Error t-value Pr(>|t|)
## dvage 0.019376   0.070879  0.2734   0.7846
## 
## Total Sum of Squares:    404.82
## Residual Sum of Squares: 404.79
## R-Squared:      5.606e-05
## Adj. R-Squared: -0.34951
## F-statistic: 0.0747324 on 1 and 1333 DF, p-value: 0.78461

Asking for the twoways effect means that we fit the model with individual and year fixed effects.

Note that the effect size is exactly the same in models 7 and 8 (becuase this is essentially the same model), but plm will work faster. With plm you may be able to estimate the fixed effects model with the full data for your reports, although the estimation will take you some time. I will not do this here.

12.3 Further reading

If you want to use linear models in your assignment I strongly recommend you consult the following book (available as an e-book in the library):

  1. J.Fox, S.Weisberg. (2011). An R Companion to Applied Regression. 2nd ed. Sage.

The chapter on factors and interactions will be particularly useful.

If you want to know more about fixed effects models see

  1. P.D.Allison. (2008). Fixed Effect Regression Models. Sage.

To learn how to use the package plm read

  1. Y.Croissant, G.Millo. Panel Data Econometrics in R: The plm Package. https://cran.r-project.org/web/packages/plm/vignettes/plm.pdf