1) This question uses the variables dis (the weighted mean of distances

to five

1) This question uses the variables dis (the weighted mean of distances

to five Boston employment centers) and nox (nitrogen oxides concentration in parts per 10 million) from the Boston data. We will treat

dis as the predictor and nox as the response.

(a) Use the poly() function to fit a cubic polynomial regression to

predict nox using dis. Report the regression output, and plot

the resulting data and polynomial fits.

(b) Plot the polynomial fits for a range of different polynomial

degrees (say, from 1 to 10), and report the associated residual

sum of squares.

(c) Perform cross-validation or another approach to select the optimal degree for the polynomial, and explain your results.

(d) Use the bs() function to fit a regression spline to predict nox

using dis. Report the output for the fit using four degrees of

freedom. How did you choose the knots? Plot the resulting fit.

(e) Now fit a regression spline for a range of degrees of freedom, and

plot the resulting fits and report the resulting RSS. Describe the

results obtained.

(f) Perform cross-validation or another approach in order to select

the best degrees of freedom for a regression spline on this data.

Describe your results.

2) This question relates to the College data set.

(a) Split the data into a training set and a test set. Using out-of-state

tuition as the response and the other variables as the predictors,

perform forward stepwise selection on the training set in order

to identify a satisfactory model that uses just a subset of the

predictors.

(b) Fit a GAM on the training data, using out-of-state tuition as

the response and the features selected in the previous step as

the predictors. Plot the results, and explain your findings.

(c) Evaluate the model obtained on the test set, and explain the

results obtained.

(d) For which variables, if any, is there evidence of a non-linear

relationship with the response?

3) In Section 7.7, it was mentioned that GAMs are generally fit using

a backfitting approach. The idea behind backfitting is actually quite

simple. We will now explore backfitting in the context of multiple

linear regression.

Suppose that we would like to perform multiple linear regression, but

we do not have software to do so. Instead, we only have software

to perform simple linear regression. Therefore, we take the following

iterative approach: we repeatedly hold all but one coefficient estimate fixed at its current value, and update only that coefficient

estimate using a simple linear regression. The process is continued until convergence—that is, until the coefficient estimates stop changing.

We now try this out on a toy example.

(a) Generate a response Y and two predictors X1 and X2, with

n = 100.

(b) Initialize βˆ1 to take on a value of your choice. It does not matter

what value you choose.

(c) Keeping βˆ1 fixed, fit the model

Y − βˆ1X1 = β0 + β2X2 + .

You can do this as follows:

> a=y-beta1*x1

> beta2=lm(a∼x2)$coef[2]

(d) Keeping βˆ2 fixed, fit the model

Y − βˆ2X2 = β0 + β1X1 + .

You can do this as follows:

> a=y-beta2*x2

> beta1=lm(a∼x1)$coef[2]

(e) Write a for loop to repeat (c) and (d) 1,000 times. Report the

estimates of βˆ0, βˆ1, and βˆ2 at each iteration of the for loop.

Create a plot in which each of these values is displayed, with βˆ0,

βˆ1, and βˆ2 each shown in a different color.

(f) Compare your answer in (e) to the results of simply performing

multiple linear regression to predict Y using X1 and X2. Use

the abline() function to overlay those multiple linear regression

coefficient estimates on the plot obtained in (e).

(g) On this data set, how many backfitting iterations were required

in order to obtain a “good” approximation to the multiple regression coefficient estimates?

Submit a .doc file. Remember to include the following

* Answers and justifications

* R script and related outputs (if any)