#  Linear Models in Recommender Systems

**N. Matloff, UC Davis**

## Overview

In the collaborative filtering approach to recommender systems modeling,
a very simple but common model for the rating user i gives to item j is

Y<sub>ij</sub> = &mu; + u<sub>i</sub> + v<sub>j</sub> +
&epsilon;<sub>ij</sub>

where 

- &mu; is the overall mean rating over all users and items

- u<sub>i</sub> is the propensity of user i to rate items liberally or
  harshly 

- v<sub>j</sub> is the propensity of item j to be rated liberally or
  harshly 

- &epsilon;<sub>ij</sub> is an error term, incorporating all other
  factors

- taken as random variables as i and j vary through all users and
  items, u<sub>i</sub>, v<sub>j</sub>, and &epsilon;<sub>ij</sub> 
  are independent with mean 0

The form of the above model suggests using linear model software, e.g. 

``` r
library(dslabs)         
data(movielens)
ml <- movielens
ml <- ml[,c(5,1,6)]
ml$userId <- as.factor(ml$userId)
ml$movieId <- as.factor(ml$movieId)
lm(rating ~ .,data=ml)
```

At first glance, this seems like a questionable idea.  In this version
of the MovieLens data, there are 671 users and 9066 movies, thus nearly
10,000 dummy variables generated by **lm()**.  With only 100,000 data
points (and which are not independent), we run a real risk of
overfitting.  Worse, the code is quite long-running (over 2 hours in the
run I tried on an ordinary PC).

But it turns out there is a simple, fast, closed-form solution, both for
this model and for some more advanced versions featuring interaction
terms.

## Analysis:  Noniteractive model

Estimating &mu; is easy.  From its definition, we take our estimate to
be

Y<sub>..</sub> = 
&Sigma;<sub>i</sub> 
&Sigma;<sub>j</sub> 
Y<sub>ij</sub> / n

where is the total number of data points.

Write the above model in population form.  

Y = &mu; + U + I + e

Now consider user i, taking expectation conditioned on U = i:

E(Y | U = i) = &mu; + u<sub>i</sub> 

The natural estimate of the LHS is

Y<sub>..</sub> = &Sigma;<sub>i</sub> N<sub>i</sub>

where N<sub>i</sub> is the number of items rated by user i.

Our estimate for u<sub>i</sub> is then

Y<sub>i.</sub> - Y<sub>..</sub>

A similar derivation yields our estimate for v<sub>j</sub>,

Y<sub>.j</sub> - Y<sub>..</sub>
 
(under construction)
