6.8 Unified Approach to Balanced ANOVA Models
We can develop a unified approach to obtaining orthogonal projection operatores in arbitrary balanced \(k\)-way ANOVA models by exploting the structure of design matrix. The structure of the design matrix can be easily examined using Kronecker products. Therefore, before we proceed further, we need to establish some more properties of Kronecker products.
Kronecker Product \(:= A \otimes B = (a_{ij}B)\).
Consider the balanced two-way ANOVA model with interaction. This model is given by
$ Y_{ijk} = + _ i + j + {ij} + _{ijk} $
where \(i=1, \cdots, a\), \(j=1, \cdots, b\), \(k=1, \cdots, N\), and \(n=abN\).
We want to write
$ (M) = (M_) + (M_) + (M_) + (M_) $
and be able to compute the orthogonal projection operators in an easy and unified way.
We can represent each subspace making up \(\mathcal{C}(M)\) in terms of Kronecker produdcts. Once we do this, we can easily obtain the orthogonal projection operator for that space.
※ Notation: let \(s\) be an arbitrary index. Define \(J_s\) as the \(s \times 1\) vector of ones, $P_s = J_s J_s ’ $ and \(Q_s = I_s - P_s\), where \(I_s\) is the \(s \times s\) identity matrix.
Thus, \(P_s\) is the orthogonal projection operator onto \(\mathcal{C}(J_s)\) and \(Q_s\) is the orthogonal projection operator onto \(\mathcal{C}(J_s)^\perp\)
※ Facts:
- recall that the OPO onto \(\mathcal{C}(A)\) is always given by by \(A(A'A)^{-}A'\).
- if \(M\) is an OPO, then \(M^{-} = M\).
$ Kronecker Product forms for the OPO
- Computing \(M_\mu\).
We can write \(J_n = J_\a \otimes J_b \otimes J_N\), so that \(M_\mu\) is the OPO onto \(\mathcal{C} \Big( J_\a \otimes J_b \otimes J_N \Big)\). Thus by Fact 1 above, we have
$
& &&M_&& &&
\
&= &&( J_J_b J_N ) &&( ( J_a ’ J_b ’ J_N ’ ) ( J_a J_b J_N ) )^{-} &&( J_a ’ J_b ’ J_N ’ )
\
&= &&( J_a J_b J_N ) &&( J_a ’ J_a J_b ’ J_b J_N ’ J_N)^{-} &&( J_a ’ J_b ’ J_N ’ )
\
&= &&( J_a J_b J_N ) &&( ab N)^{-} &&( J_a ’ J_b ’ J_N ’ )
&= && J_a J_a ’ + J_b J_b ’ + J_N J_N’
\
&= &&P_a P_b P_N
$
Using the properties of Kronecker products, it can be easily shown that \(M = I_a \otimes I_b \otimes P_N\).
the error space is \(\mathcal{C}(I-M)\) and
$
I-M
&= I_{abN} - M \
&= ( I_a I_b I_N ) - ( I_a I_b P_N ) \
&= ( I_a ) ( I_N - P_N ) \
&= I_a I_b Q_N
$
observe that
$
\[\begin{align} M + I - M &= ( I_a \otimes I_b \otimes P_N ) + (I_a \otimes I_b \otimes Q_N) \\ &= ( I_a \otimes I_b) \otimes(P_N + Q_N) \\ &= ( I_a \otimes I_b) \otimes (I_N) \\ &= I_a \otimes I_b \otimes I_N \\ &= I_n \end{align}\]
$
We can summarize the subspace and the OPO for the two-way ANOVA model as follows.
- Excercise
Consider the three-way ANOVA model
write out the subspaces and all OPO corresponding to each term in the ANOVA model completlely in terms of Kronecker.
Find the simplest expression for \(M_\mu + M_\alpha + M_\eta\).