Log-linear model and correspondence analysis: theory and practice

Log-linear model and correspondence analysis are two methods used to study the interaction between variables, which are mostly qualitative, and distributed in several categories organised in a contingency table.

1. Log-linear model:

In the case of a two-dimensional contingency table, the interaction between two variables X1 and X2 is investigated. X1 has m categories and X2 has n categories. If Pij is the probability of categories i (i=1,2, ... ,m) of X1 and j (j=1,2, ... ,n) of X2, the logarithm of Pij is linearly decomposed as:

$\log p_{ij} = \mu + \alpha_{1}(i) + \alpha_{2}(j) + \beta_{12}(i,j)$ (1)

µ is the global effect, α₁(i) is the marginal effect resulting from X₁, α₂(j) is the marginal effect resulting from X2and β₁₂(i,j) is the cross effect (also called first order interaction). Constraints on the parameters are introduced to make the decomposition of $\log p_{ij}$ unique. These constraints are similar to those of the analysis of the variance:

$\sum\limits_{i=1}^{m}{\alpha_{1}(i)=0} \sum\limits_{j=1}^{n}{\alpha_{2}(j)=0} \sum\limits_{i=1}^{m}{\beta_{12}(ij)=0} \sum\limits_{j=1}^{n}{\beta_{12}(ij)=0}  \forall i,\forall j$

With these constraints, the model is said to be saturated.

Log-linear model aims to investigate whether, for a given dataset, some of the cross effects can be restricted to have value zero leading to the independence of the corresponding variables. For instance, if the cross effect parameters β12 in model (1), are significantly equal to zero, we can conclude that variables X1 and X2 are independent. To achieve this goal, we must compute the expected frequencies under the model of independence and evaluate whether the difference between the expected frequencies and the observed frequencies is significantly large. This evaluation can take place with the aid of Pearson’s goodness-of-fit statistic χ2 with the formula:

$\chi^{2} = \sum{\frac{(observed-expected)^{2}}{expected}}$

which, under the independence hypothesis, is distributed as chi-squared variate. In the case of the two-dimensional contingency table, this statistic can be formulated:

$\chi^{2} = \sum\limits_{ij}{\frac{(f_{ij} - f_{i.}f_{.j})^{2}}{f_{i.}f_{.j}}}$

where $f_{ij}$ are the observed frequencies, $f_{i.}$ and $f_{.j}$ are the maximum likelihood estimates of the expected frequencies p_i. and p_.j.

2. Link with correspondence analysis:

There are two reasons for the resemblance between correspondence analysis and the log-linear model:
– correspondence analysis deals mainly with contingency tables
– correspondence analysis investigates the total inertia of a cloud of points by projecting it on factorial axis crossing its gravity centre.

In fact, in the two-dimensional contingency table, X₁ categories form a cloud N(I) of m points in $\Re^{n}$, while X₂ categories form a cloud N(J) of n points in $\Re^{m}$. Inertia of each cloud is given by:

$In(I) = In(J) = \sum\limits_{ij}{\frac{(f_{ij} - f_{i.}f_{.j})^{2}}{f_{i.}f_{.j}}}$

which is precisely the distance between reduced and saturated models in log-linear analysis. Therefore, correspondence analysis decomposes the difference between two given log-linear models. We can extend this property to multiple-entry contingency tables. In fact, given a Q-entry table gathering, the categories of Q variables, the saturated log-linear model of such a table will contain the marginal effects of the Q variables and the interactions ordered from 2 to Q.

Correspondence analysis will be applied on the table F^AXB where A and B are two subsets of variables such as $A\cap B=\varnothing$ and $A\cup B=E$, E being the set of the Q variables.

Correspondence analysis could therefore decompose the difference between the two log-linear models: the saturated [AB] and the model of independence [A][B] .

3. Extending the relationship between log-linear analysis and correspondence analysis:

We propose to investigate the links between the log-linear model for a set of Q qualitative variables and multiple correspondence analysis applied on tables such as Burt matrix.