The general Nature of Saturated Designs

TL;DR

This paper explores the properties and selection criteria of saturated designs in factorial experiments, focusing on how to retain information on significant effects when resources are limited and some effects are presumed negligible.

Contribution

It provides a theoretical analysis of the flexibility in choosing saturated designs to preserve information on important effects in resource-constrained factorial experiments.

Findings

Saturated designs can be flexibly chosen to retain key effect information.

Pre-knowledge of negligible effects guides optimal design selection.

The study offers insights into design choices under resource constraints.

Abstract

We contemplate an experimental situation in a $2^k$-factorial experiment with acute resource crunch so that we need to conduct just a saturated design [SD] - with the understanding that precision of the estimates cannot be estimated from the data. It is known beforehand which effect(s)/interaction(s) are likely to be negligible. We examine the flexibility to the extent that an experimenter can make a choice of an SD in order to retain information on all the remaining [non-negligible] effects/interactions.

Equations

$$D^{T}D-\lambda I_n=\gamma I_n-V^{T}V$$

$$det(D^{T}D)=N^{n-r}\prod _{i=1}^r(N-{\gamma }_i)$$

$$C{C}^T-\lambda I_n=\gamma I_d-V{V}^T$$

$$det(C{C}^T)=N^{d-r}\prod _{i=1}^r(N-{\gamma }_i)$$

$$(000,100),(000,110),(000,101),(000,111),(100,010),(100,001),(100,011),(010,110)$$

$$(010,101),(010,111),(110,001),(110,011),(001,101),(001,111),(101,011),(011,111).$$

$$(000,100,010,001);(000,100,110,111);(000,010,110,111);(000,001,101,111)$$

$$\{0000,1000,1100,0010,1010,0110,1110,0001,1001,0101\}$$

$$\{1000,0100,0010,0110,1110,0001,0101,1101,0011,1011,1111\}$$

Full text

Keywords and phrases: Saturated designs; Negligible effects; Admissible set for deletion; Relative efficiency; Hadamard matrices

1 Introduction

Two-level factorial designs (TLFD) are widely used in scientific and industrial experimentation for various reasons. Standard text books deal with this topic at various lengths - covering such concepts as (i) Unreplicated Full Factorials, (ii) Replicated Full Factorials, (iii) Blocking, (iv) Total, Partial and Balanced/Unbalanced Confounding, (v) Fractional Factorials etc. Practitioners primarily use TLFD at an early stage of an experimentation to screen potential factors that are involved in the system being investigated. The statistical models underlying TLFD are simple and subject to relatively weak assumptions. Each factor - whether quantitative or qualitative - is assumed to have two levels that are conveniently coded as $-1$ or $1$ in the design matrix. The estimators of the effects/interactions are contrasts that are naturally simple to interpret. The effect of a factor is interpreted as a measure of the change in the response variable due to variation of the factor from low to high - averaged over all other factor levels.

In practice investigators postulate, for one reason or the other, that certain effects (usually higher order interactions) are unimportant or negligible. When that is the case it is desirable for them to conduct the experiment with the least number of runs that would ensure the unbiased estimation of the important effects that is, non-negligible effects of interest. Regular Fractional Factorial Designs (RFFD) are used in this kind of situation and there is a vast literature available on RFFD. See, for example, Montgomery [[6]].

In the framework of a two-level factorial design, one of the drawbacks of RFFD is that the number of runs needed to conduct the experiment is necessarily a multiple of $4$. Thus when the important effects to be estimated are identified beforehand, using an RFFD may lead to the use of more resources than the bare minimum needed for the estimation of the important effects . For instance if the number of factors is $k=5$ and the only important effects are the main effects plus the mean then using an $2^{5-2}_{III}$ RFFD of Resolution $III$ would require $8$ runs for the experiment. This would actually estimate the $5$ main effects plus the mean but also can provide estimate two other effects that are known to be negligible.

A Saturated Designs (SD) could be used in case of scarce resources when it is clear to the investigator which effects are important and non-negligible. However it turns out that the identification of an SD is not a trivial problem. Numerous papers available in the literature discuss how to construct SDs under certain conditions. See Hedayat and Pesotan [[2]] and [[3]]. In addition various computer algorithms have been developed to search for SDs in the TLFD set-up. Some of these are SPAN, DETMAX. See Hedayat and Haiyuan Zhu [[4]]. It is worth pointing out that when RFFD are used to estimate a certain vector parameter of interest, the estimator of each effect except the mean is a contrast in terms of the runs and it is clear to practitioners that each estimator measures an interaction or the change in the response variable due to the variation of some factor from low to high. The common practice available in the literature is to choose an SD for which the underlying design matrix is non-singular. The Ordinary Least Squares (OLS) method is then used to obtain the estimator of the vector parameter of interest.

The question we may ask is the following ” is the estimator [blue] of each parameter (except the common mean) in a SD model a contrast in terms of the runs?”. Well, if the design matrix is a Hadamard matrix then the answer is trivially ’yes’ since the SD in that case can be seen as an RFFD. However when the design matrix is not a Hadamard matrix the estimator of the vector parameter is given by $\hat{\beta_{1}}=(D^{T}D)^{-1}D^{T}Y=D^{-1}Y$ where $D$ is the saturated design matrix. In practice it is desirable to practitioners to have the estimator of each estimable effect as contrast in terms of the runs for the sake of interpretation. It is interesting to verify that it is indeed so even when the design is not based on a Hadamard matrix. This can be seen as follows.

Let $D$ be a square non-singular matrix with its first column as a vector of $1$’s. Let $e_{1}$ denote the column vector of the same dimension with elements $(1,0,0,\ldots,0)$. Then $De_{1}$ is a vector of $1$’s. Hence, whenever $D$ is non-singular, $D^{-1}$ times vector of $1$’s= $e_{1}$. This is equivalent to the statement that the estimates of all model parameters [except the common mean ] are linear observational contrasts, irrespective of the nature of elements of the matrix $D$.

The rest of the paper is organized as follows. In Section 2, we develop general theory for ’deletion of exact number of runs’ so as to ensure estimability of all non-negligible effects/interactions in a saturated $2^{k}$-factorial experiment. This is done through identification of the runs to be deleted, for any given collection of non-negligible effects/interactions. The choice is not unique. However, we do not necessarily address the question of ’optimal choice’. In Section 3, we take up some illustrative examples in the case of $2^{3}$- and $2^{4}$-factorial experiments.

2 General theory for identification of runs for deletion - retaining estimability of non-negligible effects/interactions in a saturated design

We begin by listing several properties of a Hadamard matrix of order $N=2^{k}$.

Theorem 1**.**

*Let $H_{N}$ be a Hadamard matrix of order $N$ that is partitioned into block matrices as :

$H_{N}=\begin{bmatrix}D&E\\ V&C\end{bmatrix}$ where $D$ and $C$ are square matrices of order $n$ and $d$ respectively such that $n\geq d$. Then we have the following results:

$\boldsymbol{|det(D)|=N^{\frac{n-d}{2}}|det(C)|}$** 2. 2.

*if $D$ is invertible then $C$ is also invertible and the inverse of $D$ is given by: *

$\boldsymbol{D^{-1}=\frac{1}{N}[D-EC^{-1}V]^{T}}$**

Proof.

We have $H_{N}$ is a Hadamard matrix implies that $H_{N}H_{N}^{T}=H_{N}^{T}H_{N}=NI_{N}$. Therefore we have :

$D^{T}D=-V^{T}V+(n+d)I_{n}$

$D^{T}D-\lambda I_{n}=-V^{T}V+(n+d)I_{n}-\lambda I_{n}$

$D^{T}D-\lambda I_{n}=-V^{T}V+(n+d)I_{n}-\lambda I_{n}$

$D^{T}D-\lambda I_{n}=(n+d-\lambda)I_{n}-V^{T}V$

Let $\gamma=n+d-\lambda$ then we have the following equation:

[TABLE]

From Equation (1) if $\gamma$ is an eigenvalue of $V^{T}V$ then $\lambda=(n+d)-\gamma$ is eigenvalue for $D^{T}D$ and vice versa.

Now assume $rank(V)=r$. Then $r\leq d$ the $n\times n$ matrix $V^{T}V$ has $r$ non-zero eigenvalues of $\gamma_{1},\cdots,\gamma_{r}$ and $n-r$ zero eigenvalues. We deduce that $D^{T}D$ has $n-r$ eigenvalues $\lambda_{i}=n+d=N$; $i=1,\cdots,n-r$. The remaining $r$ eigenvalues of $D^{T}D$ are

$N-\gamma_{1},\cdots,N-\gamma_{r}$ . Since the determinant of a square matrix is the product of its eigenvalues it turns out that

[TABLE]

By analogy we have $CC^{T}-\lambda I_{n}=(n+d-\lambda)I_{d}-VV^{T}$ Let $\gamma=n+d-\lambda$ then we have the following equation:

[TABLE]

Furthermore, since $rank(V)=r\leq d$, the $d\times d$ matrix $VV^{T}$ has $r$ non-zero eigenvalues of $\gamma_{1},\cdots,\gamma_{r}$ and $d-r$ zero eigenvalues.

Therefore from Equation (3) the matrix $CC^{T}$ has $d-r$ eigenvalues $\lambda_{i}=n+d=N$; $i=1,\cdots,n-r$. The remaining $r$ eigenvalues of $D^{T}D$ are

$N-\gamma_{1},\cdots,N-\gamma_{r}$ .

It turns out that

[TABLE]

By Equations (3) and (4) we get $\prod_{i=1}^{r}(N-\gamma_{i})=N^{r-d}det(CC^{T})$ which implies that

$det(D^{T}D)=N^{n-r}N^{r-d}det(CC^{T})=N^{n-d}det(CC^{T})$.

It follows that $|det(D)|=N^{\frac{n-d}{2}}|det(C)|$.

To prove the second part of the theorem, we have $|det(D)|=N^{\frac{n-d}{2}}|det(C)|$ . This means that $D$ is invertible if and only if $C$ is invertible. Thus since $H_{N}$ is Hadamard we use the inversion formula for block matrices to get

$H_{N}^{-1}=\begin{bmatrix}D&E\\ V&C\end{bmatrix}^{-1}=\begin{bmatrix}[D-EC^{-1}V]^{-1}&-[D-EC^{-1}V]^{-1}EC^{-1}\\ -C^{-1}V[D-EC^{-1}V]^{-1}&[C-VD^{-1}E]^{-1}\end{bmatrix}=\frac{1}{N}H_{N}^{T}=\frac{1}{N}\begin{bmatrix}D^{T}&V^{T}\\ E^{T}&C^{T}\end{bmatrix}$

$\frac{1}{N}D^{T}=[D-EC^{-1}V]^{-1}$

$\frac{1}{N}C^{T}=[C-VD^{-1}E]^{-1}$

The results follow easily. ∎

Corollary 1**.**

Let $\Theta_{n}$ be the set of non-singular matrices of order n with entries from $\{-1,1\}$ for which the first column is $1_{n}$. Let $D\in\Theta_{n}$. Then there exists a Hadamard matrix $H_{N}$ of the form $H_{N}=\begin{bmatrix}D&E\\ V&C\end{bmatrix}$ such that :

$\boldsymbol{D^{-1}=\frac{1}{N}[D-EC^{-1}V]^{T}}$.

Proof.

The $\{-1,1\}$-matrix $M_{n}$ of order $2^{n}\times n$ formed with all the $n$-tuples from the set $\{-1,1\}$ can be extended to a Hadamard matrix $H_{N}$. Let $\mathcal{C}=m_{1},\cdots,m_{n}$ be the set containing the columns of $M_{n}$. To construct $H_{N}$, it just suffices to add the schur product of any non-empty subset of $\mathcal{C}$ as a new column to $M_{n}$ as well as the column vector $1_{N}$. It is not hard to see that since $D$ is non-singular of order $n$ its rows appear without repetition. Therefore each row of $D$ is also a row of $M_{n}$. It turns out that for any non-singular $\{-1,1\}$-matrix $D$ there exists a Hadamard matrix $H_{N}=\begin{bmatrix}D&E\\ V&C\end{bmatrix}$. By the last result stated in Theorem 1, $\boldsymbol{D^{-1}=\frac{1}{N}[D-EC^{-1}V]^{T}}$. ∎

Lemma 1**.**

*Consider a Hadamard matrix of the form $H_{N}=\begin{bmatrix}D&E\\ V&C\end{bmatrix}$ where $D$ is invertible. Assume the first columns of $H_{N}$ and $D$ are respectively $1_{N}$ and $1_{n}$ Then we have :

$\boldsymbol{(D-EC^{-1}V)^{T}1_{n}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}}$*

Proof.

$(D-EC^{-1}V)^{T}1_{n}=D^{T}1_{n}-V^{T}(C^{-1})^{T}E^{T}1_{n}$

Since we assume the first column of $H_{N}$ is $1_{N}$ we have $D^{T}1_{n}+V^{T}1_{d}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}$ which implies that $D^{T}1_{n}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}-V^{T}1_{d}$.

Also $E^{T}1_{n}+C^{T}1_{d}=0$ implies that $E^{T}1_{n}=-C^{T}1_{d}$ .

It turns out that $(D-EC^{-1}V)^{T}1_{n}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}-V^{T}1_{d}+V^{T}(C^{-1})^{T}C^{T}1_{d}$

$(D-EC^{-1}V)^{T}1_{n}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}-V^{T}1_{d}+V^{T}(C^{-1}C)^{T}1_{d}=-V^{T}1_{d}+V^{T}1_{d}=0$

$(D-EC^{-1}V)^{T}1_{n}=\begin{bmatrix}N\\ 0_{n-1}\end{bmatrix}$ ∎

Remark 1*.*

The inverse of a non-singular $\{-1,1\}$-matrix $D$ is always of the form $D^{-1}=\frac{1}{N}(D-EC^{-1}V)^{T}$. Lemma 1 shows that if one of the columns of $D$ is $1_{n}$ then $D^{-1}$ has a row for which the entries sum up to $1$. The entries of any of the remaining rows of $D^{-1}$ sum up to zero. This property is mathematically interesting but most importantly , it shows that in general the estimator of any effect or interaction except the mean in a saturated design is a contrast in terms of the runs. This eases the interpretation of the results of a saturated design conducted in a two-level factorial setup.

Deletion Algorithm

In the context of a $2^{k}$-factorial experiment, suppose the experimenter has identified a subset of, say $n$, factorial effects and interactions which are supposed to be non-negligible. Each of the remaining [$2^{k}-n$] effects and interactions is, by default, assumed to be negligible. Furthermore suppose the experimenter has minimal resource to carry out an $n$-run saturated design. Naturally the runs must be chosen so that all the non-negligible effects are unbiasedly estimated.

The following steps seem to be plausible to follow :

Write down the vector parameter of general mean, factorial effects and interactions as a column vector of dimension $2^{k}\times 1$ so that the general mean and all non-negligible parameters form a subset at the top and the negligible effects and interactions appear at the bottom. It is well-known that in a $2^{k}$-factorial set-up, we have standard representations for the effects and interactions in terms of the $2^{k}$ observations arising out of the full factorial experiment, if that were the case. 2. 2.

Since the experiment is to be based on a suitably chosen subset of $n$ runs i.e., level-combinations of the factors, the experimenter has to make a judicious choice of these runs. Let $Y^{(1)}$ stand for the $n\times 1$ vector of observations realized after performing the experiment with suitably chosen $n$ runs. Let $Y^{(2)*}$ denote the complementary unobserved vector of dimension $(2^{k}-n)\times 1$ in this context. 3. 3.

The Hadamard matrix $H_{N}$ of order $N=2^{k}$ is decomposed in the usual manner wherein the matrix $D$ corresponds to the non-negligible parameters and the matrix $C$ corresponds to the negligible parameters. 4. 4.

Let $\theta$ denote the vector parameter of the general mean and all the effects and interactions written in the style of (1) above. We use the notations $\theta^{(1)}$ and $\theta^{(2)}$ for the decompositions described under (1). Then $\theta^{(1)}$ represents the non-negligible effects and interactions whereas $\theta^{(2)}$ represents the ’zero’ effects and interactions.

Note that $Y^{(1)}$ and $Y^{(2)}$ have already been defined as per this decomposition. Recall the partitioned matrix representation of $H_{N}$. From Yates’ representation of the factorial effects and interactions, it follows that, but for a constant multiplier, 5. 5.

$\mathbb{E}[\frac{1}{N}H_{N}^{T}Y]=\theta$ i.e., $(i)\frac{1}{N}\mathbb{E}[D^{T}Y^{(1)}+V^{T}Y^{(2)*}]=\theta^{(1)}$; (ii) $\mathbb{E}[E^{T}Y^{(1)}+C^{T}Y^{(2)*}]=0.$ 6. 6.

From (5)(ii) if $D$ is non-singular then $C$ is also non-singular by Theorem (1) and the best linear unbiased predictor [BLUP] of $Y^{(2)*}$ is given by : $\hat{Y}^{(2)*}=-(C^{(-1)})^{T}E^{T}Y^{(1)}$. 7. 7.

Therefore, the BLUE of $\theta^{(1)}$ is given by $\hat{\theta}^{(1)}=\frac{1}{N}(D-EC^{(-1)}V)^{T}Y^{(1)}.$ 8. 8.

Further, the dispersion matrix of $var(\hat{\theta}^{(1)})\propto[D-VC^{(-1)}E]^{T}[D-VC^{(-1)}E]$. 9. 9.

Incidentally, for d-optimal choice of the matrix $D$, or equivalently, of the matrix $C$, we need to minimize $|det[(D-VC^{(-1)}E)|$ which is equivalent to maximizing $|det(C)|$.

Thus far we have explained the general principle underlying choice of the $D$-matrix for any given vector $\theta^{(2)}$ of negligible parameters. A little reflection suggests that the choice of the matrices (i) $\begin{bmatrix}D&E\end{bmatrix}$ of order $n\times N$ and (ii) $\begin{bmatrix}V&C\end{bmatrix}$ of order $N-n\times N$ are dictated by the two sets of non-negligible and negligible parameters. However, their splitting can be very much arbitrary subject to fulfilling the non-singularity conditions by the square matrices $D$ and $C$. We give three simple illustrations below.

Illustrations : $2^{3}$ - factorial experiment

(i) Only one effect / interaction is negligible: It may be seen that any one run can be deleted.

(ii) A pair of effects/interactions are negligible : It is not true that any two runs can be deleted. For example, if $F_{23}$ and $F_{123}$ are negligible, we may only delete the following pairs of runs :

[TABLE]

[TABLE]

It is interesting to note that the runs $(011)$ and $(111)$ may as well be deleted.

(iii) When only the mean effect and three main effects are non-negligible, we may delete any of the following sets of four runs :

[TABLE]

Interestingly, the runs $(110),(101),(011),(111)$ do not form an admissible set for deletion.

3 Maximization criterion - an optimality consideration

3.1 A general rule for admissible set

Theorem (1) states that whenever a Hadamard matrix $H_{N}$ is partitioned into block matrices as $\begin{bmatrix}D&E\\ V&C\end{bmatrix}$ then $|det(D)|=N^{\frac{n-d}{2}}|det(C)|$ and that the matrix $D$ is non-singular if and only if the matrix $C$ is non-singular. The take-away message here in terms of a saturated design is that there is direct relationship between a saturated design matrix $D$ and the matrix $C$. What it means is that when the experimenter is deciding which runs to keep in a saturated design he has two equivalent options he can choose from. The first choice is to directly search the runs that will make the matrix $D$ underlying the parameters of interest non-singular. The second choice is to select a set of runs so that the matrix $C$ underlying the negligible parameters is non-singular. The complement of such set of runs is then the desired saturated design. Thus when the experimenter is trying to search for a saturated design matrix $D$ of order $n$ then if $d=2^{k}-n$ is such that $n>d$, it is advantageous to the experimenter to deal with a less complex problem by searching for the matrix $C$ of order $d$. The design matrix can then be taken for granted by just taking the complement of $C$ which is the matrix $D$ in the partition of the Hadamard matrix $H_{N}$ above. Furthermore since $|det(D)|\propto|det(C)|$ it means that the Fisher information contained in a saturated design $D$ is proportional to the amount of information contained in its complement matrix $C$. This is important if one desires to find a d-optimal design or classify saturated designs by their amount of Fisher information. In fact if $n>d$ the classification complexity of saturated design matrices $Ds$ boils down to the study of the determinant of the matrices $Cs$ which is less complex. For convenience we make the following definitions:

Definition 1**.**

Consider a $2^{k}$-factorial experiment where the full vector parameter $\theta$ is partitioned as $\theta=\begin{bmatrix}\theta^{(1)}&\theta^{(2)}\end{bmatrix}^{T}$ where $\theta^{(1)}$ is unknown and non-negligible and $\theta^{(2)}$ is negligible with cardinality $|\theta^{(1)}|=n$ and $|\theta^{(2)}|=2^{k}-n$ . Furthermore suppose the set of runs $R$ is partitioned as $R=\begin{bmatrix}R_{1}&R_{2}\end{bmatrix}$ with cardinality $|R_{1}|=n$ and $|R_{2}|=2^{k}-n$ such that the Hadamard matrix $H_{N}$ underlying $\theta$ and $R$ is written as :

$\theta^{(1)}$ $\theta^{(2)}$

$R_{1}$ $D$ $E$

$R_{2}$ $V$ $C$

where $D$ and $C$ are square matrices of order $n$ and $2^{k}-n$ respectively. We make the following definitions :

We shall say the set $R_{2}$ is an admissible set for deletion if the matrix $C$ is non-singular. 2. 2.

If $R_{2}$ is admissible then its complement $R_{1}$ is a saturated design for the estimation of $\theta^{(1)}$. 3. 3.

If $|det(C)|$ is maximal we shall say $R_{2}$ is a d-optimal admissible set for deletion 4. 4.

If $|det(C)|$ is maximal then $R_{2}$ is a d-optimal admissible set and we shall say that its complement $R_{1}$ is a d-optimal design for the estimation of $\theta^{(1)}$.

3.2 Example of classification of saturated designs by determinant for mean, main effects and the two-factor interactions: $2^{4}$-experiment case

Suppose for one reason or the other the experimenter is interested in a saturated design with $k=4$ factors where the important effects are $F_{0},F_{1},F_{2},F_{3},F_{4},F_{12},F_{13},F_{14},F_{23},F_{24},F_{34}$ and the negligible effects are $F_{123},F_{124},F_{134},F_{234},F_{1234}$. For such a problem the experimenter needs to find 11 runs out of the 16 possible runs that would make the underlying design matrix of order 11 non-singular. It turns out that the task of finding such a design matrix is computationally more involved than finding an admissible set for the 5 negligible parameters. Since there are only 5 negligible effects, a shortcut to finding the desired design matrix would be to first take a look at the $2^{4}$-full factorial design matrix which is a Hadamard matrix and try to come up with a non-singular design matrix for the negligible effects $F_{123},F_{124},F_{134},F_{234},F_{1234}$. The complement of such design matrix would be the desired design matrix to estimate the important effects and interactions.

In the first $2^{4}$-Hadamard matrix in Figure (1) , we display in bold the $C$- matrix $C_{1}$ underlying the runs

$\{1101,0011,1011,0111,1111\}$ and the interactions $\{F_{123},F_{124},F_{134},F_{234},F_{1234}\}$ .

$C_{1}=\begin{bmatrix}-1&1&-1&-1&-1\\ 1&1&-1&-1&1\\ -1&-1&1&-1&-1\\ -1&-1&-1&1&-1\\ 1&1&1&1&1\end{bmatrix}$ .

It is not hard to see that this particular $C$-matrix is singular since the first and the last columns are the same. Thus the set of runs $\{1101,0011,1011,0111,1111\}$ is not an admissible set and so its complement set of runs

[TABLE]

is not a valid set for the estimation of the mean, the main effects and the two-factor interactions. In fact by Theorem (1) the $D$-matrix underlying this set is also singular.

On the other hand, the $C$-matrix $C_{2}$ displayed in bold in the second $2^{4}$-Hadamard matrix in Figure (1) underlies the runs

$\{0000,1100,1010,1001,1111\}$ and the interactions $\{F_{123},F_{124},F_{134},F_{234},F_{1234}\}$ .

$C_{2}=\begin{bmatrix}-1&-1&-1&-1&1\\ -1&-1&1&1&1\\ -1&1&-1&1&1\\ 1&-1&-1&1&1\\ -1&-1&-1&1&-1\end{bmatrix}$ .

The absolute value of the determinant of this choice of $C$-matrix is $|det(C_{2})|=48$. Therefore, the set of runs $\{0000,1100,1010,1001,1111\}$ is an admissible set for deletion.

It is further observed that the choice of the matrix $C$ leading to an admissible set for deletion, is not necessarily unique. In that case, we might like to make a judicial choice for it. For this we refer to the dispersion matrix of the blue of $\theta^{(1)}$ and minimize the generalized variance.

It is readily seen that it amounts to maximization of the absolute value of $det(C)$ where $C$ is a square matrix of dimension $N-n$ with elements in $\{-1,1\}$.

It is a well known result that the maximal absolute value of the determinant of $\{-1,+1\}$-matrices of order 5 is $48$ and It is easy to verify that computationally.

Thus the absolute value of the corresponding $D$-matrix $D_{2}$ is $|det(D_{2})|=16^{\frac{11-5}{2}}\times 48$. We may thus conclude that the set

[TABLE]

is d-optimal design for the estimation of $F_{0},F_{1},F_{2},F_{3},F_{4},F_{12},F_{13},F_{14},F_{23},F_{24},F_{34}$.

The spectrum $S_{n}$ of the determinant function of $\{-1,+1\}$-matrices $M_{n}$ of order $n$ is defined to be the set of values taken by $|det(M_{n})|/2^{n-1}$. It is well known in the literature that the spectra $S_{5}=\{0,1,2,3\}$ and

$S_{11}=\{0,\cdots,268;\quad 270,\cdots,276;\quad 278,\cdots 280;\quad 282,\cdots,286;\\ 288,291;\quad 294,\cdots,297;\quad 304,312,315,320\}$ . See Orrick [[7]]. It is important to point out that the absolute value of the determinant of $\{-1,+1\}$-matrices of order $11$ can take up to $296$ values. Moreover the absolute value of the determinant of $\{-1,+1\}$-matrices of order $5$ can take up to $4$ values including the value [math]. Thus for the problem of classifying the saturated designs for $F_{0},F_{1},F_{2},F_{3},F_{4},F_{12},F_{13},F_{14},F_{23},F_{24},F_{34}$ by determinant there will be only 3 classes of designs. This is because $S_{5}$ can only take $3$ non-zero values. In fact we have $|det(M_{5})|$ takes value from $\{0,16,32,48\}$. Therefore by Theorem (1), $|det(C)|$ takes values from $\{0,16,32,48\}$ and $|det(D)|$ takes values from

$\{0,16^{3}\times 16,16^{3}\times 32,16^{3}\times 48\}$ . In Figure (2) we give 10 examples in each class of saturated designs. the last class correspond to the d-optimal design class.

4 Concluding Remarks

The construction of saturated designs for two level factorial experiment has gained a substantial interest over a long period of time. Numerous papers have been written about the classification of saturated design matrices of fixed order via the spectrum of the determinant function. Thus the spectra of the determinant function $S_{n}$ for $\{-1,+1\}$-matrices of order $n$ are well known in the literature for order up to $11$. The spectrum of order $n=8$ is due to Metropolis, Stein and Well [[5]]. For $n=9$ and $n=10$, the spectra were computed by Živković [[8]] and the spectrum for $n=11$ is due to Orrick [[7]]. Furthermore many other papers have studied d-optimal saturated design matrices for a fix order. Orrick [[7]] constructed a d-optimal design matrix of order $15$. T. Chadjipantelis, S. Kounias and C. Moyssiadis [[1]] came up with a d-optimal design of order $21$. The the d-optimal design matrix discussed by these papers is the $\{-1,+1\}$-matrix of order $n$ with the largest absolute value of the determinant among all other $\{-1,+1\}$-matrices of order $n$. In some sense these d-optimal design matrices are the global d-optimal design matrices. From a statistical design perspective, these matrices are really d-optimal if the only parameters that are important are the mean and the main effects. However when the experimenter is willing to construct a d-optimal design matrix that includes the mean, main effect and a certain number of interactions, the columns of the interactions in the design matrix are obtained by the Schür product of the appropriate columns of main effects. With that being said the columns of the interactions are deterministic once the columns of the main effects are chosen. It turns out that because of the restriction imposed by the interaction columns, a d-optimal design matrix of order $n$ for a chosen vector parameter that includes the mean , the main effects and a selected set of interaction may not be the global d-optimal design of order $n$. In many cases Theorem (1) can simplify the problem of studying the spectrum or finding a d-optimal design that includes the mean, the main effects and a selected set of interaction. The example given in subsection (3.2) is a nice illustration.

5 Acknowledgements

This work is partially supported by the US National Science Foundation(NSF Grant 1809681).