A Minimal Set of Shannon-type Inequalities for Functional Dependence Structures

TL;DR

This paper characterizes a minimal set of Shannon-type inequalities considering functional dependence constraints, aiding in the analysis of network coding capacity and inequality redundancy.

Contribution

It introduces a method to identify a minimal set of Shannon-type inequalities under functional dependence constraints, reducing redundancy.

Findings

Identifies redundancies among elemental inequalities with functional dependencies

Provides a framework for computing Shannon outer bounds in network coding

Simplifies the analysis of feasible source rate regions

Abstract

The minimal set of Shannon-type inequalities (referred to as elemental inequalities), plays a central role in determining whether a given inequality is Shannon-type. Often, there arises a situation where one needs to check whether a given inequality is a constrained Shannon-type inequality. Another important application of elemental inequalities is to formulate and compute the Shannon outer bound for multi-source multi-sink network coding capacity. Under this formulation, it is the region of feasible source rates subject to the elemental inequalities and network coding constraints that is of interest. Hence it is of fundamental interest to identify the redundancies induced amongst elemental inequalities when given a set of functional dependence constraints. In this paper, we characterize a minimal set of Shannon-type inequalities when functional dependence constraints are present.

Equations

$$h=[h(\mathcal{A}):\mathcal{A}\subseteq \mathcal{V}\setminus \varnothing {]}^{\text{T}}\in {\mathbb{R}}^{2^n-1}.$$

$$H_h(A\mid \mathcal{V}\setminus A)$$

$$I_h(A;B\mid \mathcal{C})$$

$$H_h(A|\mathcal{B})$$

$$I_h(A;B|\mathcal{C})$$

$$m=n+\left(\genfrac{}{}{0ex}{}{n}{2}\right)2^{n-2}$$

$$\Gamma \triangleq \{h:\mathbf{G}h\ge 0\}$$

$$\tilde{\mathcal{C}}=\bigcup _{A\in \mathcal{C}}\tilde{A}.$$

$$h(\mathcal{C})=\mu (\tilde{\mathcal{C}}).$$

$$\bigcap _{A\notin \mathcal{C}}\tilde{A}\setminus \tilde{\mathcal{C}}$$

$$H_h(\mathcal{A}|\mathcal{C})$$

$$I_h(\mathcal{A};\mathcal{B}|\mathcal{C})$$

$$T_h(\alpha )\triangleq \mu ([\alpha ]),\alpha ⊊\mathcal{V}.$$

$$h(\beta )=\sum _{\alpha :\beta \setminus \alpha \ne \varnothing }{T}_h(\alpha ).$$

$$H_h(\mathcal{X}|\mathcal{Y})=0.$$

$$\Phi \triangleq \{({\mathcal{X}}_{\ell },{\mathcal{Y}}_{\ell }){\}}_{\ell =1}^L$$

$$\begin{gathered} \text{Optimize }f(h)\text{ subject to} \\ \{\begin{array}{l}h\in \Gamma \\ h\text{ satisfies functional dependencies in }\Phi .\end{array} \end{gathered}$$

$$\begin{gathered} \text{Minimize }{\mathbf{b}}^{\text{T}}h \\ \text{Subject to}\{\begin{array}{l}h\in \Gamma \\ h\text{ satisfies functional dependencies in }\Phi .\end{array} \end{gathered}$$

$$H_h(\mathcal{D}|\mathcal{C})=0$$

$$T_h(\mathcal{C})=0$$

$$H(A|\mathcal{V}\setminus A)$$

$$I(A;B|\mathcal{C})$$

$$I(A;B|\mathcal{C})=I(A;B|\mathcal{D})$$

$$\text{cl}(\mathcal{C})=\text{cl}({\mathcal{C}}^{\prime })$$

$$\{\text{cl}(AC),\text{cl}(BC)\}=\{\text{cl}(A^{\prime }{\mathcal{C}}^{\prime }),\text{cl}(B^{\prime }{\mathcal{C}}^{\prime })\},$$

$$I(A;B|\mathcal{C})\ge 0⟺I(A^{\prime };B^{\prime }|{\mathcal{C}}^{\prime })\ge 0.$$

$$I(A;B|\mathcal{C})\ge 0.$$

$$D\in \text{cl}(A\mathcal{C}).$$

$$H_h(D|A\mathcal{C})=0$$

$$I_h(A;B|\mathcal{C})=I_h(D;B|\mathcal{C})+I_h(A;B|\mathcal{C},D).$$

$$I(D;B|\mathcal{C})\ge 0$$

$$I(A;B|\mathcal{C},D)\ge 0.$$

Full text

I Introduction

Shannon-type inequalities (also called basic inequalities) are critical tools to obtain converse coding theorems (or outer bounds) for the capacity of communication systems. Often the structure of a communication network induces additional functional dependence constraints on the random variables involved in the system model. In [1], Yeung gave a framework for information inequalities and characterized a minimal set of Shannon-type inequalities for random variables (in the absence of further functional dependency structures). The characterization of these inequalities provides a mechanical framework for proof of information inequalities and numerical computation of outer bounds for communication networks. A notable example is the explicitly computable outer bound (often called the Linear Programming bound or LP bound) for the multi-source multi-sink network coding problem (see [2] and [3]). A related problem is to determine whether a given information inequality is Shannon-type (i.e. is implied by the Shannon inequalities). This is a redundancy check problem. A computer program called Information Theoretic Inequality Prover (ITIP) [4] is available to solve such linear programs.

In this context, the problems of (a) computing the network coding LP bound, and (b) proving basic information inequalities can both be formulated as linear optimizations with the elemental inequalities as a subset of the constraints. One practical challenge in solving these optimization problems is the large number of variables and constraints. The number of elemental inequalities grows exponentially with the number of variables, making it challenging and sometimes computationally infeasible to generate the set of constraints directly. Therefore, it is of fundamental importance to reduce complexity by eliminating redundant constraints. In this paper, we address this problem by characterizing a minimal set of Shannon-type inequalities subject to the presence of additional functional dependence constraints.

Section II presents the framework for information inequalities, elemental inequalities, set-theoretic interpretation of information measures and some applications of the set of elemental inequalities under equality constraints such as functional dependence. In Section III, we characterize a set of redundant elemental inequalities for a given functional dependence structure. Theorem 1 provides characterization of minimal elemental inequalities for functional dependence constraints and the proof is presented in Section IV. In Section V, we discuss some applications of the main results and future directions.

II Background

II-A Entropy space

For a set of random variables $\mathcal{V}=\{A,B,\dots\}$ with $|\mathcal{V}|=n$, let $h:2^{{\mathcal{V}}}\mapsto{\mathbb{R}}$ be a real-valued function defined on the non-empty subsets of $\mathcal{V}$111With a slight abuse of notation, $2^{\mathcal{A}}$ denotes the set of all non-empty subsets of $\mathcal{A}$ rather than the power set of $\mathcal{A}$. Singletons are represented without braces.. The function $h$ can also be viewed as a point in a $2^{n}-1$ dimensional Euclidean space, where the non-empty subsets of $\mathcal{V}$ are the indexes of the coordinate axes. This space is the entropy space $\mathcal{H}$ [2]. For notational simplicity, represent $h$ as a vector

[TABLE]

Let $\Gamma$ be the set of all vectors satisfying the elemental Shannon-type inequalities. These basic inequalities define the polymatroid axioms:

[TABLE]

where

[TABLE]

In cases when the vector $h$ is understood implicitly, we will denote $H_{h}(A|\mathcal{B})$ and $I_{h}(A;B|\mathcal{C})$ simply as $H(A|\mathcal{B})$ and $I(A;B|\mathcal{C})$.

The region $\Gamma$ is a polyhedron. In particular it is a pointed cone in the non-negative orthant ${\mathbb{R}}_{+}^{2^{n}-1}$. We refer to (1) resp. (2) as the non-decreasing resp. submodular elemental elemental inequalities. Straightforward enumeration shows that there are

[TABLE]

elemental inequalities. It has been proved that these elemental inequalities are non-redundant and that every basic inequality is implied by this set [5].

As the inequalities (1)–(2) are linear, the set $\Gamma$ can be written in matrix form as

[TABLE]

where $\mathbf{G}$ is a $m\times 2^{n}$ matrix with entries from $\{-1,0,1\}$, and $\mathbf{0}$ is a length $2^{n}$ all-zero vector. Each row of $\mathbf{G}$ encodes one elemental inequality. The ordering of columns in $\mathbf{G}$ is consistent with the coordinates of ${h}$, e.g., lexicographical ordering on subsets of $\mathcal{V}$.

II-B Entropy characterization using atoms

An alternative geometric representation based on a set-theoretic interpretation of information measures was given in [5], which we will re-state below.

For each variable $A$ in ${\mathcal{V}}$, it corresponds to a set labelled as $\tilde{A}$. Similarly, for a subset of variables $\mathcal{C}$ of ${\mathcal{V}}$, we will use $\tilde{\mathcal{C}}$ to denote the corresponding union of all sets where $A\in\mathcal{C}$. In other words,

[TABLE]

For a given function $h$, it is associated with a signed measure $\mu_{h}$ (or $\mu$ for short) such that for any $\mathcal{C}\subseteq{\mathcal{V}}$,

[TABLE]

Here, $\mu(\tilde{\mathcal{C}})$ is the signed measure for the set $\tilde{\mathcal{C}}$.

An atom is a set of the following form

[TABLE]

where $\mathcal{C}$ is a proper subset of ${\mathcal{V}}$. To simplify notation, will denote the atom defined in (5) as $[\mathcal{C}]$.

There are in total $2^{n}-1$ atoms. It has been proved in [5] that the signed measure for the atoms is uniquely determined from $h$ (and vice versa). In addition, there is also a one-to-one correspondence between Shannon’s information measures and a unique signed measure denoted $\mu$. Following the convention in [5],

[TABLE]

Further, define

[TABLE]

In other words, $T_{h}(\alpha)$ is the signed measure for the atom $[\alpha]$ induced by $\mu$ (or accordingly by the function $h$). It is easy to see that

[TABLE]

II-C Optimization under functional dependencies

Definition 1** (Functional dependency)**

A functional dependency is a binary tuple $(\mathcal{X},\mathcal{Y})$ where $\mathcal{X},\mathcal{Y}$ are disjoint subsets of $\mathcal{V}$. It means that the set of variables indexed by $\mathcal{X}$ are functionally imply those by $\mathcal{Y}$.

Further, a polymatroid $h$ (satisfying the basic inequalities) satisfies the functional dependency $(\mathcal{X},\mathcal{Y})$ if and only if

[TABLE]

Let $\Phi$ be a set of $L$ functional dependencies

[TABLE]

Consider the following optimization problem

[TABLE]

Note that $h\in\Gamma$ constrains $h$ to be a polymatroid and the test (8) applies.

In the context of proving Shannon-type constrained inequalities, $\Phi$ can be given a set of functional dependency constraints. In a network coding problem, $\Phi$ is the set of functional dependency constraints induced by the network topology and the multicast requirement.

Since all of these constraints are linear inequalities or equalities, (10) is a linear optimization problem if the objective function $f$ is also linear.

Proving whether a given inequality is basic is a redundancy check problem. More generally, a constrained information inequality $\mathbf{b}^{\textsf{T}}{h}\geq 0$ is redundant (subject to given functional dependencies $\Phi$) if the minimum value of the linear program

[TABLE]

is zero.

In general, the set of constraints in (10) can have a very large and complex structure. Therefore, it is desirable to simplify, or reduce these constraints prior to numerical solution. In this paper, we take a first step to achieve this goal by exploiting the functional dependency structure induced by $\Phi$.

Remark 1

Functional dependence relations naturally define equivalence classes on the set of joint entropies. These equivalence classes can be used as a basis for describing all constraints and thus reduce the dimension of the optimization problem. See Section V for further discussion.

III Main results

Definition 2** (Closure)**

Let $\mathcal{C}\subseteq{\mathcal{V}}$. Its closure ${\text{cl}({\mathcal{C}})}$ subject to a given set of functional dependencies $\Phi$ is the maximal set $\mathcal{D}$ such that

[TABLE]

for all polymatroids $h$ satisfying $\Phi$.

For any subset of random variables $\mathcal{C}$, its closure is the largest set of random variables that will be be functionally implied by $\mathcal{C}$, for every set of random variables satisfies the functional dependencies $\Phi$.

Definition 3** (Close Atoms)**

An atom $[\mathcal{A}]$ is called close with respect to a set of functional dependencies $\Phi=\{(\mathcal{X}_{\ell},\mathcal{Y}_{\ell})\}_{\ell=1}^{L}$, if $X_{\ell}\subseteq\mathcal{A}$ whenever $\mathcal{Y}_{\ell}\subseteq\mathcal{A}$, $\ell=1,2,\dots,L$.

Definition 4** (Vanishing atoms)**

An atom $[\mathcal{C}]$ is called vanishing with respect to the functional dependencies $\Phi$ if

[TABLE]

for all polymatroids $h$ satisfying $\Phi$.

Proposition 1** (Vanishing atoms)**

An atom $[\mathcal{A}]$ is vanishing subject to functional dependencies $\Phi$ if and only if $[\mathcal{A}]$ is not close subject to $\Phi$.

Proposition 2

A polymatroid $h$ satisfies all functional dependencies in $\Phi$ if and only if $T_{h}(\mathcal{C})=0$ for all vanishing (non-close) atoms.

In the absence of functional dependency constraints, the set of minimal Shannon inequalities was obtained in [5] as

[TABLE]

and

[TABLE]

where $A,B\in\mathcal{V}$ and $\mathcal{C}$ is a subset of $\mathcal{V}$.

Subject to further functional dependency constraints $\Phi$, some of these inequalities may become redundant. For example, it can be proved easily that if ${\text{cl}({\mathcal{C}})}={\text{cl}({\mathcal{D}})}$, then

[TABLE]

and hence $I(A;B|\mathcal{C})\geq 0\iff I(A;B|\mathcal{D})\geq 0$. In the following, we aim to identify such redundant inequalities.

Lemma 1

Let $\Phi$ be a given set of functional dependency constraints. If subject to $\Phi$,

[TABLE]

and

[TABLE]

then $I(A;B|\mathcal{C})=I(A^{\prime};B^{\prime}|\mathcal{C}^{\prime})$. Consequently,

[TABLE]

Lemma 1 illustrates that two inequalities, distinct in the absence of functional dependencies, can become equivalent when functional dependencies are introduced. This paper will identify all such redundant inequalities.

Definition 5** (Equivalence)**

Let $\mathcal{C}$ be close with respect to a given list of functional dependencies $\Phi$ and $A,B\not\in\mathcal{C}$. If $B\in{\text{cl}({A\mathcal{C}})}$ and $A\in{\text{cl}({B\mathcal{C}})}$, then we say $A\sim_{\mathcal{C}}B$.

It is easy to see that $\sim_{\mathcal{C}}$ is an equivalence relation on $\mathcal{V}\setminus\mathcal{C}$. The relation $A\sim_{\mathcal{C}}B$ means that $A$ and $B$ imply each other when conditioning on $\mathcal{C}$.

Definition 6** (Minimal atom)**

Let $\mathcal{C}$ be close with respect to a given list of functional dependencies $\Phi$. A variable $A\in\mathcal{V}\setminus\mathcal{C}$ is called $\mathcal{C}$-minimal if whenever there exists $B\in\mathcal{V}\setminus\mathcal{C}$ such that $B\in{\text{cl}({A\mathcal{C}})}$, then $A\in{\text{cl}({B\mathcal{C}})}$.

Proposition 3** (Reduction 1)**

Consider the inequality

[TABLE]

If $A$ is not $\mathcal{C}$-minimal with respect to functional dependencies $\Phi$, then (11) is redundant.

Proof:

If $A$ is not $\mathcal{C}$-minimal, then by definition there exists $D\not\sim_{\mathcal{C}}A$ such that

[TABLE]

In other words,

[TABLE]

for all polymatroids $h$ satisfying $\Phi$. Consequently,

[TABLE]

Thus, (11) is implied by the inequalities

[TABLE]

and

[TABLE]

∎

Corollary 1

Similarly, the inequality

[TABLE]

is redundant subject to functional dependencies $\Phi$ if $A$ is not $\mathcal{C}$-minimal with respect to $\Phi$.

Proposition 4** (Reduction 2)**

Let $A$ and $B$ be $\mathcal{C}$-minimal with respect to given functional dependencies $\Phi$, and suppose $A\sim_{\mathcal{C}}B$. Then

[TABLE]

is redundant if there exists $\mathcal{C}$-minimal $D$ such that $D\not\sim_{\mathcal{C}}A$.

Proof:

[TABLE]

The inequality $H(A|\mathcal{C},D)\geq 0$ is implied by $I(A;B|\mathcal{C},D)\geq 0$ or more precisely $I(A;B|{\text{cl}({\mathcal{C}D})})\geq 0$. ∎

The remaining inequalities of interest are of the form

[TABLE]

such that $\mathcal{C}$ is close, and $A$, $B$ are both $\mathcal{C}$-minimal.

Proposition 5** (Reduction 3)**

Let $A$ be $\mathcal{C}$ minimal with respect to given functional dependencies $\Phi$. If there exists $\mathcal{C}$-minimal $B\not\in\mathcal{C}$ such that $A\not\sim_{\mathcal{C}}B$ then

[TABLE]

is redundant.

Proof:

Notice that

[TABLE]

Hence, (16) is implied by $H(A|B\mathcal{C})\geq 0$ and $I(A;B|\mathcal{C})\geq 0$. ∎

In the above propositions, we have identified numerous redundant inequalities. The following theorem summarises above results by charactersing a minimal set of inequalities that characterise all polymatroids satisfying the functional dependencies $\Phi$.

Theorem 1** (Minimal characterization)**

A function $h$ is polymatroidal and satisfies all functional dependencies in $\Phi$ if and only if it satisfies every Type 1 and Type 2 inequality below:

Type 1:

[TABLE]

where $\mathcal{C}$ is close, $A\not\sim_{\mathcal{C}}B$ and $A,B$ are $\mathcal{C}$-minimal.

Type 2:

[TABLE]

where $\mathcal{C}$ is close and $A$ is $\mathcal{C}$-minimal such that $B\sim_{\mathcal{C}}A$ whenever $B$ is also $\mathcal{C}$-minimal.

Moreover, this set of inequalities are minimal, in the sense that each Type 1 and Type 2 inequality is non-redundant.

Again, we take the convention that two inequalities

[TABLE]

and

[TABLE]

are equivalent, if 1) ${\text{cl}({\mathcal{C}})}={\text{cl}({\mathcal{C}^{\prime}})}$, and 2) either $A\sim_{\mathcal{C}}A^{\prime}$ and $B\sim_{\mathcal{C}}B^{\prime}$, or $A\sim_{\mathcal{C}}B^{\prime}$ and $B\sim_{\mathcal{C}}A^{\prime}$.

Similarly for Type 2, inequalities

[TABLE]

and

[TABLE]

are deemed equivalent, if 1) ${\text{cl}({\mathcal{C}})}={\text{cl}({\mathcal{C}^{\prime}})}$, and 2) $A\sim_{\mathcal{C}}A^{\prime}$.

IV Proof of Theorem 1

Since the set of inequalities in Theorem 1 is obtained by eliminating all redundant inequalities, they certainly will still characterise all polymatroids satisfying the functional dependencies $\Phi$. In the following, we will prove that our obtained Type 1 and Type 2 inequalities are indeed minimal.

To prove the theorem, we will show that for each Type 1 or Type 2 inequality, we can construct a function $h$ that 1) violates the chosen inequality, 2) satisfies all remaining Type 1 and Type 2 inequalities, and 3) satisfies all functional dependencies.

IV-A Type 1 inequalities

Consider a Type 1 inequality of the form

[TABLE]

By definition, 1) $\mathcal{C}$ is close, 2) $A$ and $B$ are $\mathcal{C}$-minimal, and 3) $A\not\sim_{\mathcal{C}}B$.

To prove that the above Type 1 inequality is non-redundant, we will construct a function $h$ satisfying all the functional dependencies and all the polymatroidal inequalities except (20). Instead of directly defining $h$, we define its corresponding “atomic” function $T_{h}$ as follows:

[TABLE]

Note that, by definition,

[TABLE]

Now, we will show that (20) is indeed non-redundant.

First, we will show that the so constructed function $h$ (or equivalently its atomic version $T_{h}$) violates the inequality (20). Note that

[TABLE]

It can be verified directly that $I_{h}(A;B|\mathcal{C})=-1\leq 0$, and hence violating (20). Next, we will prove that function $h$ satisfies all other Types 1 and 2 inequalities, and also the functional dependencies.

From (21), $T_{h}(\beta)=0$ if $\beta$ is vanishing. Hence, the function $h$ satisfies all the functional dependencies. Now, let us consider a Type 1 inequality

[TABLE]

which is different from (20). Again, by definition, 1) ${\mathcal{K}}$ is close, 2) $i$ and $j$ are ${\mathcal{K}}$-minimal, and 3) $i\not\sim_{{\mathcal{K}}}j$. Notice that

[TABLE]

We will prove that $h$ satisfies (23) by considering different cases. In the first case, $\mathcal{C}$ does not contain ${\mathcal{K}}$ as a subset. In this case, the inequality (23) will not involve the atom $\mathcal{C}$, and hence will be satisfied by $h$.

Now, suppose that ${\mathcal{K}}$ is a proper subset of $\mathcal{C}$. In this second case, ${\mathcal{K}}\neq\mathcal{C}$. As ${\mathcal{K}}$ is close, it is non-vanishing. Therefore, $T_{h}({\mathcal{K}})=2$. Thus, $I_{h}(i;j|{\mathcal{K}})\geq 1$ and hence (23) is satisfied by $h$. It now remains to consider the third case when ${\mathcal{K}}=\mathcal{C}$.

If (23) is different from (20), then

[TABLE]

In addition, as $A\not\sim_{\mathcal{C}}B$ and $i\not\sim_{\mathcal{C}}j$, we may also assume without loss of generality that ${\text{cl}({i\mathcal{C}})}\neq{\text{cl}({A\mathcal{C}})}$ and ${\text{cl}({j\mathcal{C}})}\neq{\text{cl}({A\mathcal{C}})}$. In that case, ${\text{cl}({A\mathcal{C}})}\supseteq\mathcal{C}$ and $i,j\not\in{\text{cl}({A\mathcal{C}})}$. Therefore, the non-vanishing atom ${\text{cl}({A\mathcal{C}})}$ is involved in the inequality (23) and hence the inequality is satisfied by $h$.

So far, we have proved that $h$ satisfies all Type 1 inequalities except (20). Now, it remains to show that $h$ also satisfies all Type 2 inequalities.

Consider a Type 2 inequality

[TABLE]

By definition, ${\mathcal{K}}$ must be close such that $i$ is ${\mathcal{K}}$-minimal, and $i\sim_{\mathcal{K}}j$ whenever $j$ is also ${\mathcal{K}}$-minimal. Again, if ${\mathcal{K}}$ is not a subset of $\mathcal{C}$, then the inequality does not involve the atom $\mathcal{C}$. Hence, $H_{h}(i|{\mathcal{K}})$ is nonnegative. On the other hand, since both $A$ and $B$ are $\mathcal{C}$-minimal and $A\sim_{\mathcal{C}}B$, ${\mathcal{K}}$ is not equal to $\mathcal{C}$. So, it remains to consider the case where ${\mathcal{K}}$ is a proper subset of $\mathcal{C}$. By definition, $T_{h}({\mathcal{K}})=2$. Hence, it is obvious that from the definition that $H_{h}(i|{\mathcal{K}})\geq 1$. The function $h$ thus satisfies all Type 2 inequalities. And the non-redundancy of Type 1 inequalities have been proved.

IV-B Type 2 inequality

Next, we will prove that Type 2 inequalities are also non-redundant. Consider a Type 2 inequality of the form

[TABLE]

By definition, 1) $\mathcal{C}$ must be close and 2) $A$ is $\mathcal{C}$-minimal such that $A\sim_{\mathcal{C}}B$ whenever $B$ is also $\mathcal{C}$-minimal.

For this inequality, we define $h$ as follows:

[TABLE]

Again, $h$ can be directly obtained via (7).

Recall that

[TABLE]

From our construction, it is not difficult to see that $H_{h}(A|\mathcal{C})=-1$ and hence $h$ does not satisfy the inequality (25) but satisfies all functional dependencies. Now, it remains to prove that $h$ satisfies all other Type 1 and Type 2 inequalities.

First, consider a Type 1 inequality

[TABLE]

If ${\mathcal{K}}$ is not a subset of $\mathcal{C}$, then the inequality does not involve the atom $[\mathcal{C}]$. Hence, $I_{h}(i,j|{\mathcal{K}})$ is nonnegative. Also, as $\mathcal{C}\neq{\mathcal{K}}$, $T_{h}({\mathcal{K}})=2$ and hence $I_{h}(i,j|{\mathcal{K}})\geq 0$. Now, suppose ${\mathcal{K}}$ is a proper subset of $\mathcal{C}$. By definition, $T_{h}({\mathcal{K}})=2$. Hence, $I_{h}(i,j|{\mathcal{K}})\geq 0$. The function $h$ thus satisfies all Type 1 inequalities.

Next, we consider a Type 2 inequality

[TABLE]

If ${\mathcal{K}}=\mathcal{C}$, then $i\sim_{\mathcal{K}}A$ and thus (29) and (25) are the same inequality. Suppose ${\mathcal{K}}\neq\mathcal{C}$. If ${\mathcal{K}}$ is a proper subset of $\mathcal{C}$, then $T_{h}({\mathcal{K}})=2$. And if ${\mathcal{K}}$ is not a subset of $\mathcal{C}$, then $T_{h}(\mathcal{C})$ is not involved in the inequality (29). In any cases, this implies that $h$ will satisfy (29). Non-redundancy of (25) and also Theorem 1 is thus proved.

V Applications and Future work

It is desirable to obtain directly the reduced matrix representing the functions $h$ in the constraint region of (10). In [6] (see also [7], [8]), we gave a graph based recursive algorithms to find implied functional dependencies from local functional dependencies. Though the graph based algorithm does not always give all implied functional dependence relations, it gives many functional dependencies depending on the structure of local functional dependencies without depending on the linear programming framework. In [9] (see also [8]), we gave algorithms to directly obtain a reduced size matrix defining the constraint region. Despite the fact that the reduction was not minimal, it was demonstrated that for the well known butterfly network, the matrix size can be reduced by $98\%$. The number of variables, $2^{n}-1$, in the optimization problems can be reduced to the number of equivalence classes for a given functional dependency structure. Given the functional dependence structure, the* minimal* set of inequalities defining the constraint region can also be obtained in a matrix form directly using the approach similar to [9]. Applications of this matrix include solving the optimization problem (10).

As a continuation of research work in this direction, we aim to employ the results of this paper to develop more refined algorithms (compared to [9]), for obtaining the “minimal” matrix directly. Moreover, we are investigating further generalizations of the ideas presented in this paper.

Acknowledgment

This work is supported in part by Science and Engineering Research Board, Department of Science and Technology, Government of India, under project SB/S3/EECE/265/2016. It is also supported in part by Australian Research Council under Discovery Project DP150103658.