Parsing with CYK over Distributed Representations

TL;DR

This paper introduces D-CYK, a neural network-compatible version of the CYK parsing algorithm that operates over distributed representations, bridging symbolic parsing and neural network approaches.

Contribution

The paper presents D-CYK, a novel formulation of the CYK algorithm that functions entirely over distributed representations using matrix operations, enabling neural network integration.

Findings

D-CYK approximates the original CYK algorithm.

Matrix operations used are compatible with neural networks.

Demonstrates symbolic parsing can be implemented with neural-compatible methods.

Abstract

Syntactic parsing is a key task in natural language processing. This task has been dominated by symbolic, grammar-based parsers. Neural networks, with their distributed representations, are challenging these methods. In this article we show that existing symbolic parsing algorithms can cross the border and be entirely formulated over distributed representations. To this end we introduce a version of the traditional Cocke-Younger-Kasami (CYK) algorithm, called D-CYK, which is entirely defined over distributed representations. Our D-CYK uses matrix multiplication on real number matrices of size independent of the length of the input string. These operations are compatible with traditional neural networks. Experiments show that our D-CYK approximates the original CYK algorithm. By showing that CYK can be entirely performed on distributed representations, we open the way to the definition of recurrent layers of CYK-informed neural networks.

Equations

$${\vec{a}}^{\mathrm{T}}\vec{b}$$

$$[a{]}^{\oplus }$$

$$[a{]}^{\ominus }$$

$$[a{]}^{\oplus }[b{]}^{\ominus }$$

$$\text{textsc}{A}_{\circ }^{\mathrm{T}}\text{textsc}{B}_{\circ }$$

$$B$$

$$L^{\prime }$$

$${\mathbf{P}}_{left}[i,j,X]$$

$${\mathbf{P}}_{right}[i,j,X]$$

$${\mathbf{P}}_{left}$$

$$\mathbf{P}_{\mathit{left}}=\framebox{{\centering\begin{tabular}[]{ccc}\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}&\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}&\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny b}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\end{tabular} \@add@centering}}$$

$${\mathbf{P}}_{left}$$

$${\mathbf{P}}_{right}$$

$${\mathbf{R}}_u^{(A)}$$

$$P_A$$

$${\mathbf{R}}_u^{(A)}[a_i{]}^{\ominus }$$

$${\mathbf{P}}_{left}$$

$${\mathbf{P}}_{right}$$

$$\mathbf{R}_{{u}}^{({D})}=\framebox{ \begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\hskip 14.22636pt\mathbf{R}_{{u}}^{({E})}=\framebox{ \begin{tabular}[]{c}{\tiny b}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}$$

$$\mathbf{P}_{\mathit{left}}=\framebox{ \begin{tabular}[]{ccc}\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}&\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}&\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny b}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\end{tabular} }$$

$${\mathbf{P}}_{left}$$

$$Update$$

$$\displaystyle\approx\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\sigma(\framebox{\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{})\approx\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\sigma(\framebox{\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}}\begin{tabular}[]{c}{\tiny a}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{})\approx\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}{}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}$$

$${\mathbf{P}}_{left}$$

$${\mathbf{P}}_{right}$$

$${\mathbf{r}}_{A\to BC}$$

$${\mathbf{R}}_b^{(A)}$$

$$P_A=\sigma ([j{]}^{\oplus }[i{]}^{\ominus }{\text{P}}_{left}{\mathbf{R}}_b^{(A)}{\text{P}}_{right})\otimes \mathbf{I}.$$

$$\mathbf{R}_{{b}}^{({S})}=\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}$$

$$P_{A}=\sigma(\framebox{\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\textbf{P}_{\mathit{left}}\;\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\textbf{P}_{\mathit{right}})\otimes\mathbf{I}$$

$$\framebox{\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\textbf{P}_{\mathit{left}}\;\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\textbf{P}_{\mathit{right}}=$$

$$\framebox{\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\framebox{\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt $\ldots$}\;\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt $\ldots$}\;\approx$$

$$\framebox{\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\framebox{\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny S}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}}\;\framebox{\begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 0}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{.5,0,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny D}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0.5,0.5,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 1}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{1,0,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt \begin{tabular}[]{c}{\tiny E}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,1,0}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 2}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,.5,.5}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(1.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\begin{tabular}[]{c}{\tiny 3}\\ \parbox[c][10.0pt][c]{8.0pt}{{\color[rgb]{0,0,1}{ \leavevmode\resizebox{20.0pt}{20.0pt}{\vbox to15.0pt{\vfil\hbox to25.00003pt{\framebox{{\begin{picture}(3.0,5.0)(0.0,-5.0)\put(1.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-1.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-2.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(2.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-3.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-4.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\put(3.0,-5.0){\kern-2.0pt\rule{2.0pt}{2.0pt}}\end{picture}}}}\vfil}}}}}\end{tabular}\hskip 2.84544pt $\ldots$}\;\approx$$

Full text

1 Introduction

The area of natural language parsing has been dominated for decades by the so-called symbolic paradigm of artificial intelligence, which embraces the collection of methods that are based on high-level, human-readable, symbolic representations. The Cocke-Younger-Kasami algorithm (CYK) Cocke (1969); Younger (1967); Kasami (1965), the Early algorithm Earley (1970) and the shift-reduce algorithm Sippu and Soisalon-Soininen (1988) are at the core of most common algorithms for natural language parsing, both constituency-based and dependency-based, and they all use symbolic representations for grammar rules, parser states and syntactic trees.

Starting with the early 90’s and the surge of data-driven methods in natural language processing, grammar rules and their weights have been estimated from large data sets of syntactically annotated sentences, and probabilistic parsers and parsers based on discriminative models have flourished. Nonetheless, all these methods are based on the above mentioned parsing algorithms, with grammar rules and parser states still symbolically represented. With the recent development of deep learning techniques, fast and accurate dependency parsers have been designed on top of the shift-reduce algorithm, where parser actions are parameterized using neural networks Chen and Manning (2014); Ambati, Deoskar, and Steedman (2016). Furthermore, the use of recurrent neural networks has made it possible to design models that condition parsing actions on the entire syntactic derivation history Titov and Henderson (2007); Dyer et al. (2016), resulting in considerable enhancement of parsing performance. Still, for all of these parsers based on neural networks, the underlying configuration of the parser is a stack of symbols and the search space is represented at that level.

The main goal in the above mentioned research is to inform parsing decisions using contextual information as rich as possible. This means that the context-free assumption of the underlying formalism is dropped. In case of the neural network parsers, this is achieved using distributed representations and matrix multiplication operations embedded into the underlying network architecture. In this article we explore a somewhat tangential direction: we attempt to entirely remove the symbolic level of representation from the underlying parser. There are at least two existing lines of research in this direction. In a first line Vinyals et al. (2014) parsing is seen as a sequence-to-sequence translation task, with the input sentence mapped to a linear syntactic interpretation, and training is done with long-short term memories (LSTM) networks over millions of sentences which have been annotated by existing parsers. In this case it seems that the LSTM network learns two things: the associations among fragments of sentences and fragments of trees and a way of recombining these fragments in the final interpretation. The grammar representation is then hidden in the weights of the LSTM. In the second line of research Zanzotto and Dell’Arciprete (2013); Senay et al. (2015) both sentences and trees are represented in distributed vectors and neutral networks learn a way to map sentence vectors to tree vectors. However, the overall model does not have the ability to replicate distributed vectors for trees from distributed vectors for sentences. Resulting vectors are not accurate enough to have an impact on final tasks Zanzotto and Dell’Arciprete (2013). Moreover, in this case as well the grammar is represented in an unpredictable way in the weights of the multi-layer perceptron or the LSTM.

In this article we take a rather different route, and show that traditional parsing algorithms can cross the border of distributed representations. We propose a version of the CYK algorithm, called D-CYK, that entirely works on distributed representations. This is achieved by transforming the parsing table at the base of the CYK algorithm into a real number square matrix, and by implementing the basic operations of the algorithm using matrix multiplication. Implementations of the CYK algorithm using matrix multiplications are well known in the literature Graham and Harrison (1976); Valiant (1975) but they use symbolic representations, while in our proposal grammar symbols as well as constituent indices are all encoded into real numbers and in a distributed way. A second, important difference is that in the standard CYK algorithm the parsing table has size $(n+1)\times(n+1)$, with $n$ the length of the input string, while in our D-CYK the parsing table has size $d\times d$, where $d$ depends on the distributed representation. This means that, up to some extent, we can parse input sentences of different lengths without changing the size of the parsing table in the D-CYK algorithm. We are not aware of any parsing algorithm for context-free grammars having such property.

We report experiments on toy grammars showing that our D-CYK can successfully approximate the CYK algorithm. This novel approach opens the way to the investigation of parsing algorithms where the symbolic level of representation is entirely dropped.

2 Preliminaries

In this section we introduce the basics about the CYK algorithm and overview a class of distributed representation called holographic reduced representation.

2.1 CYK algorithm

The CYK algorithm is a classical algorithm for parsing based on context-free grammars, using dynamic programming. We provide here a brief description of the algorithm in order to introduce the notation used in later sections; we closely follow the presentation in Graham and Harrison (1976). The algorithm requires context-free grammars in Chomsky Normal Form (CNF), where each rule has the form $A\rightarrow BC$ or $A\rightarrow a$, where $A$, $B$, $C$ are nonterminal symbols and $a$ is a terminal symbol. We write $R$ to denote the set of all rules of the grammar.

Given an input string $w=a_{1}\cdots a_{n}$, $n\geq 1$, where each $a_{i}$ is an alphabet symbol, the algorithm uses a 2-dimensional table $P$ of size $(n+1)\times(n+1)$, where each entry stores a set of nonterminals representing partial parses of the input string. More precisely, for $0\leq i<j\leq n$, a nonterminal $A$ belongs to set $P[i,j]$ if and only if there exists a parse tree with root $A$ generating the substring $a_{i+1}\cdots a_{j}$ of $w$. Thus, $w$ can be parsed if the initial nonterminal of the grammar $S$ is added to $P[0,n]$. Algorithm 1 shows how table $P$ is populated. $P$ is first initialized using unary rules, at line 3. Then each entry $P[i,j]$ is filled at line 9 by looking at pairs $P[i,k]$ and $P[k,j]$ and by using binary rules.

A running example is presented in Figure 1, showing a set $R$ of grammar rules along with the table $P$ produced by the algorithm when processing the input string $w=aab$. For instance, $S$ is added to $P[1,3]$, since $D\in P[1,2]$, $E\in P[2,3]$, and $(S\rightarrow DE)\in R$. Since $S\in P[0,3]$, we conclude that $w$ can be parsed by the grammar.

2.2 Distributed Representations with Holographic Reduced Representations

Holographic reduced representations (HRR; Plate 1995) are distributed representations well-suited for our aim of encoding the 2-dimensional parsing table $P$ of the CYK algorithm and for implementing the operation of selecting the content of its cells $P[i,j]$.

In the following, we introduce the operations we use, along with a graphical way to represent their properties. The graphical representation is based on Tetris-like pieces.

The starting point of a distributed representation is how to encode symbols into vectors: symbol $a$ can be encoded using random vector $\vec{a}\in\mathbb{R}^{d}$ drawn from a multivariate normal distribution $\vec{a}\sim N(0,\mathbf{I}\frac{1}{\sqrt{d}})$. These vectors are used as basis vectors for the Johnson-Lindenstrauss Tranform Johnson and Lindenstrauss (1984) as well as for random indexing Sahlgren (2005). The major property of these random vectors is the following

[TABLE]

Given the above representation of symbols, we can define a basic operation $[\>]^{\oplus}$ and its approximate inverse $[\>]^{\ominus}$. These operations take as input a symbol and provide a matrix in $\mathbb{R}^{d\times d}$, and are the basis for our encoding and decoding. The first operation is defined as

[TABLE]

where ${\textsc{A}_{\circ}}$ is the circulant matrix of the vector $\vec{a}$ and $\Phi$ is a permutation matrix. This operation has a nice approximated inverse in

[TABLE]

We then have

[TABLE]

since $\Phi$ is a permutation matrix and therefore $\Phi\Phi^{\rm{T}}=I$, and since

[TABLE]

due to the fact that ${\textsc{A}_{\circ}}$ and ${\textsc{B}_{\circ}}$ are circulant matrices based on random vectors $\vec{a},\vec{b}\sim N(0,\mathbf{I}\frac{1}{\sqrt{d}})$.

With the $[\;]^{\oplus}$ and $[\;]^{\ominus}$ operations at hands, we can now encode and decode strings, that is, finite sequences of symbols. As an example, the string $abc$ can be represented as the matrix product $[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$. In fact, we can check that $[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$ starts with $a$ but not with $b$ or with $c$, since we have $[a]^{\ominus}[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}\approx[b]^{\oplus}[c]^{\oplus}$, which is different from $\mathbf{0}$, while $[b]^{\ominus}[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}\approx\mathbf{0}$ and $[c]^{\ominus}[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}\approx\mathbf{0}$. Knowing that $[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$ starts with $a$, we can also check that the second symbol in $[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$ is $b$, since $[b]^{\ominus}[a]^{\ominus}[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$ is different from $\mathbf{0}$. Finally, knowing that $[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}$ starts with $ab$, we can check that the string ends in $c$, since $[c]^{\ominus}[b]^{\ominus}[a]^{\ominus}[a]^{\oplus}[b]^{\oplus}[c]^{\oplus}\approx\mathbf{I}$.

Using the above operations, we can also encode sets of strings. For instance, the string set ${\cal S}=\{abS,DSa\}$ is represented as the sum of matrix products $[a]^{\oplus}[b]^{\oplus}[S]^{\oplus}+[D]^{\oplus}[S]^{\oplus}[a]^{\oplus}$. We can then test whether $abS\in{\cal S}$ by computing the matrix product $[S]^{\ominus}[b]^{\ominus}[a]^{\ominus}([a]^{\oplus}[b]^{\oplus}[S]^{\oplus}+[D]^{\oplus}[S]^{\oplus}[a]^{\oplus})\approx\mathbf{I}$, meaning that the answer is positive. Similarly, $aDS\in{\cal S}$ is false, since $[S]^{\ominus}[D]^{\ominus}[a]^{\ominus}([a]^{\oplus}[b]^{\oplus}[S]^{\oplus}+[D]^{\oplus}[S]^{\oplus}[a]^{\oplus})\approx\mathbf{0}$. We can also test whether there is any string in ${\cal S}$ starting with $a$, by computing $[a]^{\ominus}([a]^{\oplus}[b]^{\oplus}[S]^{\oplus}+[D]^{\oplus}[S]^{\oplus}[a]^{\oplus})\approx[b]^{\oplus}[S]^{\oplus}$ and providing a positive answer since the result is different from $\mathbf{0}$.

Not only our operations above can be used to encode sets, as described above, they can also be used to encode multi-sets, that is, they can keep a count of the numer of occurrences of a given symbol/string within a collection. For instance, consider a multi-set with two occurrences of symbol $a$. This can be encoded by means of the sum $[a]^{\oplus}+[a]^{\oplus}$. In fact, we can test the number of occurrences of symbol $a$ in the multi-set using the product $[a]^{\ominus}([a]^{\oplus}+[a]^{\oplus})=\mathbf{I}+\mathbf{I}=2\mathbf{I}$.

To visualize the encoding and decoding ability of the above operations, we will use a graphical representation based on Tetris. Symbols under the above operations are represented as Tetris pieces: for example, $[a]^{\oplus}$ =

a

 , $[a]^{\ominus}$ =

a

 , $[b]^{\oplus}$ =

b

and $[b]^{\ominus}$ =

b

. In this way strings are sequences of pieces; for example,

a

b

S

encodes $abS$ (equivalently, $[a]^{\oplus}[b]^{\oplus}[S]^{\oplus}$). Then, like in Tetris, elements with complementary shapes are canceled out and removed from a sequence; for example, if

a

is applied to the left of

a

b

S

, the result is

b

S

as

a

a

disappears. Sets of strings (sums of matrix products) are represented in boxes, as for instance

[TABLE]

which encodes set $\{abS,DSa\}$ (equivalently, $[a]^{\oplus}[b]^{\oplus}[S]^{\oplus}+[D]^{\oplus}[S]^{\oplus}[a]^{\oplus}$). In addition to the usual Tetris rules, an element with a certain shape will select from a box only elements with the complementary shape. For instance, if

a

is applied to the left of the above box $B$, the result is the new box

[TABLE]

as

a

selects

a

b

S

but not

D

S

a

.

With these operations and with this Tetris metaphor, we can describe our model to encode $P$ tables in matrices and to implement rule applications by means of matrix multiplication, as discussed in the next section.

3 The CYK algorithm on Distributed Representations

The distributed CYK algorithm (D-CYK) is our version of the CYK algorithm running over distributed representations and using matrix algebra. As the traditional CYK, this algorithm recognizes whether or not a string $w$ can be generated by a context-free grammar with a set of rules $R$ in Chomsky Normal Form. Yet, unlike the traditional CYK algorithm, the parsing table $P$ and the rule set $R$ are encoded through matrices in $\mathbb{R}^{d\times d}$, using the distributed representation of Section 2.2, and rule application is obtained with matrix algebra.

In this section we describe how the D-CYK algorithm encodes: (i) the table $P$ by means of two matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$; (ii) the unary rules in $R$ by means of a matrix $\mathbf{R}_{{u}}^{({A})}$ for each nonterminal symbol $A$ in the grammar; and, (iii) binary rules in $R$ by means of matrices $\mathbf{R}_{{b}}^{({A})}$ for each nonterminal $A$. We then specify the steps of the D-CYK algorithm and illustrate its execution using the running example of Figure 1.

3.1 Encoding the Table $P$ in matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$

The table $P$ of the CYK algorithm can be seen as a collection of triples $(i,j,X)$. More precisely, the collection of triples contains element $(i,j,X)$ if and only if $X\in P[i,j]$. Given the representation of Section 2.2, the table $P$ is encoded by means of two matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$ in $\mathbb{R}^{d\times d}$, each containing the collection of triples $(i,j,X)$ in distributed representation. More precisely, each triple $(i,j,X)$ is encoded as:

[TABLE]

Then matrix $\mathbf{P}_{\mathit{left}}$ is the sum of all elements $\mathbf{P}_{\mathit{left}}[i,j,X]$, encoding the collection of all triples from $P$. Similarly, $\mathbf{P}_{\mathit{right}}$ is the sum of all elements $\mathbf{P}_{\mathit{right}}[i,j,X]$. To visualize this representation, the matrix $\mathbf{P}_{\mathit{left}}$ of our running example is represented in the Tetris-like notation in Figure 1, where we have used the pieces in Figure 2.

3.2 Encoding and Using Unary Rules

The CYK algorithm uses symbols $a_{i}$ from the input string $w$ and unary rules to fill in cells $P[i-1,i]$, as seen in Algorithm 1. We simulate this step in our D-CYK using our distributed representation and matrix operations.

D-CYK represents the input $a_{i}$ using the matrix $\mathbf{P}_{\mathit{left}}$. Hence, before the application of unary rules we have

[TABLE]

For processing unary rules we use the first part of the D-CYK algorithm, called D-CYK_unary and reported in Algorithm 2. D-CYK_unary takes $\mathbf{P}_{\mathit{left}}$, encoding symbols from the input string, and produces updated matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$ encoding nonterminal symbols resulting from the application of unary rules.

For the running example of Figure 1, the initial content of $\mathbf{P}_{\mathit{left}}$ is

[TABLE]

Taking as input the above matrix, D-CYK_unary produces the two matrices

[TABLE]

In D-CYK_unary, we use matrices $\mathbf{R}_{{u}}^{({A})}$, for each nonterminal $A$ in the left-hand side of some unary rule. These matrices are conceived to detect the applicability of rules of the form $A\rightarrow a$, where $a$ is some alphabet symbol, to matrix $\mathbf{P}_{\mathit{left}}$, and are also used to update matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$. Matrix $\mathbf{R}_{{u}}^{({A})}$ is defined as

[TABLE]

where $R$ is the set of rules of the grammar. The operation between $\mathbf{R}_{{u}}^{({A})}$ and $\mathbf{P}_{\mathit{left}}$ (line 4 in Algorithm 2), which detects whether some rule $A\rightarrow a$ is applicable at position $(i-1,i)$ of the input string, is

[TABLE]

where $\sigma(x)$ is a sigmoid function $\sigma(x)=\frac{1}{1+e^{-(x-0.5)*\beta}}$. In fact, $[i]^{\oplus}[i-1]^{\oplus}\mathbf{P}_{\mathit{left}}\approx[a_{i}]^{\ominus}$ extracts the distributed representation of terminal symbol $a_{i}$. Then

[TABLE]

is reinforced by the subsequent use of the sigmoid function. Hence, if some unary rule with left-hand side symbol $A$ is applicable, the resulting matrix is approximately the identity matrix $\mathbf{I}$, else the resulting matrix is approximately the zero matrix $\mathbf{0}$. Then, the operations in lines 5 and 6:

[TABLE]

add a non-zero matrix to $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$, respectively, only if rules for $A$ are matched in $\mathbf{P}_{\mathit{left}}$ containing the input sentence.

We describe the application of D-CYK_unary using the running example in Figure 1 and the Tetris-like representation. The two unary rules $D\rightarrow a$ and $E\rightarrow b$ are represented as $\mathbf{R}_{{u}}^{({D})}=[a]^{\oplus}$ and $\mathbf{R}_{{u}}^{({E})}=[b]^{\oplus}$, that is

[TABLE]

in the Tetris-like form. As already seen, given the input sequence $aab$ the matrix $\mathbf{P}_{\mathit{left}}$ is initialized as

[TABLE]

We focus on the application of rule $D\rightarrow a$ to cell $P[0,1]$ of the parsing table, represented through matrices $\mathbf{R}_{{u}}^{({D})}$ and $\mathbf{P}_{\mathit{left}}$, respectively. At steps 4 and 5 of Algorithm 2, taken together, we have

[TABLE]

The $\mathit{Update}$ part of the assignment can be expressed as

[TABLE]

This results in the insertion of the distributed representation of element $(0,1,D)$ in $\mathbf{P}_{\mathit{left}}$. A symmetrical operation is carried out to update $\mathbf{P}_{\mathit{right}}$. After the application of matrices $\mathbf{R}_{{u}}^{({D})}$ and $\mathbf{R}_{{u}}^{({E})}$ at each cell $P[i-1,i]$, the matrices $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$ provide the following values

[TABLE]

3.3 Encoding and Using Binary Rules

To complete the specification of algorithm D-CYK, we describe here how to encode binary rules in such a way that these rules can be fire over the distributed representation of table $P$ through operations in our matrix algebra. We introduce the second part of the algorithm, called D-CYK_binary and specified in Algorithm 3, and we clarify why both $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$, introduced in section 3.3, are needed.

Binary rules in $R$ with nonterminal symbol $A$ in the left-hand side are all encoded in a matrix $\mathbf{R}_{{b}}^{({A})}$ in $\mathbb{R}^{d\times d}$. $\mathbf{R}_{{b}}^{({A})}$ is conceived for defining matrix operations that detect if rules of the form $A\rightarrow BC$, for some $B$ and $C$, fire in position $(i,j)$, given $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$. This operations result in a nearly identity matrix $\mathbf{I}$ for a specific position $(i,j)$ if at least one specific rule coded in $\mathbf{R}_{{b}}^{({A})}$ fires in position $(i,j)$ over positions $(i,k)$ and $(k,j)$, for any value of $k$. This will enable the insertion of new symbols in $\mathbf{P}_{\mathit{left}}$ and $\mathbf{P}_{\mathit{right}}$.

To define $\mathbf{R}_{{b}}^{({A})}$ we encode the right-hand side of each binary rule $A\rightarrow BC$ in $R$ as

[TABLE]

All the right-hand sides of binary rules with symbol $A$ in the left-hand side are then collected in matrix $\mathbf{R}_{{b}}^{({A})}$

[TABLE]

Algorithm 3 uses these rules to determine whether a symbol $A$ can fire in a position $(i,j)$ for any $k$. The key part is line 4 of Algorithm 3

[TABLE]

$\mathbf{R}_{{b}}^{({A})}$ selects elements in $\textbf{P}_{\mathit{left}}$ and $\textbf{P}_{\mathit{right}}$ according to rules for $A$. Matrices $\textbf{P}_{\mathit{left}}$ and $\textbf{P}_{\mathit{right}}$ have been designed in such a way that, after the nonterminal symbols in the selected elements have been annihilated, the associated spans $(i,k)$ and $(k,j)$ merge into span $(i,j)$. Finally, the terms $[j]^{\oplus}[i]^{\ominus}$ are meant to check whether the span $(i,j)$ has survived. If this is the case, the resulting matrix will be very close to the identity matrix $\mathbf{I}$, otherwise it will be very close to the null matrix $\mathbf{0}$. To reinforce this matrix, similarly to Algorithm 2, we use the sigmoid function $\sigma(x)$. Finally, we apply an element-wise multiplication with $\mathbf{I}$, which is helpful to remove noise.

To visualize the behavior of the algorithm D-CYK_binary and the effect of using $\mathbf{R}_{{b}}^{({A})}$, we use again the running example of Figure 1. The only nonterminal with binary rules is $S$, and matrix $\mathbf{R}_{{b}}^{({S})}$ is:

[TABLE]

Let’s focus on the position $(1,3)$. The operation is the following:

[TABLE]

where $\textbf{P}_{\mathit{left}}$ and $\textbf{P}_{\mathit{right}}$ are as in equations 45 and 83. We can then write

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Then, this operation detects that $S$ is active in the position $(1,3)$.

With this second part of the algorithm, CYK has been reduced to D-CYK, which works with distributed representations and matrix operations compliant with neural networks.

4 Experiments

The aim of these experiments is to show that the distributed CYK algorithm behaves as the original CYK algorithm. For this purpose, we do not need huge datasets but small well-defined set of sentences derived from fixed grammars as defined in the following sections.

4.1 Experimental Set-up

We experimented with five different grammars with increasing sets of rules and a set of 2,000 sentences. The grammars are: $G_{0}$ is the basic grammar with $8$ rules with $3$ unary rules; $G_{1}$ with $25$ rules expands only unary rules over $G_{0}$; $G_{2}$ with $28$ rules expands only binary rules over $G_{1}$; $G_{3}$ with $34$ rules expands only binary rules over $G_{1}$; and, finally, $G_{4}$ with $41$ rules expands only binary rules over $G_{1}$. The set of sentences has been produced by randomly generated 2,000 sentences of different lengths by using the grammar $G_{0}$. These sentences can be recognized by all the 5 grammars as all the other grammars are obtained by adding rules to $G_{0}$.

As we want to understand whether D-CYK is able to reproduce the computation of the original CYK, we used cell f1-measure ($f1$), which evaluates whether the distributed P is similar to the original P. To evaluate $f1$, we decoded the distributed versions $\mathbf{P}_{\mathit{left}}$ of $P$ by using a simple decoding algorithm $Dec$ (Algorithm 4).

We compared $Dec(\mathbf{P}_{\mathit{left}})$ and the matrix $P$ obtained by applying the traditional CYK algorithm with the corresponding grammar on test sentences. By comparing $Dec(\mathbf{P}_{\mathit{left}})$ and the corresponding $P$, we evaluated the cell precision and the cell recall by considering $P$ the oracle and $Dec(\mathbf{P}_{\mathit{left}})$ the system. $f1$ is then computed according to the traditional equation of mixing precision and recall.

We experimented with six different dimensions of matrices $\mathbf{P}$: 100, 1000, 2000, 3000, 4000, 5000 and 6000.

4.2 Results

Results are really encouraging showing that, as the dimensions of the matrices increase, D-CYK can approximate with its operations what is done by the traditional CYK. The f1-measure is in fact increasing with the dimension of the matrix. This is mainly due to an improvement of the cell symbol precision as the cell symbol recall is substantially stable. Hence, as the dimension increases D-CYK gets more precise in replicating the original matrix.

The size of the grammar is instead a major problem. In fact, the precision of the algorithm is affected by the number of rules whereas the recall is substantially similar across the three different grammars.

These results confirm that it is possible to transfer a traditional algorithm on a version, which is defined on distributed representations.

5 Conclusions and Future Work

In these days, the predominance of symbolic, grammar-based syntactic parsers for natural language has been successfully challenged by neural networks, which are based on distributed representations. Years of results and understanding can be lost. We proposed D-CYK that is a distributed version of the CYK, a classical parsing algorithm. Experiments show that D-CYK can do the same task of the original CYK in this new setting.

Neural networks are a tremendous opportunity to develop novel solutions for known tasks. Our solution opens an avenue to an innovative set of possibilities: revitalizing symbolic methods in neural networks. In fact, our algorithm is the first step towards the definition of a “complete distributed CYK algorithm” that builds trees in distributed representations during the computation. Moreover, it can foster the definition of recurrent layers of CYK-informed neural networks.

\starttwocolumn