Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing---and Back

TL;DR

Pseudo-task augmentation enhances single-task deep learning by simulating related tasks through multiple decoders, leading to improved performance and state-of-the-art results when combined with multitask learning.

Contribution

The paper introduces pseudo-task augmentation, a novel method that applies multitask learning ideas to single-task settings using multiple decoders, boosting performance.

Findings

Improves single-task learning performance.

Achieves state-of-the-art on CelebA dataset.

Complementary to traditional multitask learning.

Abstract

Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task. Pseudo-tasks simulate the effect of training towards closely-related tasks drawn from the same universe. In a suite of experiments, pseudo-task augmentation is shown to improve performance on single-task learning problems. When combined with multitask learning, further improvements are achieved, including state-of-the-art performance on the CelebA dataset, showing that pseudo-task augmentation and multitask learning have complementary value. All in all, pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems.

Tables

Table 1. Table 1: Omniglot 50-task results. Test error averaged across all tasks for each setup is shown. Overall, the performance gains from MTL complement those from PTA, with PTA-F again the highest-performing and most robust method.

Method	Single-task Learning		Multitask Learning
Baseline	35.49		29.02
	$D=4$	$D=10$	$D=4$	$D=10$
PTA-I	31.72	32.56	27.26	24.50
PTA-HGD	31.63	30.39	25.77	26.55
PTA-F	29.37	28.48	23.45	23.36
PTA-Mean	30.91	30.48	25.49	24.80

Table 2. Table 2: IMDB Results. All PTA methods outperform the LSTM baseline. The best performance is achieved by PTA-HGD with D = 10 𝐷 10 D=10 . This method receives a substantial boost from increasing the number of decoders from 4 to 10, as the greedy algorithm gets to perform broader search. On the other hand, PTA-I and PTA-F do not improve with the additional decoders, suggesting that, without careful control, too many decoders can overconstrain ℱ ℱ \mathcal{F} .

Method	Test Accuracy %
LSTM Baseline ($D=1$)	82.75 ($\pm 0.13$)
	$D=4$	$D=10$
PTA-I	83.20 ($\pm 0.07$)	83.02 ($\pm 0.11$)
PTA-HGD	83.22 ($\pm 0.05$)	83.51 ($\pm 0.08$)
PTA-F	83.30 ($\pm 0.12$)	83.30 ($\pm 0.08$)

Table 3. Table 3: CelebA results. Comparison of PTA against state-of-the-art methods for CelebA, with and without ensembling. Test error is averaged across all attributes. PTA-HGD outperforms all other methods, establishing a new state-of-the-art in this benchmark.

MTL Method	% Error
Single Task (He et al., 2017)	10.37
MOON (Rudd et al., 2016)	9.06
Adaptive Sharing (Lu et al., 2017)	8.74
MCNN-AUX (Hand & Chellappa, 2017)	8.71
Soft Order (Meyerson & Miikkulainen, 2018)	8.64
VGG-16 MTL (Lu et al., 2017)	8.56
Adaptive Weighting (He et al., 2017)	8.20
AFFACT (Günther et al., 2017) (best of 3)	8.16
MTL Baseline (Ours; mean of 3)	8.14
PTA-HGD, $D=10$ (mean of 3)	8.10
Ensemble of 3: AFFACT (Günther et al., 2017)	8.00
Ensemble of 3: PTA-HGD, $D=10$	7.94

Equations

$${\hat{\mathbf{y}}}_{ti}={\mathcal{D}}_t(\mathcal{F}({\mathbf{x}}_{ti};{\theta }_{\mathcal{F}});{\theta }_{{\mathcal{D}}_t}).$$

$${\theta }^{\ast }=\underset{\theta }{\mathrm{argmin}}\frac{1}{T}\sum _{t=1}^T\frac{1}{N_t}\sum _{i=1}^{N_t}\mathcal{L}({\mathbf{y}}_{ti},{\hat{\mathbf{y}}}_{ti})$$

$${\hat{\mathbf{y}}}_{tdi}={\mathcal{D}}_{td}(\mathcal{F}({\mathbf{x}}_{ti};{\theta }_{\mathcal{F}});{\theta }_{{\mathcal{D}}_{td}}),$$

$${\theta }^{\ast }=\underset{\theta }{\mathrm{argmin}}\frac{1}{TD}\sum _{t=1}^T\frac{1}{N_t}\sum _{i=1}^{N_t}\sum _{d=1}^D\mathcal{L}({\mathbf{y}}_{ti},{\hat{\mathbf{y}}}_{tdi}),$$

$${\theta }_{\text{eval}}^{\ast }=\underset{\theta }{\mathrm{argmin}}\frac{1}{T}\sum _{t=1}^T\frac{1}{N_t}\underset{d\in \mathrm{1..}D}{\mathrm{argmin}}\sum _{i=1}^{N_t}\mathcal{L}({\mathbf{y}}_{ti},{\hat{\mathbf{y}}}_{tdi}).$$

$$\theta^{*}_{\text{ens}}=\operatorname*{argmin}_{\theta}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{N_{t}}\sum_{i=1}^{N_{t}}\mathcal{L}\bigg{(}\bm{y}_{ti},\frac{1}{D}\sum_{d=1}^{D}\hat{\bm{y}}_{tdi}\bigg{)}.$$

$$({\mathcal{D}}_{td},{\theta }_{td},\{{\mathbf{x}}_{ti},{\mathbf{y}}_{ti}{\}}_{i=1}^{N_t})$$

$${\theta }_{\mathcal{F}}^{\ast }=\underset{{\theta }_{\mathcal{F}}}{\mathrm{argmin}}\frac{1}{TD}\sum _{t=1}^T\frac{1}{N_t}\sum _{i=1}^{N_t}\sum _{d=1}^D\mathcal{L}({\mathbf{y}}_{ti},{\hat{\mathbf{y}}}_{tdi}).$$

$$\theta_{\mathcal{F}}\coloneqq\theta_{\mathcal{F}}-\alpha\sum_{d=1}^{D}\nabla_{\mathcal{F}}\big{(}y-\bm{w}_{d}^{\top}\mathcal{F}(\bm{x};\theta_{\mathcal{F}})\big{)}^{2}\,.$$

$$\begin{gathered} \alpha \sum _{d=1}^D{\nabla }_{\mathcal{F}}(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k){)}^2= \\ \gamma {\nabla }_{\mathcal{F}}(y-{\mathbf{w}}_o^{\top }\mathcal{F}(\mathbf{x};{\theta }_k){)}^2 \\ ⟹\alpha \sum _{d=1}^{D}2(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k)){\mathbf{w}}_d^{\top }{J}_{\mathcal{F}}(\mathbf{x};{\theta }_k)= \\ 2\gamma (y-{\mathbf{w}}_o^{\top }\mathcal{F}(\mathbf{x};{\theta }_k)){\mathbf{w}}_o^{\top }{J}_{\mathcal{F}}(\mathbf{x};{\theta }_k), \end{gathered}$$

$$\begin{gathered} \alpha \sum _{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_d^i= \\ \gamma (y-{\mathbf{w}}_o^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_o^i\forall i\in \mathrm{1..}M. \end{gathered}$$

$$\begin{gathered} \frac{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_d^i}{(y-{\mathbf{w}}_o^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_o^i}= \\ \frac{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_d^j}{(y-{\mathbf{w}}_o^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_o^j}\forall (i,j) \\ ⟹\frac{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_d^i}{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_k))w_d^j}=\frac{w_o^i}{w_o^j}. \end{gathered}$$

$$\begin{gathered} \frac{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_1))w_d^i}{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_1))w_d^j}\ne \\ \frac{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_2))w_d^i}{{\sum }_{d=1}^D(y-{\mathbf{w}}_d^{\top }\mathcal{F}(\mathbf{x};{\theta }_2))w_d^j}. \end{gathered}$$

$$\begin{gathered} \frac{1}{D}\sum _{d=1}^D{\hat{\mathbf{y}}}_{tdi}=\frac{1}{D}\sum _{d=1}^D{\mathbf{W}}_{td}^{\top }\mathcal{F}({\mathbf{x}}_{ti};{\theta }_{\mathcal{F}}) \\ ⟹{\mathbf{W}}_{to}^{\top }=\frac{1}{D}\sum _{d=1}^D{\mathbf{W}}_{td}^{\top }\text{ and }\beta =\alpha . \end{gathered}$$

Full text

Deep Learning, Multitask Learning, Pseudo-task Augmentation, Multiple Models, Model Search

1 Introduction

Multitask learning (MTL) (Caruana, 1998) improves performance by leveraging relationships between distinct learning problems. In recent years, MTL has been extended to deep learning, in which it has improved performance in applications such as vision (Zhang et al., 2014; Bilen & Vedaldi, 2016; Misra et al., 2016; Rudd et al., 2016; Lu et al., 2017; Rebuffi et al., 2017; Yang & Hospedales, 2017), natural language (Collobert & Weston, 2008; Dong et al., 2015; Liu et al., 2015a; Luong et al., 2016; Hashimoto et al., 2017), speech (Huang et al., 2013; Seltzer & Droppo, 2013; Huang et al., 2015; Wu et al., 2015), reinforcement learning (Devin et al., 2016; Jaderberg et al., 2017b; Teh et al., 2017), and even seemingly unrelated tasks from disparate domains (Kaiser et al., 2017; Meyerson & Miikkulainen, 2018). Deep MTL relies on training signals from multiple datasets to train deep structure that is shared across tasks. Since the shared structure must support solving multiple problems, it is inherently more general, which leads to better generalization to holdout data.

This paper adapts ideas from deep MTL to the single-task learning (STL) case, i.e., when only a single task is available for training. The method is formalized as pseudo-task augmentation (PTA), in which a single task has multiple distinct decoders projecting the output of the shared structure to task predictions. By training the shared structure to solve the same problem in multiple ways, PTA simulates the effect of training towards distinct but closely-related tasks drawn from the same universe. Theoretical justification shows how training dynamics with multiple pseudo-tasks strictly subsumes training with just one, and a class of algorithms is introduced for controlling pseudo-tasks in practice.

In an array of experiments, PTA is shown to significantly improve performance in single-task settings. Although different variants of PTA traverse the space of pseudo-tasks in qualitatively different ways, they all demonstrate substantial gains. Experiments also show that when PTA is combined with MTL, further improvements are achieved, including state-of-the-art performance on the CelebA dataset. In other words, although PTA can be seen as a base case of MTL, PTA and MTL have complementary value in learning more generalizable models. The conclusion is that pseudo-task augmentation is an efficient, reliable, and broadly applicable method for boosting performance in deep learning systems.

The remainder of the paper is organized as follows: Section 2 covers background on deep learning methods that train multiple models; Section 3 introduces the pseudo-task augmentation framework and practical implementations; Section 4 describes experimental setups and results; Sections 5 and 6 discuss future work and overall implications.

2 Training Multiple Deep Models

There is a broad range of methods that exploit synergies across multiple deep models. This section reviews these methods by classifying them into three types: (1) methods that jointly train a model for multiple tasks; (2) methods that train multiple models separately for a single task; and (3) methods that jointly train multiple models for a single task. This review motivates the development of methods in (3) that unify the advantages of (1) and (2).

2.1 Joint training of models for multiple tasks

There are many real-world scenarios where harnessing data from multiple related tasks can improve overall performance. In general, there are $T$ tasks $\{\{\bm{x}_{ti},\bm{y}_{ti}\}_{i=1}^{N_{t}}\}_{t=1}^{T}$, where $N_{t}$ is the number of samples for the $t$th task. Note that it is possible that for $t_{1}\neq t_{2}$, $N_{t_{1}}\neq N_{t_{2}}$, $\dim(\bm{x}_{t_{1}})\neq\dim(\bm{x}_{t_{2}})$, and/or $\dim(\bm{y}_{t_{1}})\neq\dim(\bm{y}_{t_{2}})$. The only requirement for multitask learning to be useful is that there is some amount of information shared across tasks, and, in theory, this is always the case (Mahmud & Ray, 2008; Mahmud, 2009).

Joint training of neural network models for multiple tasks was proposed decades ago (Caruana, 1998). Modern approaches have extended this early work to deep learning. Though more sophisticated methods now exist, the most common approach is still based on the original work, in which a joint model is decomposed into an underlying model $\mathcal{F}$ (parameterized by $\theta_{\mathcal{F}}$) that is shared across all tasks, and task-specific decoders $\mathcal{D}_{t}$ (parameterized by $\theta_{\mathcal{D}_{t}}$) for each task. The model for the $t$th task is then defined as

[TABLE]

Given a fixed model architecture for all $\mathcal{D}_{t}$ and $\mathcal{F}$, the joint model is completely defined by the parameters $\theta=(\{\theta_{\mathcal{D}_{t}}\}_{t=1}^{T},\theta_{\mathcal{F}})$. To maximize overall performance, the goal is to find optimal parameters $\theta^{*}$ such that

[TABLE]

for a suitable sample-wise loss function $\mathcal{L}$, e.g., mean squared error or cross-entropy loss. More sophisticated deep MTL approaches can be characterized by the design decision of how learned structure is shared across tasks. For example, some methods supervise different tasks at different depths of the shared structure (Zhang & Weiss, 2016; Hashimoto et al., 2017; Toshniwal et al., 2017); other methods duplicate the shared structure into columns and define mechanisms for sharing information across columns (Jou & Chang, 2016; Misra et al., 2016; Long et al., 2017; Yang & Hospedales, 2017). More detailed characterizations of deep MTL methods can be found in previous work (Ruder, 2017; Meyerson & Miikkulainen, 2018).

MTL has also been explored extensively outside of deep learning. Many such techniques take a similar approach of having shared structure with a separate linear decoder for each task, while enforcing regularization of shared convex structure (Evgeniou & Pontil, 2004; Argyriou et al., 2008; Kang et al., 2011; Kumar & Daumé, 2012). Overall, by requiring models to fit multiple real world datasets simultaneously, MTL is a promising approach to learning more realistic, and thus more generalizable, models.

2.2 Separate training of multiple models for STL

How to construct and train deep a neural network effectively is an open-ended design problem even in the case of a single task. A range of methods have been developed that aim at overcoming this problem by training multiple models separately for a single task. One class of methods searches for optimal fixed designs, e.g., by automatically optimizing learning hyperparameters (Bergstra et al., 2011; Snoek et al., 2012) or more open-ended network topologies (Miikkulainen et al., 2017; Real et al., 2017; Zoph & Le, 2017). The multiple models synergize by providing complementary information about different areas of the search space, and, over time, the results of past models can be used to generate better models. Population-based training takes this one step further, by copying the weights of successful models to new models (Jaderberg et al., 2017a). This weight-copying is similar to methods that transfer learned behavior across a sequence of pre-defined architectures (Hinton et al., 2015; Chen et al., 2016; Wei et al., 2016). The synergy of multiple models can also be exploited via ensembling (Dietterich, 2000). Overall, the widespread success of the above methods have shown the value of training multiple models separately, both sequentially and in parallel.

2.3 Joint training of multiple models for STL

Some existing methods can be viewed as jointly training multiple models for a single task. For instance, to improve training of deep models, deep supervision includes loss layers at multiple depths (Lee et al., 2015). As a by-product, this approach yields a distinct model for the task at each such depth, though only the deepest model is ever evaluated. As another example, dropout (Srivastava et al., 2014), and pseudo-ensembles more generally (Bachman et al., 2014), can be seen as implicitly training many relatively weak models that are combined during evaluation. Also, PathNet (Fernando et al., 2017) jointly trains multiple networks induced by various paths through a set of shared modules. However, the goal is not to improve single task performance, but discover structure that can be effectively reused by future tasks. Although these existing methods jointly train multiple models for a single task, they do not perform joint training in the MTL sense. Ideally, the benefits of the methods in Sections 2.1 and 2.2 could be combined, yielding methods that train multiple models that share underlying parameters and sample complementary high-performing areas of the model space. This paper takes first steps in that direction, showing that such methods are indeed promising. The specific approach, PTA, is introduced in the next section.

3 Pseudo-task Augmentation (PTA)

This section introduces the PTA method. First, the classical deep MTL approach is extended to the case of multiple decoders per task. Then, the concept of a pseudo-task is introduced, and increased training dynamics under multiple pseudo-tasks is demonstrated. Finally, practical methods for controlling pseudo-tasks during training are described, which will be compared empirically in Section 4.

3.1 A Classical Approach

The most common approach to deep MTL is still the “classical” approach (Eq. 1), in which all layers are shared across all tasks up to a high level, after which each task learns a distinct decoder that maps high-level points to its task-specific output space (Caruana, 1998; Ranjan et al., 2016; Lu et al., 2017). Even when more sophisticated methods are developed, the classical approach is often used as a baseline for comparison. The classical approach is also computationally efficient, in that the only additional parameters beyond a single task model are in the additional decoders. Thus, when applying ideas from deep MTL to single-task multi-model learning, the classical approach is a natural starting point.

Consider again the case where there are $T$ distinct true tasks, but now let there be $D$ decoders for each task. Then, the model for the $d$th decoder of the $t$th task is given by

[TABLE]

and the overall loss for the joint model from Eq. 2 becomes

[TABLE]

where $\theta=(\{\{\theta_{\mathcal{D}_{td}}\}_{d=1}^{D}\}_{t=1}^{T},\theta_{\mathcal{F}})$. In the same way as the classical approach to MTL encourages $\mathcal{F}$ to be more general and robust by requiring it to support multiple tasks, here $\mathcal{F}$ is required to support solving the same task in multiple ways. A visualization of a resulting joint model is shown in Figure 1.

A theme in MTL is that models for related tasks will have similar decoders, as implemented by explicit regularization (Evgeniou & Pontil, 2004; Kumar & Daumé, 2012; Long et al., 2017; Yang & Hospedales, 2017). Similarly, in Eq. 4, through training, two decoders for the same task will instantiate similar models, and, as long as they do not converge completely to equality, they will simulate the effect of training with multiple closely-related tasks.

Notice that the innermost summation in Eq. 4 is over decoders. This calculation is computationally efficient: because each decoder for a given task takes the same input, $\mathcal{F}(\bm{x}_{ti})$ (usually the most expensive part of the model) need only be computed once per sample (and only once over all tasks if all tasks share $\bm{x}_{ti}$). However, when evaluating the performance of a model, since each decoder induces a distinct model for a task, what matters is not the average over decoders, but the best performing decoder for each task, i.e.,

[TABLE]

Eq. 4 is used in training because it is smoother; Equation 5 is used for model validation, and to select the best performing decoder for each task from the final joint model. This decoder is then applied to future data, e.g., a holdout set. Once the models are trained, in principle they form a set of distinct and equally powerful models for each task. It may therefore be tempting to ensemble them for evaluation, i.e.,

[TABLE]

However, with linear decoders, training with Eq. 6 is equivalent to training with a single decoder for each task, while training with Eq. 4 with multiple decoders yields more expressive training dynamics. These ideas are developed more fully in the next section.

3.2 Pseudo-tasks

Following the intuition that training $\mathcal{F}$ with multiple decoders amounts to solving the task in multiple ways, each “way” is defined by a pseudo-task

[TABLE]

of the true underlying task $\{\bm{x}_{ti},\bm{y}_{ti}\}_{i=1}^{N_{t}}$. It is termed a pseudo-task because it derives from a true task, but has no fixed labels. That is, for any fixed $\theta_{td}$, there are potentially many optimal outputs for $\mathcal{F}(\bm{x}_{ti})$. When $D>1$, training $\mathcal{F}$ amounts to training each task with multiple pseudo-tasks for each task at each gradient update step. This process is the essence of PTA.

As a first step, this paper considers linear decoders, i.e. each $\mathcal{D}_{td}$ consists of a single dense layer of weights (any following nonlinearity can be considered part of the loss function). Prior work has assumed that models for closely-related tasks differ only by a linear transformation (Evgeniou & Pontil, 2004; Kang et al., 2011; Argyriou et al., 2008). Similarly, with linear decoders, distinct pseudo-tasks for the same task simulate multiple closely-related tasks. When $\theta_{td}$ are considered fixed, the learning problem (Eq. 4) reduces to

[TABLE]

In other words, although the overall goal is to learn models for $T$ tasks, $\mathcal{F}$ is at each step optimized towards $DT$ pseudo-tasks. Thus, training with multiple decoders may yield positive effects similar to training with multiple true tasks.

After training, the best model for a given task is selected from the final joint model, and used as the final model for that task (Eq. 5). Of course, using multiple decoders with identical architectures for a single task does not make the final learned predictive models more expressive. It is therefore natural to ask whether including additional decoders has any fundamental effect on learning dynamics. It turns out that even in the case of linear decoders, the training dynamics of using multiple pseudo-tasks strictly subsumes using just one.

Definition 1** (Pseudo-task Simulation).**

A set of pseudo-tasks $S_{1}$ simulates another $S_{2}$ on $\mathcal{F}$ if for all $\theta_{\mathcal{F}}$ the gradient update to $\theta_{\mathcal{F}}$ when trained with $S_{1}$ is equal to that with $S_{2}$.

Theorem 1** (Augmented Training Dynamics).**

There exist differentiable functions $\mathcal{F}$ and sets of pseudo-tasks of a single task that cannot be simulated by a single pseudo-task of that task, even when all decoders are linear.

Proof.

Consider a task with a single sample $(\bm{x},y)$, where $y$ is a scalar. Suppose $\mathcal{L}$ (from Eq. 8) computes mean squared error, $\mathcal{F}$ has output dimension $M$, and all decoders are linear, with bias terms omitted for clarity. $\mathcal{D}_{d}$ is then completely specified by the vector $\bm{w}_{d}=\langle w_{d}^{1},w_{d}^{2},...,w_{d}^{M}\rangle^{\top}$. Suppose parameter updates are performed by gradient descent. The update rule for $\theta_{\mathcal{F}}$ with fixed decoders $\{\mathcal{D}_{d}\}_{d=1}^{D}$ and learning rate $\alpha$ is then given by

[TABLE]

For a single fixed decoder to yield equivalent behavior, it must have equivalent update steps. The goal then is to choose $(\bm{x},y)$, $\mathcal{F}$, $\{\theta_{k}\}_{k=1}^{K}$, $\{\bm{w}_{d}\}_{d=1}^{D}$, and $\alpha>0$, such that there are no $\bm{w}_{o}$, $\gamma>0$, for which $\forall\,k$

[TABLE]

where $J_{\mathcal{F}}$ is the Jacobian of $\mathcal{F}$. By choosing $\mathcal{F}$ and $\{\theta_{k}\}_{k=1}^{K}$ so that all $J_{\mathcal{F}}(\bm{x};\theta_{k})$ have full row rank, Eq. 10 reduces to

[TABLE]

Choosing $\mathcal{F}$, $\{\theta_{k}\}_{k=1}^{K}$, $\{\bm{w}_{d}\}_{d=1}^{D}$, and $\alpha>0$ such that the left hand side of Eq. 11 is never zero, we can safely write

[TABLE]

Then, since $\bm{w}_{o}$ is fixed, it suffices to find $\mathcal{F}(\bm{x};\theta_{1})$, $\mathcal{F}(\bm{x};\theta_{2})$ such that for some $(i,j)$

[TABLE]

For instance, with $D=2$, choosing $y=1$, $w_{1}=\langle 2,3\rangle^{\top}$, $w_{2}=\langle 4,5\rangle^{\top}$, $\mathcal{F}(\bm{x};\theta_{1})=\langle 6,7\rangle^{\top}$, and $\mathcal{F}(\bm{x};\theta_{1})=\langle 8,9\rangle^{\top}$ satisfies the inequality. Note $\mathcal{F}(\bm{x};\theta_{1})$ and $\mathcal{F}(\bm{x};\theta_{2})$ can be chosen arbitrarily since $\mathcal{F}$ is only required to be differentiable, e.g., implemented by a neural network. ∎

Showing that a single pseudo-task can be simulated by $D$ pseudo-tasks for any $D>1$ is more direct: For any $\bm{w}_{o}$ and $\gamma$, choose $\bm{w}_{d}=\bm{w}_{o}\ \forall\ d\in 1..D$ and $\alpha=\nicefrac{{\gamma}}{{D}}$. Further extensions to tasks with more samples, higher dimensional outputs, and cross-entropy loss are straightforward. Note that this result is related to work on the dynamics of deep linear models (Saxe et al., 2014), in that adding additional linear structure complexifies training dynamics. However, training an ensemble directly, i.e., via Eq. 6, does not yield augmented training dynamics, since

[TABLE]

Now that we know that training with additional pseudo-tasks yields augmented training dynamics that may be exploited, the question is how to take advantage of these dynamics in practice. The next section introduces methods to address this question.

3.3 Control of Multiple Pseudo-task Trajectories

Given linear decoders, the primary goal is to optimize $\mathcal{F}$; if an optimal $\mathcal{F}$ were found, optimal decoders for each task could be derived analytically. So, given multiple linear decoders for each task, how should their induced pseudo-tasks be controlled to maximize the benefit to $\mathcal{F}$? For one, their weights $\{\bm{W}_{td}\}_{d=1}^{D}$ must not all be equal, otherwise we would have $\bm{W}_{to}=\bm{W}_{t1}$ and $\gamma=D\alpha$ in the proof of Theorem 1. Following Eq. 4, decoders can be trained jointly with $\mathcal{F}$ via gradient-based methods, so that they learn to work well with $\mathcal{F}$. Through optimization, a trained decoder induces a trajectory of pseudo-tasks. Going beyond this implicit control, Algorithm 1 gives a high-level framework for applying explicit control to pseudo-task trajectories.

An instance of the algorithm is parameterized by choices for DecInitialize, which defines how decoders are initialized; and DecUpdate, which defines non-gradient-based updates to decoders every $M$ gradient steps, i.e., every meta-iteration, based on the performance of each decoder (DecUpdate defaults to no-op). As a first step, several intuitive methods are evaluated in this paper for instantiating Algorithm 1. These methods can be used together in any combination:

Independent Initialization (I) DecInitialize randomly initializes all $\theta_{\mathcal{D}_{td}}$ independently. This is the obvious initialization method, and is assumed in all methods below.

Freeze (F) DecInitialize freezes all decoder weights except $\theta_{\mathcal{D}_{t1}}$ for each task. Frozen weights do not receive gradient updates in Line 7 of Algorithm 1. Because they cannot adapt to $\mathcal{F}$, constant pseudo-task trajectories provide a stricter constraint on $\mathcal{F}$. One decoder is left unfrozen so that the optimal model for each task can still be learned.

Independent Dropout (D) DecInitialize sets up the dropout layers preceding linear decoder layers to drop out values independently for each decoder. Thus, even when the weights of two decoders for a task are equal, their resulting gradient updates to $\mathcal{F}$ and to themselves will be different.

For the next three methods, let $c_{t}^{\text{min}}=\min(c_{t1},\ldots,c_{tD})$.

Perturb (P) DecUpdate adds noise $\sim\mathcal{N}(\bm{0},\epsilon_{p}\bm{I})$ to each $\theta_{\mathcal{D}_{td}}$ for all $d$ where $c_{td}\neq c_{t}^{\text{min}}$. This method ensures that $\theta_{\mathcal{D}_{td}}$ are sufficiently distinct before each training period.

Hyperperturb (H) Like Perturb, except DecUpdate updates the hyperparameters of each decoder other than the best for each task, by adding noise $\sim\mathcal{N}(0,\epsilon_{h})$. In this paper, each decoder has only one hyperparameter: the dropout rate of any Independent Dropout layer, because adapting dropout rates can be beneficial (Ba & Frey, 2013; Li et al., 2016; Jaderberg et al., 2017a).

Greedy (G) For each task, let $\theta_{t}^{\text{min}}$ be the weights of a decoder with cost $c_{t}^{\text{min}}$. DecUpdate updates all $\theta_{td}\coloneqq\theta_{t}^{\text{min}}$, including hyperparameters. This biases training to explore the highest-performing areas of the pseudo-task space. When combined with any of the previous three methods, decoder weights are still ensured to be distinct through training.

Combinations of these six methods induce an initial class of PTA training algorithms PTA-* for the case of linear decoders. The next section evaluates eight representative combinations of these methods, i.e., PTA-I, PTA-F, PTA-P, PTA-D, PTA-FP, PTA-GP, PTA-GD, and PTA-HGD, in various experimental settings. Note that H and G are related to methods that copy the weights of the entire network (Jaderberg et al., 2017a). Also note that, in a possible future extension to the nonlinear case, the space of possible PTA control methods becomes much more broad, as will be discussed in Section 5.

4 Experiments

In this section, PTA methods are evaluated and shown to excel in a range of settings: (1) single-task character recognition; (2) multitask character recognition; (3) single-task sentiment classification; and (4) multitask visual attribute classification. All experiments are implemented using the Keras framework (Chollet et al., 2015). For PTA-P and PTA-GP, $\epsilon_{p}=0.01$; for PTA-HGD, $\epsilon_{h}=0.1$ and dropout rates range from 0.2 to 0.8. A dropout layer with dropout rate initialized to 0.5 precedes each decoder.

4.1 Omniglot Character Recognition

This section evaluates and compares the various PTA methods on Omniglot character recognition (Lake et al., 2015). The Omniglot dataset consists of 50 alphabets of handwritten characters, each of which induces its own character recognition task. Each character instance is a $105\times 105$ black-and-white image, and each character has 20 instances, each drawn by a different individual. To reduce variance and improve reproducibility of experiments, a fixed random 50/20/30% train/validation/test split was used for each task. (These splits will be released with the paper.) Methods are evaluated with respect to all 50 tasks as well as a subset consisting of the first 20 tasks in a fixed random ordering of alphabets used in previous work (Meyerson & Miikkulainen, 2018). The underlying model $\mathcal{F}$ for all setups is a simple four layer convolutional network that has been shown to yield good performance on Omniglot (Meyerson & Miikkulainen, 2018). This model has four convolutional layers each with 53 filters and $3\times 3$ kernels, and each followed by a $2\times 2$ max-pooling layer and dropout layer with 0.5 dropout probability. At each meta-iteration, 250 gradient updates are performed via Adam (Kingma & Ba, 2014); each setup is trained for 100 meta-iterations.

4.1.1 Omniglot: Single-task Learning

The single-task learning case is considered first. For each of the 20 initial Omniglot tasks, the eight PTA methods were applied to the task with 2, 3, and 4 decoders. At least three trials were run with each setup; the mean performance averaged across trials and tasks is shown in Figure 2.

Every PTA setup outperforms the baseline, i.e., training with a single decoder. The methods that use decoder freezing, PTA-F and PTA-FP, perform best, showing how this problem can benefit from strong regularization. Notably, the mean improvement across all methods increases with $D$: 1.86% for $D=2$; 2.33% for $D=3$; and 2.70% for $D=4$. Like MTL can benefit from adding more tasks (Caruana, 1998; Hashimoto et al., 2017; Jaderberg et al., 2017b), single-task learning can benefit from adding more pseudo-tasks.

4.1.2 Omniglot: Multitask Learning

Omniglot models have also been shown to benefit from MTL (Maclaurin et al., 2015; Rebuffi et al., 2017; Yang & Hospedales, 2017; Meyerson & Miikkulainen, 2018). This section extends the experiments in Section 4.1.1 to MTL. The setup is exactly the same, except now the underlying convolutional model is fully shared across all tasks for each method. The results are shown in Figure 3.

All setups outperform the STL baseline, and all, except for PTA-I with $D=2$, outperform the MTL baseline. Again, PTA-F and PTA-FP perform best, and the mean improvement across all methods increases with $D$. The results show that although PTA implements behavior similar to MTL, when combined, their positive effects are complementary. Finally, to test the scalability of these results, three diverse PTA methods with $D=4$ and $D=10$ were applied to the complete 50-task dataset: PTA-I, because it is the baseline PTA method; PTA-F, because it is simple and high-performing; and PTA-HGD, because it is the most different from PTA-F, but also relatively high-performing. The results are given in Table 1.

The results agree with the 20-task results, with all methods improving upon the baseline, and performance overall improving as $D$ is increased.

4.2 IMDB Sentiment Analysis

The experiments in this section apply PTA to LSTM models in the IMDB sentiment classification problem (Maas et al., 2011). The dataset consists of 50K natural-language movie reviews, 25K for training and 25K for testing. There is a single binary classification task: whether a review is positive or negative. As in previous work, 2500 of the training reviews are withheld for validation (McCann et al., 2017). The underlying model $\mathcal{F}$ is the off-the-shelf LSTM model for IMDB provided by Keras, with no parameters or preprocessing changed. In particular, the vocabulary is capped at 20K words, the LSTM layer has 128 units and dropout rate 0.2, and each meta-iteration consists of one epoch of training with Adam (Kingma & Ba, 2014). This is not a state-of-the-art model, but it is a very different architecture from that used in Omniglot, and therefore serves to demonstrate the broad applicability of PTA.

The final three PTA methods from Section 4.1 were evaluated with 4 and 10 decoders (Table 2).

As in Section 4.1, all PTA methods outperform the baseline. In this case, however, PTA-HGD with $D=10$ performs best. Notably, PTA-I and PTA-F do not improve from $D=4$ to $D=10$, suggesting that underlying models have a critical point after which, without careful control, too many decoders can be overconstraining. To contrast PTA with standard regularization, additional Baseline experiments were run with dropout rates $[0.3,0.4,...,0.9]$. At 0.5 the best accuracy was achieved: 83.14 ($\pm 0.05$), which is less than all PTA variants except PTA-I with $D=10$, thus confirming that PTA adds value. To help understand what each PTA method is actually doing, snapshots of decoder parameters taken every epoch are visualized in Figure 4 with t-SNE (van der Maaten & Hinton, 2008) using cosine distance.

The behavior matches our intuition for what should be happening in each case: When decoders are only initialized independently, their pseudo-tasks gradually converge; when all but one decoder is frozen, the unfrozen one settles between the others; and when a greedy method is used, decoders perform local exploration as they traverse the pseudo-task space together.

4.3 CelebA Facial Attribute Recognition

To further test applicability and scalability, PTA was evaluated on CelebA large-scale facial attribute recognition (Liu et al., 2015b). The dataset consists of $\approx$200K $178\times 218$ color images. Each image has binary labels for 40 facial attributes; each attribute induces a binary classification task. Facial attributes are related at a high level that deep models can exploit, making CelebA a popular deep MTL benchmark. Thus, this experiment focuses on the MTL setting.

The underlying model was Inception-ResNet-v2 (Szegedy et al., 2016), with weights initialized from training on ImageNet (Russakovsky et al., 2015). Due to computational constraints, only one PTA method was evaluated: PTA-HGD with $D=10$. PTA-HGD was chosen because of its superior performance on IMDB, and because CelebA is a large-scale problem that may require extended pseudo-task exploration; Figure 4 shows how PTA-HGD may support such exploration above other methods. Each meta-iteration consists of 250 gradient updates with batch size 32. The optimizer schedule is co-opted from previous work (Günther et al., 2017): RMSprop is initialized with a learning rate of $10^{-4}$, which is decreased to $10^{-5}$ and $10^{-6}$ when the model converges. PTA-HGD and the MTL baseline were each trained three times. The computational overhead of PTA-HGD is marginal, since the underlying model has 54M parameters, while each decoder has only 1.5K. Table 3 shows the results.

PTA-HGD outperforms all other methods, thus establishing a new state-of-the-art in CelebA. Figure 5 shows resulting dropout schedules for PTA-HGD.

No one type of schedule dominates; PTA-HGD gives each task the flexibility to adapt its own schedule via the performance of its pseudo-tasks.

5 Discussion and Future Work

The experiments in this paper demonstrated that PTA is broadly applicable, and that it can boost performance in a variety of single-task and multitask problems. Training with multiple decoders for a single task allows a broader set of models to be visited. If these decoders are diverse and perform well, then the shared structure has learned to solve the same problem in diverse ways, which is a hallmark of robust intelligence. In the MTL setting, controlling each task’s pseudo-tasks independently makes it possible to discover diverse task-specific learning dynamics (Figure 5). Increasing the number of decoders can also increase the chance that pairs of decoders align well across tasks.

The crux of PTA is the method for controlling pseudo-task trajectories. Experiments showed that the amount of improvement from PTA is dependent on the choice of control method. Different methods exhibit highly structured but different behavior (Figure 4). The success of initial methods indicates that developing more sophisticated methods is a promising avenue of future work. In particular, methods from Section 2.2 can be co-opted to control pseudo-task trajectories more effectively. Consider, for instance, the most involved method evaluated in this paper: PTA-HGD. This online decoder search method could be replaced by methods that generate new models more intelligently (Bergstra et al., 2011; Snoek et al., 2012; Miikkulainen et al., 2017; Real et al., 2017; Zoph & Le, 2017). Such methods will be especially useful in extending PTA beyond the linear case considered in this paper, to complex nonlinear decoders. For example, since a set of decoders is being trained in parallel, it could be natural to use neural architecture search methods (Miikkulainen et al., 2017; Real et al., 2017; Zoph & Le, 2017) to search for optimal decoder architectures. While ensembling separate PTA models is useful (Table 3), in preliminary tests naïvely ensembling decoders for evaluation (Eq. 6) did not yield remarkable improvements over the single best (Eq. 5). In a further preliminary test with IMDB, when $\mathcal{F}$ was not shared, PTA-I outperformed PTA-HGD and PTA-F, indicating that the latter two methods address dynamics that arise in joint training but not naïve ensemble training. Developing PTA training methods for generating a more complementary set of decoders, coupled with effective methods for ensembling this set, could push performance even further, especially when decoders are more complex.

6 Conclusion

This paper has introduced pseudo-task augmentation, a method that makes it possible to apply ideas from deep MTL to single-task learning. By training shared structure to solve the same task in multiple ways, pseudo-task augmentation simulates training with multiple closely-related tasks, yielding performance improvements similar to those in MTL. However, the methods are complementary: combining pseudo-task augmentation with MTL results in further performance gains. Broadly applicable, pseudo-task augmentation is thus a promising method for improving deep learning performance. Overall, this paper has taken first steps towards a future class of efficient model search algorithms that exploit intratask parameter sharing.

Acknowledgements

We would like to thank Xin Qiu, Antoine Saliou, and the reviewers for providing valuable feedback that helped to solidify this work.