TRUST: Token-dRiven Ultrasound Style Transfer for Cross-Device Adaptation

TL;DR

TRUST is a novel ultrasound style transfer framework that selectively filters style features to improve cross-device adaptation, enhancing downstream task accuracy while preserving source content.

Contribution

The paper introduces TRUST, a token-driven dual-stream method with a new module for selecting relevant style tokens, explicitly aligning style transfer with downstream task needs.

Findings

Outperforms existing UI2I methods in visual quality.

Improves downstream task performance on ultrasound datasets.

Effectively preserves source content during style transfer.

Abstract

Ultrasound images acquired from different devices exhibit diverse styles, resulting in decreased performance of downstream tasks. To mitigate the style gap, unpaired image-to-image (UI2I) translation methods aim to transfer images from a source domain, corresponding to new device acquisitions, to a target domain where a frozen task model has been trained for downstream applications. However, existing UI2I methods have not explicitly considered filtering the most relevant style features, which may result in translated images misaligned with the needs of downstream tasks. In this work, we propose TRUST, a token-driven dual-stream framework that preserves source content while transferring the common style of the target domain, ensuring that content and style remain unblended. Given multiple styles in the target domain, we introduce a Token-dRiven (TR) module that operates from two perspectives: (1) a data view--selecting "suitable" target tokens corresponding to each source token, and (2) a model view--identifying ``optimal" target tokens for the downstream model, guided by a behavior mirror loss. Additionally, we inject auxiliary prompts into the source encoder to match content representation with downstream behavior. Experimental results on ultrasound datasets demonstrate that TRUST outperforms existing UI2I methods in both visual quality and downstream task performance.

Tables

Table 1. Table 1 : Performance comparison of style transfer methods on 6 6 cross-device transfer tasks for both classification and segmentation. Classification performance is evaluated by Accuracy (Acc) and Area Under the Curve (AUC), while segmentation performance is evaluated by Dice score (Dice) and Intersection over Union (IoU). The results are reported in the order: Acc / AUC / Dice / IoU. w/o ST indicates the baseline setting without Style Transfer . The top and second-best results are highlighted in bold and underline .

Method	UCLM $\to$ BUSI	BUSI $\to$ UCLM	UCLM $\to$ UDIAT	UDIAT $\to$ UCLM	BUSI $\to$ UDIAT	UDIAT $\to$ BUSI
\rowcolorgray!20 w/o ST	70.0/74.8/77.1/66.5	65.6/68.8/77.7/68.0	63.8/68.2/82.2/72.1	75.5/77.1/85.1/75.6	73.3/73.2/79.5/70.6	85.7/91.9/84.3/74.8
CycleGAN [5]	51.3/64.6/65.2/52.4	69.2/72.4/75.7/65.5	56.3/56.9/74.7/63.9	67.4/69.2/80.4/69.2	70.8/72.0/79.7/70.4	79.5/81.6/82.5/72.6
DiscoGAN [15]	70.0/72.0/67.9/56.3	63.1/74.1/73.2/63.2	65.0/70.1/42.6/30.6	69.4/81.4/44.1/33.0	72.3/73.8/77.4/67.3	87.8/88.0/79.2/68.0
S2WAT [7]	72.5/75.5/76.2/65.9	62.6/57.4/77.4/67.5	65.0/45.7/80.5/70.0	75.5/71.6/84.6/75.2	71.8/73.4/80.1/71.0	85.7/93.4/81.8/72.1
TransColors [6]	72.5/77.3/75.9/65.4	64.1/65.0/77.7/67.9	62.5/68.4/82.6/72.6	77.6/73.9/85.7/76.2	69.2/71.1/80.4/71.4	83.7/92.1/83.4/74.0
TRUST	80.0/81.9/79.9/69.7	71.8/77.9/78.6/69.0	72.5/76.7/83.3/73.4	85.7/89.7/86.9/77.9	79.5/84.5/80.7/71.8	89.8/94.9/85.4/76.5

Table 2. Table 2 : Ablation study on different components of our framework . “w/o ST” denotes the case without style transfer. CA refers to the baseline model employing Cross Attention, TR represents the Token-dRiven module, and TRUST integrates TR with auxiliary prompts for enhanced style adaptation.

Settings	BUSI $\to$ UCLM	UCLM $\to$ UDIAT	UDIAT $\to$ BUSI
\rowcolorgray!20 w/o ST	65.6/68.8/77.7/68.0	63.8/68.2/82.2/72.1	85.7/91.9/84.3/74.8
CA	66.7/64.9/77.8/68.0	60.0/66.3/82.9/72.9	87.8/92.3/83.6/73.9
TR	68.7/76.7/78.5/68.6	71.3/76.2/83.1/72.9	89.8/92.5/85.4/76.2
TRUST	71.8/77.9/78.6/69.0	72.5/76.7/83.3/73.4	89.8/94.9/85.4/76.5

Equations

$$F_s=E_s(x_s,P),F_s\in {\mathbb{R}}^{N\times d},$$

$$F_t=E_t(X_T^{batch}),F_t\in {\mathbb{R}}^{B\times N\times d},$$

$$M_{ij}=\text{dot}(F_s^i,F_t^j),$$

$$F_{align}=F_s+M{F}_t.$$

$$\mathcal{L}_{bm}\big{(}G(x_{s}^{i}),y_{s}^{i}\big{)}=\ell_{down}\big{(}f_{downs}(G(x_{s}^{i})),y_{s}^{i}\big{)},$$

$${\hat{x}}_t=D(F_{align}),$$

$$F_{s}=E_{s}\big{(}\{P^{(l)}\oplus x_{s}^{(l)}\}_{l=1}^{L}\big{)},$$

$${\mathcal{L}}_{total}={\mathcal{L}}_c({\hat{x}}_t,x_s)+{\mathcal{L}}_s({\hat{x}}_t,x_t)+{\mathcal{L}}_{bm}({\hat{x}}_t,y_s^i).$$

Full text

††Equal contribution: Nhat-Tuong Do-Tran and Ngoc-Hoang-Lam Le.††Corresponding author: Ching-Chun Huang.

1 Introduction

Ultrasound (US) imaging is widely adopted in clinical diagnosis due to its efficiency, affordability, safety, and real-time capability [1, 2]. However, images acquired from different devices or medical centers often exhibit substantial variation in visual characteristics—such as intensity, contrast, resolution, and speckle patterns—due to differences in hardware configurations and acquisition protocols. These cross-device discrepancies pose serious challenges to both automated systems and human interpretation. Deep learning models, once trained on a specific source domain, tend to suffer significant performance degradation when deployed in unseen domains. Similarly, doctors—accustomed to images from familiar devices—may face difficulty interpreting cases from new devices that display unfamiliar style distributions. While retraining on new devices is a possible solution, it is often impractical: (1) For doctors, adapting to new device styles requires considerable effort in diagnosing new image characteristics. (2) For deployed AI models, fine-tuning is typically not feasible due to proprietary software constraints (i.e., cannot be reconfigured model parameters). To mitigate such domain shifts, Ultrasound Style Transfer (UST) has emerged as a promising solution. Instead of retraining, UST translates the source-domain images to match the style of the target domain while preserving their structural content. In this way, both doctors and downstream models operate in a target-style environment without altering their original behavior.

During the last decades, unpaired image-to-image translation methods have been explored, aiming to adapt images from the source domain to the target domain without requiring paired data. Among these, GAN-based methods [3, 4, 5] have been early solutions for bridging domain gaps, but are mainly effective for small shifts. Moreover, their shared discriminator often entangles content and style. Transformer-based models [6, 7] have shown strong capabilities in disentangling content and style by separating their respective branches, effectively preventing undesired blending. Their self-attention mechanism enables a global understanding of spatial structure, which helps preserve content integrity and maintain consistent patterns across layers [8, 9]. Despite these advantages, existing transformer-based methods often rely on pairwise style translation, which limits their ability to capture diverse styles, especially when the target domain exhibits multiple stylistic variations. Furthermore, transformer-based methods lack trustworthy token alignments, which are key to reliable downstream performance.

To address the above challenges, we propose the TRUST, a Token-dRiven Ultrasound Style Transfer for Cross-Device Adaptation. Our method begins with a disentangling strategy that separates the source and target into content and style branches, respectively. This strategy enables explicit extraction and independent processing of style and content features, allowing finer control over the transfer. Inspired by Einstein’s mindset that “not all information is knowledge!”, TRUST recognizes that not all style features are beneficial for transfer. Naively integrating global style tokens can lead to artifacts, such as overly bright tumor regions or suppressed tissue contrast. To this end, TRUST employs a selective mechanism that searches through a pool of style tokens to identify those best aligned with each content token (Figure 1). Crucially, the selection of style features is guided not only by the content tokens but also by auxiliary prompts injected into the content encoder layers. These prompts help the TR module more effectively retain task-specific and highly relevant style features. Together, the TR module and auxiliary prompts strengthen the connection between content and style branches, both at the data and model levels, enabling more accurate knowledge transfer. enabling more accurate knowledge transfer. Additionally, to align the output with the downstream task, we construct a mimic model that replicates the behavior of the black-box downstream model. Then, we use the mimic model’s predictions as supervision to align the styled image output through a behavior mirror loss. We summarize the key contributions of TRUST as follows:

The Token-dRiven (TR) module employs attention to identify trustworthy style tokens and selectively fuses them based on content relevance, enabling more precise and natural style integration. 2. 2.

We inject auxiliary prompts into the content encoder to enhance feature representation and more effectively minimize the domain gap. 3. 3.

We introduce a behavior mirror loss that uses a mimic downstream to replicate black-box behavior and supervise the styled image output. 4. 4.

Comprehensive experiments demonstrate that TRUST consistently outperforms baseline methods across multiple datasets, achieving outstanding results with well-preserved content structures and desirable style patterns.

2 Proposed method

2.1 Problem definition

We formulate the ultrasound style transfer for cross-device adaptation as an unpaired image-to-image translation problem. Let $\mathcal{D}_{S}=\{(x_{s}^{i},y_{s}^{i})\}_{i=1}^{N_{labeled}}\cup\{x_{s}^{k}\}_{k=1}^{N_{unlabeled}}$ represent the source domain dataset, where a few $x_{s}^{i}$ denotes an labeled image with corresponding annotations $y_{s}^{i}$, while the majority of source samples $x_{s}^{k}$ are unlabeled. Similarly, $\mathcal{D}_{T}=\{x_{t}^{j}\}_{j=1}^{N_{T}}$ represents the target domain dataset without annotations, where images $x_{t}^{j}$ are acquired from different ultrasound scanners.

Our goal is to learn a translation function $G:X_{S}\rightarrow X_{T}$ capable of mapping images $x_{s}\in X_{S}$ to the target domain $\mathcal{D}_{T}$, thus addressing the domain shift caused by cross-device variations. The translated images $\hat{x}_{t}=G(x_{s})$ should preserve the anatomical content structures inherent in $x_{s}$ while effectively adapting to the style characteristics found in $X_{T}$.

To achieve this, we design our style transfer network $G(x_{s})$, TRUST, as illustrated in Figure 2. TRUST comprises three key components: feature extractors (Section 2.2) to encode representations from the source and target domains, a Token-dRiven (TR) module (Section 2.3) that aligns one-source tokens with multiple-target tokens via data-view and model-view, and a decoder (Section 2.4) that reconstructs stylized images while preserving source content and transferring target style. In addition, auxiliary prompts (Section 2.5) are injected into the content encoder to enhance feature representations.

2.2 Feature extractors

Our feature extraction module consists of two Transformer-based branches: a content branch and a style branch. Both branches are constructed with multiple Transformer layers [10], leveraging the self-attention mechanism to effectively preserve structural information and progressively enhance semantic representations across layers.

Content branch. Focuses on capturing structural and anatomical details from the source domain. Transformer-based architectures are particularly suitable for this purpose because they keep resolution unchanged, while enriching the semantic information layer by layer, as demonstrated in [9]. Given a source image $x_{s}\in\mathbb{R}^{H\times W\times C}$, the image is first divided into non-overlapping patches of size $P\times P$, resulting in $N=\frac{H\cdot W}{P^{2}}$ patches. After linear projection, the patch tokens are added with learnable positional embeddings to encode spatial information. The source tokens are then processed by the transformer layers as follows:

[TABLE]

where $E_{s}$ represents the source encoder, $N$ is the number of patches, and $d$ is the embedding dimension of each token. $P$ is a set of auxiliary prompts that can be inserted into the token sequence, as described in Section 2.5.

Style branch. Processes target domain images to extract target tokens that reflect the distributional and stylistic patterns of the target domain. Given a batch of target images $X_{T}^{batch}=\{x_{t}^{1},x_{t}^{2},\dots,x_{t}^{B}\}$, where each $x_{t}^{j}\in\mathbb{R}^{H\times W\times C}$ and $B$ is the batch size, the encoder produces target tokens:

[TABLE]

where $E_{t}$ denotes the target encoder.

The use of stacked Transformer layers in both branches ensures that spatial information is preserved while semantic understanding is progressively improved, enabling effective feature extraction for both content and style domains [9].

2.3 Token-dRiven module

To effectively select suitable and informative multiple-target tokens for one-source tokens, we propose the TR module via data-view and model-view.

Data-view. From a data perspective, we design the TR module to align each source token with target tokens from multiple samples by computing a correlation matrix that captures their pairwise relationships. Formally, given source tokens $F_{s}\in\mathbb{R}^{N\times d}$ and target tokens $F_{t}\in\mathbb{R}^{B\times N\times d}$, the correlation matrix $M\in\mathbb{R}^{N\times(B\cdot N)}$ is computed as:

[TABLE]

where $\text{dot}(\cdot,\cdot)$ represents the dot product between each source token and the aggregated target tokens across the batch.

The aligned features $F_{align}\in\mathbb{R}^{N\times d}$ are then obtained by aggregating the target tokens weighted by this correlation matrix, followed by a skip connection to preserve the original source content:

[TABLE]

In this way, each source token selectively attends to the most relevant target tokens across different samples, enabling style adaptation while maintaining source content structure.

Model-view. To further enforce consistency with downstream tasks, we introduce the behavior mirror loss ($\mathcal{L}_{bm}$), which leverages supervision signals from a downstream model trained on the labeled source samples, $D_{s}^{labeled}=\{(x_{s}^{i},y_{s}^{i})\}_{i=1}^{N_{labeled}}$. The behavior mirror loss is formulated as:

[TABLE]

where $f_{downs}(\cdot)$ is the downstream model, selected according to the deployment scenario (i.e., ViT-B/16 [11] for classification and SAMUS [12] for segmentation), and $\ell_{down}(\cdot,\cdot)$ represents the corresponding loss function. By minimizing $\mathcal{L}_{bm}$, the source tokens $F_{s}$ are directly optimized toward the downstream objective, ensuring that the translated image $G(x_{s}^{i})$ produces predictions consistent with the label $y_{s}^{i}$.

2.4 Decoder

The decoder reconstructs the stylized image from the aligned token features through a convolutional architecture following previous style transfer works [9, 13]. Given the aligned tokens $F_{align}\in\mathbb{R}^{N\times d}$, the decoder first reshapes them into a spatial feature map of size $H^{\prime}\times W^{\prime}\times d$, where $H^{\prime}$ and $W^{\prime}$ equal to $\sqrt{N}$. The final output is obtained via CNN-based decoder layers as follows:

[TABLE]

where $D$ denotes the decoder, and $\hat{x}_{t}\in\mathbb{R}^{H\times W\times 3}$ is the reconstructed stylized image.

2.5 Auxiliary prompts

To enhance content representations for better compatibility with the downstream model. We inspired of VPT [14] to inject a set of learnable prompt tokens $P=\{P^{(l)}\in\mathbb{R}^{L_{p}\times d}\}_{l=1}^{L}$ into the source encoder (Section 2.2), where $L$ is the number of transformer layers, $L_{p}$ is the number of prompt tokens per layer, and $d$ is the token dimension.

At each transformer layer, a unique set of prompt tokens $P^{(l)}$ is prepended to the input token sequence. The prompt-enhanced content extraction is thus formulated as:

[TABLE]

where $x_{s}^{(l)}$ denotes the input tokens at layer $l$, and $\oplus$ represents the concatenation operation at the token level.

2.6 Training Scheme

Following previous style transfer works [13, 9, 7], we adopt a content loss ($\mathcal{L}_{c}$) to preserve source structure and a style loss ($\mathcal{L}_{s}$) to match target style statistics, both computed from the feature representations of a pre-trained VGG network. The objective function to optimize TRUST is defined as:

[TABLE]

3 Experimental results

3.1 Experimental setup

Dataset. We conduct experiments on three publicly available breast ultrasound datasets: BUSI [17], UDIAT [18], and UCLM [19]. Each dataset is randomly divided into training and testing subsets with a ratio of $7{:}3$. In each experimental setting, one dataset (e.g., BUSI) is assigned as the source domain, $\mathcal{D}_{S}$, while another serves as the target domain, $\mathcal{D}_{T}$ (e.g., UDIAT or UCLM). The proposed TRUST framework is trained on the source training set with style information transferred from the target training set, and evaluated on the source testing set. The testing set of the target domain is reserved for selecting the best downstream model during validation. Notably, the target labels are only used to train the downstream model.

Implementation details. All experiments are conducted on a single NVIDIA RTX $4090$ GPU with input images resized to $256\times 256$. In the labeled source set $D_{s}^{labeled}=\{(x_{s}^{i},y_{s}^{i})\}_{i=1}^{N_{labeled}}$, only $20$ samples per class are available. The entire TRUST framework is initialized using Xavier uniform initialization [20]. Both the source encoder $E_{s}$ and target encoder $E_{t}$ consist of $3$ Transformer [10] layers with $L_{p}=1024$ prompt tokens per layer and a patch size of $P=8$. We adopt the Adam optimizer with an initial learning rate of $5\times 10^{-4}$, following the warm-up adjustment strategy [21], and use a batch size of $B=6$. The total number of iterations is set to $20,000$. The classification downstream is ViT-B/16 [11], trained using SGD with a learning rate of $0.001$, momentum of $0.9$, weight decay of $0.0005$, and batch size of $16$. For the segmentation downstream, we employ SAMUS [12] with its default training configuration.

3.2 Comparison results

In this section, we provide quantitative and qualitative comparisons between the proposed TRUST and prior style transfer methods, including GAN-based approaches (CycleGAN [5], DiscoGAN [15]) and Transformer-based approaches (S2WAT [7], TransColors [6]). These comparisons demonstrate that TRUST effectively preserves the structural content of the source and injects suitable, task-optimal styles from the target domain. Surprisingly, TRUST achieves state-of-the-art results on $6$ cross-device ultrasound tasks.

Quantitative results. Table 1 reports the classification (Accuracy/AUC) and segmentation (Dice/IoU) performance before and after applying style transfer. In particular, TRUST substantially shows improvements compared to without style transfer (w/o ST) and GAN-based works. Furthermore, compared to the Transformer-based method TransColors, TRUST achieves notable gains of $+10.0\%$ in Accuracy and $+8.3\%$ in AUC on the UCLM $\rightarrow$ UDIAT classification task. In segmentation, it outperforms S2WAT under the UDIAT $\rightarrow$ BUSI setting, with improvements of $+3.6\%$ in Dice and $+4.4\%$ in IoU. These results indicate that S2WAT or TransColors relies heavily on one-to-one (source-target image) pairings, limiting their ability to capture the common target styles. Moreover, their naive style injection often introduces unwanted noise into the source content, resulting in degraded performance—even below w/o ST—on tasks, such as BUSI $\rightarrow$ UCLM and UDIAT $\rightarrow$ BUSI.

Qualitative results. In Figure 3, we present visual comparisons and Grad-CAM [16] results on the test set, revealing several key observations. First, TRUST outperforms TransColors in capturing diverse styles within a specific target domain. Its outputs clearly reflect this strength, showing the closest resemblance to the target samples in color and brightness. Second, TRUST generates finer tissue details and clearer tumor boundaries, highlighting the advantage of using reliable information. Third, TRUST improves semantic cues for downstream classification, as shown by Grad-CAM maps that focus more precisely on tumors and their boundaries, indicating more effective attention to diagnostically relevant regions. Overall, TRUST achieves precise style transfer and task-focused attention, providing clear advantages over conventional approaches that treat all target information equally.

4 Analysis

Ablation studies. Table 2 presents an ablation study across three cross-device settings to evaluate the effectiveness of each component in TRUST. Applying naive cross attention (CA) introduces noisy target tokens and can degrade performance—for example, on UCLM $\rightarrow$ UDIAT, classification accuracy drops from $63.8$ (w/o ST) to $60.0$. This supports our key motivation: “Not all information is knowledge.” Replacing CA with our Token-dRiven (TR) module leads to consistent improvements across all tasks. Compared to w/o ST, TR improves classification accuracy by $+3.1\%$, $+7.5\%$, and $+4.1\%$ for BUSI $\rightarrow$ UCLM, UCLM $\rightarrow$ UDIAT, and UDIAT $\rightarrow$ BUSI, respectively. Similar trends are observed in segmentation, where TR enhances Dice scores by up to $+0.8\%$ and IoU by up to $+0.6\%$ on the BUSI $\rightarrow$ UCLM task. Finally, integrating TR with auxiliary prompts further refines content representations. TRUST achieves the best overall performance, notably improving AUC from $92.5$ to $94.9$ and IoU from $76.2$ to $76.5$ on UDIAT $\rightarrow$ BUSI. These results confirm the complementary benefits of prompt tuning in guiding source tokens toward downstream predictions.

Feature space. Figure 4 presents t-SNE [22] visualizations of feature distributions from the downstream classifier (ViT-B/16) under the UDIAT $\rightarrow$ UCLM task. Before style transfer (Figure 4(a)), a clear domain gap exists between source samples ($\circ$) and target samples ($\star$), leading to potential misclassification in the absence of style adaptation (w/o ST). After applying TRUST (Figure 4(b)), the translated source features are better aligned with the target distribution. Through visualizing, we can conclude that TRUST facilitates domain-invariant feature learning while preserving class-discriminative structures essential for downstream tasks.

Computational cost. We further analyze the efficiency of TRUST in terms of floating-point operations (FLOPs). The overall complexity of TRUST amounts to $138.63$G FLOPs for an input resolution of $256\times 256$.

5 Conclusion

In this work, we present TRUST, a novel token-driven framework for structure-preserving style transfer in cross-device ultrasound analysis. TRUST introduces a content-aware mechanism that selectively integrates target style cues without distorting the anatomical structure of the source. Through extensive experiments on $6$ cross-domain tasks, TRUST consistently outperforms existing GAN-based and Transformer-based methods across both classification and segmentation, achieving state-of-the-art performance. Furthermore, ablation studies and t-SNE visualizations confirm that TRUST produces domain-invariant and class-discriminative features. These results demonstrate the potential of TRUST as a reliable solution for real-world deployment across heterogeneous ultrasound devices.

Acknowledgement. This work was financially supported in part (project number: 112UA10019) by the Co-creation Platform of the Industry Academia Innovation School, NYCU, under the framework of the National Key Fields Industry-University Cooperation and Skilled Personnel Training Act, from the Ministry of Education (MOE) and industry partners in Taiwan. It also supported in part by the National Science and Technology Council, Taiwan, under Grant NSTC-114-2218-E-A49 -024, - Grant NSTC-112-2221-E-A49-089-MY3, Grant NSTC-114-2425-H-A49-001, Grant NSTC-113-2634-F-A49-007, Grant NSTC-112-2221-E-A49-092-MY3, and in part by the Higher Education Sprout Project of the National Yang Ming Chiao Tung University and the Ministry of Education (MOE), Taiwan. It is also partly supported by MediaTek Inc., Hon Hai Research Institute, and Industrial Technology Research Institute.