NewsReX: A More Efficient Approach to News Recommendation with Keras 3 and JAX

TL;DR

NewsReX is an open-source news recommendation library built on Keras 3 and JAX, offering increased computational efficiency and easier reproducibility for research using diverse datasets and hardware configurations.

Contribution

The paper introduces NewsReX, a modern, efficient implementation for news recommendation research that simplifies reproducibility and broadens accessibility across hardware platforms.

Findings

NewsReX outperforms existing implementations in speed.

It enables training on less powerful GPUs like RTX 3060 Ti.

Provides insights into training parameter effects and best practices.

Abstract

Reproducing and comparing results in news recommendation research has become increasingly difficult. This is due to a fragmented ecosystem of diverse codebases, varied configurations, and mainly due to resource-intensive models. We introduce NewsReX, an open-source library designed to streamline this process. Our key contribution is a modern implementation built on Keras 3 and JAX, which provides an increase in computational efficiency. Experiments show that NewsReX is faster than current implementations. To support broader research, we provide a straightforward guide and scripts for training models on custom datasets. We validated this functionality using a proprietary Japanese news dataset from Nikkei News, a leading Japanese media corporation renowned for its comprehensive coverage of business, economic, and financial news. NewsReX makes reproducing complex experiments faster and more accessible to a wider range of hardware making sure the speed up it also achieved for less powerful GPUs, like an 8GB RTX 3060 Ti. Beyond the library, this paper offers an analysis of key training parameters often overlooked in the literature, including the effect of different negative sampling strategies, the varying number of epochs, the impact of random batching, and more. This supplementary analysis serves as a valuable reference for future research, aiming to reduce redundant computation when comparing baselines and guide best practices. Code available at https://github.com/igor17400/NewsReX.

Tables

Table 1. Table 1 . Performance comparison of news recommendation models on MIND-small (Wu et al., 2020 ) and Japanese proprietary datasets.

Dataset	Model	nDCG@10	nDCG@5	AUC	MRR
MIND-small	NRMS	40.58	34.28	65.63	30.98
	NAML	41.65	35.34	66.53	31.91
	LSTUR	40.43	33.99	65.98	30.74
Nikkei News	NRMS	37.77	28.45	59.38	22.53
	NAML	36.14	26.35	56.93	21.49
	LSTUR	38.98	28.97	58.81	23.96

Table 2. Table 2 . Model Performance Comparison: Random vs Chronological Validation Set Composition

Model	Validation Set	NDCG@10	NDCG@5	MRR	AUC
NRMS	Random	40.04	33.63	30.70	65.16
	Chronological	38.64	32.22	29.45	63.52
NAML	Random	41.52	35.25	31.78	66.96
	Chronological	40.25	33.90	31.08	64.83
LSTUR	Random	40.19	33.84	30.66	65.92
	Chronological	39.69	33.38	30.51	65.02

Table 3. Table 3 . Model Performance Comparison: Negative Sampling Strategies

Model	Sampling Strategy	NDCG@10	NDCG@5	MRR	AUC
NRMS	Random	40.36	33.99	31.04	65.07
	Unshuffled	40.83	34.54	31.45	65.78
NAML	Random	41.35	34.96	31.78	66.45
	Unshuffled	41.76	35.47	32.02	66.97
LSTUR	Random	40.82	34.49	31.29	66.24
	Unshuffled	40.97	34.70	31.30	66.50

Table 4. Table 4 . Model Performance Comparison: Varying Batch Size

Model	Batch Size	NDCG@10	NDCG@5	MRR	AUC
NAML	256	40.29	33.92	30.67	65.75
	128	40.90	34.62	31.18	66.26
	64	41.16	34.87	31.40	66.59
	32	41.32	35.03	31.60	66.73
	16	41.29	34.90	31.53	66.42
NRMS	256	39.29	32.83	30.07	64.41
	128	39.50	33.10	30.22	65.02
	64	39.91	33.50	30.58	65.32
	32	40.62	34.34	31.15	65.91
	16	40.83	34.54	31.45	65.79

Table 5. Table 5 . NRMS Model Performance: Varying the Number of Epochs

Epochs	NDCG@10	NDCG@5	MRR	AUC
10	40.83	34.54	31.45	65.79
20	40.83	34.54	31.45	65.78
40	40.83	34.54	31.44	65.79
80	40.82	34.53	31.44	65.78

Equations

$$s_i$$

$$a_i$$

$${\mathbf{e}}_n=\sum _{i=1}^L{a}_i{\mathbf{e}}_{w_i}$$

$$\text{Attention}({\mathbf{Q}}_u,{\mathbf{K}}_H,{\mathbf{V}}_H)=\text{softmax}\left(\frac{{\mathbf{Q}}_u{\mathbf{K}}_H^{\top }}{\sqrt{d_k}}\right){\mathbf{V}}_H$$

Full text

News Recommendation, Open-Source Library, News Modeling, Keras 3 & JAX

††copyright: acmlicensed††journalyear: 2018††doi: XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY††isbn: 978-1-4503-XXXX-X/2018/06††ccs: Information systems Personalization

1. Introduction

In recent years, neural news recommendation (NNR) has become the cornerstone of modern news recommender systems. The ability of NNR models to accurately capture and model user behavior has proven to be far superior to traditional methods like matrix factorization.

However, despite the abundance of novel NNR model designs, the field still faces significant challenges. A major issue is the lack of research reproducibility, as many NNR implementations are not publicly released. Even for open-source repositories, a lack of transparency regarding evaluation datasets, experimental setups, and hyperparameter settings severely hurts direct and fair model comparisons (Iana et al., 2023). Consequently, it is difficult for researchers to evaluate the impact of specific architectural components or training decisions on a model’s overall performance.

To address these limitations, several open-source frameworks for news recommender systems have been proposed. Notable examples include the Microsoft Recommenders library (Graham et al., 2019) built on TensorFlow (Abadi et al., 2015) and, more recently, NewsRecLib (Iana et al., 2023). Built on PyTorch-Lightning (Falcon and The PyTorch Lightning team, 2019) and Hydra (Yadan, 2019), NewsRecLib provides a unified and highly configurable framework designed to promote reproducible research and rigorous experimental evaluation. It is also worth mentioning the JAX-recommenders library 333https://github.com/google/jax-recommenders, which, while using JAX for recommender systems, does not provide the same level of support for NNR as the aforementioned libraries.

Both open-source libraries, NewsRecLib and Recommenders by Microsoft, offer out-of-the-box implementations of several prominent neural models for news recommendation, including classic architectures like NRMS (Wu et al., 2019c), NAML (Wu et al., 2019a), and LSTUR (An et al., 2019). The primary objective of these libraries is to provide a consistent and reproducible framework, which simplifies the replication of experimental results and enables fair comparisons between models.

Following this direction, NewsReX is an open-source library dedicated to facilitating the reproduction of NNR experiments. However, NewsReX distinguishes itself from existing frameworks, as far as we are concerned, by possessing the following key properties:

• •

High Efficiency with JAX and Keras 3: NewsReX is the first NNR library to be built with an optimization-oriented focus, leveraging the high-performance capabilities of Keras 3.0 (Chollet et al., 2015) and JAX (Bradbury et al., 2018). This provides a significantly faster and more efficient environment, making it accessible for researchers with consumer-grade GPUs, such as the RTX 3060 Ti, and addressing the high computational costs of NNR models.

• •

Extensive Analysis and Visualization: In addition to standard evaluation protocols, NewsReX provides extensive tools for dataset and recommendation analysis, including visualizations. These features are particularly useful when working with proprietary datasets where transparency and interpretability are crucial.

In addition to these distinctive features, NewsReX adopts similar design principles to NewsRecLib and Recommenders by Microsoft, ensuring a familiar and powerful user experience:

• •

Hydra-based Configurability: Like NewsRecLib, NewsReX is powered by Hydra, allowing experiments to be defined and configured through a single, modular file. This design ensures that all experimental settings are automatically stored, guaranteeing reproducibility.

• •

Multilingual Support: The library includes out-of-the-box support for multilingual pre-trained embeddings, such as GloVe (English support) (Pennington et al., 2014) and BPEmb (275 languages supports) (Heinzerling and Strube, 2018).

• •

Comprehensive Logging: Integration with Weights & Biases (commonly referred as ”wandb”) (Biewald, 2020) facilitates extensive logging and monitoring of experiments, including losses, metrics, and hyperparameter settings.

• •

Easy Customization: NewsReX is designed with a modular architecture, making it easy to interchange model components and integrate new ones, thereby simplifying model customization and experimentation.

In summary, NewsReX is an accessible and comprehensive framework that focuses on the latest NNR architectures, provides practical visualizations for industry applications, and, most importantly, offers a more efficient environment to support reproducible research for a wider range of researchers, especially those with limited access to expensive computational resources.

2. NewsReX Framework

NewsReX is built upon a component-based architecture, prioritizing modularity and extensibility. This design principle ensures that every model and dataset in the library inherits from a foundational base class, which not only simplifies long-term maintenance but also significantly streamlines the implementation of new models. Following the standard methodology established by foundational NNR models such as NRMS (Wu et al., 2019c), NAML (Wu et al., 2019a), and LSTUR (An et al., 2019), every recommendation model in our framework is composed of two primary components: a User Encoder and a News Encoder.

2.1. Encoders

2.1.1. News Encoder

The News Encoder ($NE$) is responsible for generating a fixed-size vector representation, or embedding, for a given news article. This embedding, denoted as $\mathbf{e}_{n}$, aims to capture the semantic and topical content of the article. A common approach involves processing the news title, abstract, or other textual features.

Mathematically, let a news title be represented as a sequence of word embeddings, $\mathbf{E}_{T}=[\mathbf{e}_{w_{1}},\mathbf{e}_{w_{2}},\dots,\mathbf{e}_{w_{L}}]$, where $L$ is the length of the title. The News Encoder then maps this sequence to a single dense vector $\mathbf{e}_{n}$. A popular and effective method is to use a text encoder, such as a convolutional neural network (CNN) (Qi et al., 2022; Wu et al., 2019a) or a multi-head self-attention network (Wu et al., 2019c), followed by an additive attention mechanism to aggregate the word embeddings.

The attention mechanism learns a context vector $\mathbf{v}_{n}$ and uses it to compute an attention weight $a_{i}$ for each word embedding $\mathbf{e}_{w_{i}}$:

[TABLE]

The final news embedding $\mathbf{e}_{n}$ is the weighted sum of the word embeddings:

[TABLE]

More advanced News Encoders may use pre-trained language models (PLMs) to generate the initial word representations, which are then further processed to produce the final news embedding (Gao et al., 2024; Yu et al., 2022; Iana et al., 2023).

2.1.2. User Encoder

The User Encoder ($UE$) is a key component that generates a representation for a user, denoted as $\mathbf{u}$. This representation is learned by aggregating the embeddings of the news articles a user has previously clicked on. The user’s historical click behavior is a sequence of news articles, $H_{u}=\{n_{1},n_{2},\dots,n_{K}\}$, where $K$ is the number of clicked news articles in their history.

For each news article $n_{i}$ in the user’s history, we first obtain its embedding $\mathbf{e}_{n_{i}}$ using the News Encoder. The User Encoder then takes these historical news embeddings, $\mathbf{E}_{H}=[\mathbf{e}_{n_{1}},\mathbf{e}_{n_{2}},\dots,\mathbf{e}_{n_{K}}]$, as input to produce the user embedding $\mathbf{u}$. A common approach is to use an attention network to selectively focus on the most relevant historical news items when creating the user representation.

Similar to the News Encoder, a multi-head self-attention mechanism is often employed to model user preferences. The user embedding $\mathbf{u}$ is derived as a weighted sum of the historical news embeddings, where the weights are determined by an attention mechanism that can be influenced by a candidate news item $\mathbf{e}_{c}$ to produce a ”candidate-aware” user representation.

Let $\mathbf{Q}_{u}$ be the user’s attention query, and $\mathbf{K}_{H}$ and $\mathbf{V}_{H}$ be the keys and values derived from the historical news embeddings. The attention-based user embedding is:

[TABLE]

where $d_{k}$ is the dimension of the key vectors. The final user embedding $\mathbf{u}$ is the output of this attention mechanism. Other user encoders may employ recurrent neural networks (RNNs) (An et al., 2019) or graph neural networks (GNNs) (Yang et al., 2023; Ko et al., 2025), to capture temporal dependencies or relationships between news articles.

2.1.3. Implementation

Following the modular design, the News and User Encoders are implemented as independent components. The News Encoder for models like NRMS processes news titles using embeddings, multi-head self-attention, and additive attention. Similarly, the User Encoder in NRMS encodes user history via time-distributed news encoding and a self-attention mechanism. This component-based approach allows for different encoder implementations to be swapped out easily; for example, NAML uses separate encoders for news title, abstract, and categories, while LSTUR employs GRU components to model sequential user behavior.

2.2. Model Architecture and Inheritance

The system’s core architecture is built around a sophisticated inheritance hierarchy centered on the BaseModel class. This class provides a unified interface and essential functionalities for all recommendation models, including precomputed vector-based evaluation and caching of both news and user representations to avoid redundant computations.

The diagram 1 illustrates how specific models like NRMS, NAML, and any other state-of-the-art models models inherit from the BaseModel, leveraging its core capabilities while implementing their own unique encoder and scoring components.

2.3. Dataset Handling and Preprocessing

NewsReX provides robust, multilingual support for datasets, including the English MIND dataset (Wu et al., 2020) and several multilingual news datasets based on the support given by BPEmb (Heinzerling and Strube, 2018). This is achieved through a three-tier inheritance hierarchy that ensures a consistent interface for different languages and data formats as illustrated in figured 2. The multilingual capabilities were demonstrated on a proprietary Nikkei News dataset from Nikkei News, a multinational news platform based in Japan.

The system handles language-specific preprocessing, such as tokenization and normalization, and supports various embedding strategies like GloVe for English and BPEmb for multilingual scenarios. To optimize I/O and loading times, the data pipeline uses a pickle-based caching system that serializes processed datasets, vocabularies, and embedding matrices for quick retrieval.

2.4. Experiments Configuration

Following the paradigm set by NewsRecLib (Iana et al., 2023), every model, dataset, and experiment is defined through a dedicated configuration file. This system is managed by the Hydra framework, which allows for hierarchical configuration and runtime overrides. The overall idea is illustrated by the figure 3.

The configuration system is organized into distinct categories. As a high-level example, figure 4 illustrates a subset of the parameters a user can configure for the NRMS model applied to our proprietary Nikkei News dataset. This structure enables researchers to define a complete experimental setup by composing a single configuration file from different component-specific files.

2.5. Efficiency

As aforementioned, NewsReX is built on top of Keras 3.0 and JAX. The JAX backend enables significant performance optimizations through its Just-In-Time (JIT) compilation and XLA (Accelerated Linear Algebra) optimization, which accelerates numerical computations on GPUs. The framework employs an intelligent JIT warmup strategy to trigger compilation with sample data at the beginning of a run, ensuring optimal performance from the start as we will demonstrate in section 3 below. In addition, the precomputation of both news and user vectors, along with efficient vectorization and memory management, drastically reduces computational overhead during training and evaluation.

2.6. Overall Architecture

NewsReX represents a sophisticated, production-ready news recommendation system that successfully balances modularity, performance, and extensibility. Key architectural strengths include:

1. (1)

Clean Separation of Concerns: Models, datasets, and utilities are well-isolated. 2. (2)

Performance Optimization: The JAX backend with JIT compilation delivers high performance. 3. (3)

Multilingual Support: A robust architecture accommodates different languages and datasets. 4. (4)

Configuration Management: Hydra enables flexible experiment management. 5. (5)

Precomputation Strategy: Intelligent caching reduces computational overhead. 6. (6)

Extensible Design: It is easy to add new models, datasets, and features.

The system demonstrates best practices in modern code engineering, with careful attention to performance, maintainability, and scientific reproducibility. The modular architecture facilitates both research experimentation and production deployment scenarios.

3. Experiments

This section demonstrates the use of the NewsReX framework to train three prominent neural news recommendation (NNR) models—NRMS (Wu et al., 2019c), NAML (Wu et al., 2019a), and LSTUR (An et al., 2019)—on two distinct datasets: the public MIND dataset (Wu et al., 2020) and our proprietary Nikkei News dataset. We present a series of experiments designed to highlight the framework’s efficiency and visualization capabilities. Additionally, we provide a list of ablation studies with the ultimate goal of establishing a comprehensive reference for future researchers. This reference aims to assist them in making informed experimental decisions, thereby conserving valuable time and computational resources, and enabling a thorough analysis of how various architectural and training choices affect model performance.

3.1. Dataset Results

In this section, we conducted a series of experiments to evaluate the models using standard accuracy-based metrics commonly employed in previous research (Wu et al., 2019c, a; An et al., 2019; Yang et al., 2023; Qi et al., 2022; Iana et al., 2023; Graham et al., 2019). These metrics include AUC (Area Under the Curve), MRR (Mean Reciprocal Rank), and nDCG@k (Normalized Discounted Cumulative Gain at k) for $k=5$ and $10$.

The performance results for the NRMS, NAML, and LSTUR models on both the MIND-small and proprietary Nikkei News dataset are presented in Table 1. To ensure the statistical reliability of our findings, all experiments were executed at least three times, and the reported scores represent the average of these runs. Our experimental setup was standardized across all models and datasets to allow for a fair comparison. We used a batch size of 16, a floating-point precision of 32 , and learning rate of 0.0001. All models were trained for a minimum of 5 epochs. For hardware, all experiments for this section were run on an NVIDIA A100 GPU to leverage its advanced computational capabilities.

3.1.1. Metrics Calculations

A distinguishing feature of NewsReX is its approach to computing these evaluation metrics. Instead of relying on external libraries like Scikit-learn, the framework utilizes a custom, JAX-optimized metrics calculator built for high performance and GPU acceleration.

The NewsRecommenderMetricsOptimized class provides the implementation of these metrics. The core of its efficiency lies in a pure JAX implementation, which enables several key optimizations:

• •

Pure JAX Implementation: All metric calculations, including AUC, MRR, and nDCG, are implemented directly in JAX.

• •

JIT Compilation: Each metric function is decorated with jax.jit, which enables Just-In-Time compilation. This compiles the Python function into an optimized, low-level XLA representation, resulting in significant speedups for repetitive computations during training and evaluation.

• •

Vectorized Batch Processing: The framework leverages jax.vmap to automatically vectorize metric calculations. This allows the system to compute metrics for an entire batch of impressions in a single, highly efficient operation.

3.2. Experimental Performance Analysis

To provide a comprehensive comparison of experimental efficiency, we benchmarked NewsReX against NewsRecLib. All experiments were conducted under a standardized setup: 32-bit floating-point precision, a batch size of 16, with learning rate 0.0001, and were run for 5 epochs on the MIND-small dataset for the three classic NNR models: NAML, NRMS, and LSTUR in a consumer NVIDIA GPU model 8GB RTX 3060 Ti. Our analysis focuses on three key metrics: training time per epoch, average training time per epoch, and total experiment time.

3.2.1. Per-Epoch Performance

Figure 5, which compares the individual model training time per epoch, reveals a key difference in the underlying execution strategies of the two frameworks. NewsRecLib, built on PyTorch-Lightning, demonstrates a relatively stable training time from the first epoch to the last. In contrast, NewsReX, which leverages the JAX backend for Keras 3, exhibits a steep decrease in training time after the initial epoch for all three models. This behavior is characteristic of JAX’s Just-In-Time (JIT) compilation. During the first epoch, JAX compiles the computational graph into an optimized low-level XLA representation. This compilation process adds an initial overhead, but for all subsequent epochs, the compiled graph is reused, leading to a significant speedup. This demonstrates a key advantage of the JAX-based architecture for long-running experiments or when training for many epochs.

3.2.2. Average Training Time and Overall Speedup

The efficiency gains of NewsReX are clearly quantified in the average training time per epoch comparison (Figure 6) and the total experiment time distribution (Figure 7).

For NAML, NewsRecLib averaged 11.66 minutes per epoch, while NewsReX completed each epoch in an average of 7.82 minutes. This results in a notable total time savings, with NewsReX being 3.1% faster overall for this model. The performance difference is even more pronounced for the NRMS model. NewsRecLib averaged 9.38 minutes per epoch, whereas NewsReX reduced this to just 3.74 minutes. This significant reduction translates to a 41.0% speedup in total experiment time, completing the task in 39.0 minutes compared to NewsRecLib’s 66.2 minutes. Similarly, the LSTUR model saw substantial gains. NewsRecLib averaged 10.73 minutes per epoch, while NewsReX averaged only 4.55 minutes. The total experiment time for LSTUR was reduced from 73.5 minutes to 46.3 minutes, marking a 37.0% speedup.

Additionally, we conducted a performance comparison with the Recommenders library by Microsoft (Graham et al., 2019), which previously served as a benchmark for implementations of NRMS, NAML, and LSTUR. Our approach to fast_evaluate, which precomputes user and news encoders for validation and testing, was inspired by this library’s methodology. However, due to its last significant update being approximately two years ago, its implementation is considerably less performant than NewsReX, with experiments taking around 60 minutes per epoch. This considerable performance gap further highlights the efficiency benefits of our modern, JAX-based architecture.

The performance-first design of NewsReX, which prioritizes JAX-based optimizations like JIT compilation and vectorized operations, is the primary driver of these results. This architectural choice makes NewsReX a highly efficient framework, particularly for researchers with limited access to expensive computational resources.

3.3. Visualizations

A key objective of NewsReX is to provide practical, industry-focused tools that extend beyond traditional performance metrics like AUC, MRR, and nDCG@k. A highly intuitive approach to this is through visualizations that provide deeper insights into both dataset characteristics and model performance.

3.3.1. Dataset Visualizations

To better understand the distribution of news articles within a dataset, we provide a heatmap visualization that captures article exposure and click behavior. This visualization displays three pieces of information simultaneously: the total news article population, and two key subsets—the bottom 95% of articles based on the number of clicks and the top 5%. This allows for a clear analysis of article behavior in terms of total impressions (x-axis), total clicks (y-axis), and click-through rate (CTR), which is represented by color and size. This analysis is performed on the test set, where our models are evaluated for the MIND-small and proprietary Nikkei News dataset.

As shown in Figure 8, the analysis reveals a highly skewed distribution in both datasets. The ”Full Dataset” and ”Bottom 95%” plots for both MIND-small and the Nikkei News dataset demonstrate that the vast majority of articles receive a very low number of clicks and impressions, forming a concentrated cluster in the bottom-left corner. This heavy-tailed distribution indicates that a small percentage of articles are highly popular, while the rest constitute a long tail. Conversely, the ”Top 5%” plots highlight the behavior of these popular articles. These articles exhibit a wide range of impressions and clicks. By providing such detailed visualizations, NewsReX aims to make the analysis of proprietary datasets more intuitive and accessible

3.3.2. Model Recommendation Visualization

Beyond understanding the dataset, it is crucial to analyze how a recommendation model’s output aligns with user behavior. To achieve this, NewsReX provides a treemap visualization that compares the distribution of a user’s actual clicks (ground truth) with the model’s top recommendations. The accompanying image, which analyzes the performance of the NRMS model, displays the distribution of subcategories for a given user. Each box in the treemap represents a subcategory, and its size corresponds to the number of clicks or recommendations within that subcategory.

As shown in Figure 9, the left panel, ”Ground Truth Clicks,” reveals the user’s primary interests are in football_nfl (20,911 clicks), newsus (8,501 clicks), and newscrime (5,520 clicks). The right panel, ”Recommended Items,” shows the NRMS model’s top recommendations for this user for MIND-small dataset. A direct comparison of the two panels highlights a clear bias in the model’s recommendations. While the model correctly identifies football_nfl as a top interest, it over-recommends it (27,951 items vs. 20,911 clicks). The model also over-recommends newscrime and

lifestyleroyals, while significantly under-recommending newsus.

This visualization provides an intuitive and actionable insight into the model’s behavior, allowing a researcher to quickly assess its alignment with user interests and identify specific biases that may not be apparent from aggregate metrics alone.

4. Ablation Studies

For the next sections we expose some studies we had done which we believe can provide information towards future research directions to provide some directions that could spare time.

4.1. Validation Set - Random VS Chronological

When constructing the validation set, we can adopt two distinct approaches: random or chronological splitting. The random approach, a conventional method in machine learning, involves randomly partitioning the training data, typically allocating 95% for training and a random 5% for validation. Conversely, the chronological approach partitions the data based on time. For example, a dataset might be split with training data up to a specific timestamp, followed by a time-consecutive validation set, and a final, later period reserved for testing.

As shown in Table 2, our experiments reveal that the random validation approach consistently yields better performance across all three models (NRMS, NAML, and LSTUR) when evaluated on the test set. A plausible explanation for this outcome is that the random split provides the model with a more diverse exposure to news articles and user behaviors across the entire timeline of the dataset. This broad exposure allows the model to learn more robust, generalized trends, whereas a chronological split may lead to models that overfit to patterns present only within a specific, limited time window. It is important to highlight that under any scenario the model has been exposed to samples from the test dataset.

4.2. Negative Sample Space

In the majority of neural recommender models, negative sampling is a fundamental technique used during training (Wu et al., 2019b; Qi et al., 2022, 2021; Wu et al., 2019c). This approach addresses the implicit feedback nature of recommender systems by creating a contrastive learning objective. By pairing positive examples (i.e., news articles a user has clicked) with a selection of negative examples (articles not clicked), the model is trained to assign a higher prediction score to positive items than to negative ones. This transforms the ranking problem into a learning task with well-defined targets.

The selection of these negative examples is a critical area of research with distinct methodologies across various domains of recommender systems (Shi et al., 2023; Ma et al., 2023). For our experiments, we implement two simple yet powerful negative sampling strategies to evaluate their impact on model performance. The first strategy, which we refer to as Random or Shuffled, involves randomly selecting $k$ negative articles from an impression list for each positive article. The order of these items is then shuffled to prevent the model from learning a positional bias. The second strategy, which we call Unshuffled or Not Random, also selects $k$ negative items, but the positive item is always placed at the first position within the sample. We then compare the performance of the NRMS, NAML, and LSTUR models under both strategies. All experiments were conducted for 5 epochs with a batch size of 16, and results are shown in Table 3

4.3. Varying the batch size

We performed a series of experiments to investigate the impact of different batch sizes on model performance. The models were trained for 10 epochs with batch sizes ranging from 16 to 256. The results, summarized in the table 4, show that for both NAML and NRMS, a smaller batch size generally leads to better performance. The best results were achieved with a batch size of 32 for NAML and 16 for NRMS except for AUC score.

4.4. Varying the number of epochs

To investigate the impact of training duration on model performance, we conducted an experiment with the NRMS model, varying the number of epochs from 10 to 80. The results, summarized in Table 5, demonstrate that the model reaches a performance plateau relatively early in the training process. Beyond 10 to 20 epochs, there is no significant improvement in any of the reported metrics. This suggests that for the NRMS model, training for an extended number of epochs does not yield better results. These findings reinforce the importance of early stopping and efficient training regimes to conserve computational resources.

5. Future Steps

Based on the current architecture and the broader trends in news recommendation, we have identified several key areas for future development:

• •

Include more multilingual datasets: Expand the library support for a wider range of multilingual news datasets to enable researchers to conduct cross-lingual and cross-cultural studies more effectively.

• •

Include the latest news models for the last 10 years: The current library includes the most traditional foundational NNR models (NRMS, NAML, and LSTUR), but we plan to integrate a more comprehensive selection of state-of-the-art models from the last decade to provide a richer historical and comparative context for new research.

• •

Include live metrics for visualization of recommendation through time during testing: Develop real-time visualization tools that allow researchers to monitor key recommendation metrics during the testing phase. This will provide dynamic insights into model performance and help in identifying potential issues as they occur.

• •

Add support to LLM integration for the recent trend of recsys + LLMs: Given the emerging trend of integrating large language models (LLMs) with recommender systems, we plan to add native support for LLM-based news encoders and user modeling techniques within the NewsReX framework.

6. Conclusion

We introduced NewsReX, a novel open-source library for neural news recommendation (NNR) designed to enhance reproducibility and computational efficiency. Built on Keras 3 and JAX, NewsReX offers a performance-first, modular architecture that significantly reduces training time compared to existing frameworks. The framework includes key features such as a pure JAX-optimized metrics calculator for faster evaluation, a comprehensive and extensible codebase for easy model development, and a Hydra-based configuration system to ensure reproducibility. Additionally, the paper provides a series of ablation studies on the MIND-small dataset, analyzing the effects of different validation set compositions (random vs. chronological), negative sampling strategies (shuffled vs. unshuffled), and varying batch sizes and epochs. NewsReX also features visualizations to provide deeper insights into dataset biases and model behavior, which are particularly valuable for practical, industry-oriented applications. The overall design of NewsReX promotes faster, more accessible, and interpretable NNR research. The code is available at https://github.com/igor17400/NewsReX and will be uploaded to GitHub upon conference acceptance.

7. Ethical Considerations

The development and deployment of recommender systems, particularly in the news domain, inherently involve significant ethical considerations, with user privacy being a huge concern. These systems often rely on extensive user data, including reading history and behavioral patterns, which can lead to privacy risks. While NewsReX is a research-oriented framework, it operates on these sensitive datasets, and its design could benefit from an explicit focus on addressing these issues. Future iterations of the library could be enhanced by incorporating dedicated modules for privacy-preserving techniques.

Acknowledgements.

This work is partially supported by JSPS KAKENHI Grant Number 23K28098.