Bregman Proximal Gradient with extrapolation to train a Reservoir Computing network for a binary classification task

Mayora-Cebollero, Carmen (Universidad de Zaragoza) ; Mayora-Cebollero, Ana (Universidad de Zaragoza) ; Lozano, Álvaro (Universidad de Zaragoza) ; Barrio, Roberto (Universidad de Zaragoza)

Resumen: This study examines the training process of Artificial Neural Networks, specifically focusing on Reservoir Computing and on binary classification tasks. Training involves optimizing the network’s parameters to minimize the error of a loss function, which quantifies the discrepancy between network outputs and target data. Several numerical optimizers are used in the literature. Here, we formulate the supervised learning problem using the recently introduced Bregman Proximal Gradient with extrapolation (BPGe) algorithm for non-convex optimization problems. We compare this new method with the classical Stochastic Gradient Descent (SGD), the Root Mean Square Propagation (RMSProp), the Adaptive Moment Estimation (Adam), and two variants with Nesterov momentum (NAG and NAdam). The new approach leads to an accurate and significantly faster numerical algorithm for solving supervised learning problems on binary classification tasks in Reservoir Computing. Three test examples are presented, one based on chaotic data classification in the classical Lorenz system, another on the Human Activity Recognition (HAR) using smartphones dataset for movement–rest classification, and the last one on the MNIST dataset for even–odd classification. We show that our approach is highly competitive with existing methods in the tests performed.

Idioma: Inglés

DOI: 10.1016/j.ins.2026.123275

Año: 2026

Publicado en: Information Sciences 742 (2026), 123275 [25 pp.]
ISSN: 0020-0255

Financiación: info:eu-repo/grantAgreement/ES/AEI/PID2021-122961NB-I00
Financiación: info:eu-repo/grantAgreement/ES/DGA/E22-23R
Financiación: info:eu-repo/grantAgreement/ES/DGA/E24-23R
Financiación: info:eu-repo/grantAgreement/ES/DGA/LMP94_21
Financiación: info:eu-repo/grantAgreement/ES/MCINN/PID2024-156032NB-I00
Financiación: info:eu-repo/grantAgreement/ES/MICINN/PID2022-140556OB-I00

Tipo y forma: Article (Published version)

Área (Departamento): Área Geometría y Topología (Dpto. Matemáticas)
Área (Departamento): Área Matemática Aplicada (Dpto. Matemática Aplicada)