A modified Lanczos Algorithm for fast regularization of extreme learning machines

Renjie Hu; Edward Ratner; David Stewart; Kaj Mikael Björk; Amaury Lendasse

doi:10.1016/j.neucom.2020.07.015

A modified Lanczos Algorithm for fast regularization of extreme learning machines

Renjie Hu, Edward Ratner, David Stewart, Kaj Mikael Björk, Amaury Lendasse

Research output: Contribution to journal › Article › peer-review

18 Scopus citations

Abstract

This paper presents a new regularization for Extreme Learning Machines (ELMs). ELMs are Randomized Neural Networks (RNNs) that are known for their fast training speed and good accuracy. Nevertheless the complexity of ELMs has to be selected, and regularization has to be performed in order to avoid underfitting or overfitting. Therefore, a novel Regularization is proposed using a modified Lanczos Algorithm: Iterative Lanczos Extreme Learning Machine (Lan-ELM). As summarized in the experimental Section, the computational time is on average divided by 4 and the Normalized MSE is on average reduced by 11%. In addition, the proposed method can be intuitively parallelized, which makes it a very valuable tool to analyze huge data sets in real-time.

Original language	English (US)
Pages (from-to)	172-181
Number of pages	10
Journal	Neurocomputing
Volume	414
DOIs	https://doi.org/10.1016/j.neucom.2020.07.015
State	Published - Nov 13 2020

Keywords

Classification
Extreme Learning machines
Lanczos Algorithm
Neural Networks
Regression
Regularization

ASJC Scopus subject areas

Computer Science Applications
Cognitive Neuroscience
Artificial Intelligence

Access to Document

10.1016/j.neucom.2020.07.015

Cite this

@article{d8f3e873954b4bb6ba1228fa1f00655d,

title = "A modified Lanczos Algorithm for fast regularization of extreme learning machines",

abstract = "This paper presents a new regularization for Extreme Learning Machines (ELMs). ELMs are Randomized Neural Networks (RNNs) that are known for their fast training speed and good accuracy. Nevertheless the complexity of ELMs has to be selected, and regularization has to be performed in order to avoid underfitting or overfitting. Therefore, a novel Regularization is proposed using a modified Lanczos Algorithm: Iterative Lanczos Extreme Learning Machine (Lan-ELM). As summarized in the experimental Section, the computational time is on average divided by 4 and the Normalized MSE is on average reduced by 11\%. In addition, the proposed method can be intuitively parallelized, which makes it a very valuable tool to analyze huge data sets in real-time.",

keywords = "Classification, Extreme Learning machines, Lanczos Algorithm, Neural Networks, Regression, Regularization",

author = "Renjie Hu and Edward Ratner and David Stewart and Bj{\"o}rk, \{Kaj Mikael\} and Amaury Lendasse",

note = "Publisher Copyright: {\textcopyright} 2020 Elsevier B.V.",

year = "2020",

month = nov,

day = "13",

doi = "10.1016/j.neucom.2020.07.015",

language = "English (US)",

volume = "414",

pages = "172--181",

journal = "Neurocomputing",

issn = "0925-2312",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - A modified Lanczos Algorithm for fast regularization of extreme learning machines

AU - Hu, Renjie

AU - Ratner, Edward

AU - Stewart, David

AU - Björk, Kaj Mikael

AU - Lendasse, Amaury

PY - 2020/11/13

Y1 - 2020/11/13

N2 - This paper presents a new regularization for Extreme Learning Machines (ELMs). ELMs are Randomized Neural Networks (RNNs) that are known for their fast training speed and good accuracy. Nevertheless the complexity of ELMs has to be selected, and regularization has to be performed in order to avoid underfitting or overfitting. Therefore, a novel Regularization is proposed using a modified Lanczos Algorithm: Iterative Lanczos Extreme Learning Machine (Lan-ELM). As summarized in the experimental Section, the computational time is on average divided by 4 and the Normalized MSE is on average reduced by 11%. In addition, the proposed method can be intuitively parallelized, which makes it a very valuable tool to analyze huge data sets in real-time.

AB - This paper presents a new regularization for Extreme Learning Machines (ELMs). ELMs are Randomized Neural Networks (RNNs) that are known for their fast training speed and good accuracy. Nevertheless the complexity of ELMs has to be selected, and regularization has to be performed in order to avoid underfitting or overfitting. Therefore, a novel Regularization is proposed using a modified Lanczos Algorithm: Iterative Lanczos Extreme Learning Machine (Lan-ELM). As summarized in the experimental Section, the computational time is on average divided by 4 and the Normalized MSE is on average reduced by 11%. In addition, the proposed method can be intuitively parallelized, which makes it a very valuable tool to analyze huge data sets in real-time.

KW - Classification

KW - Extreme Learning machines

KW - Lanczos Algorithm

KW - Neural Networks

KW - Regression

KW - Regularization

UR - https://www.scopus.com/pages/publications/85088917369

UR - https://www.scopus.com/inward/citedby.url?scp=85088917369&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2020.07.015

DO - 10.1016/j.neucom.2020.07.015

M3 - Article

AN - SCOPUS:85088917369

SN - 0925-2312

VL - 414

SP - 172

EP - 181

JO - Neurocomputing

JF - Neurocomputing

ER -

A modified Lanczos Algorithm for fast regularization of extreme learning machines

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this