Online percentile estimation (OPE)

Jorma Kilpi (Corresponding Author), Timo Kyntäjä, Tomi Räty

Research output: Contribution to journalArticleScientificpeer-review

Abstract

We describe a control loop over a sequential online algorithm. The control loop either computes or uses the sequential algorithm to estimate the temporal percentiles of any univariate input data sequence. It transforms an input sequence of numbers into output sequence of histograms. This is performed without storing or sorting all of the observations. The algorithm continuously tests whether the input data is stationary, and reacts to events that do not appear to be stationary. It also indicates how to interpret the histograms since their information content varies according to whether a histogram was computed or estimated. It works with parameter-defined, fixed-size small memory and limited CPU power. It can be used for a statistical reduction of a numerical data stream and, therefore, applied to various Internet of Things applications, edge analytics or fog computing. The algorithm has a built-in feasibility metric called applicability. The applicability metric indicates whether the use of the algorithm is justified for a data source: it works for an arbitrary univariate numerical input, but it is statistically feasible only under some requirements, which are explicitly stated here. We also show the results of a performance study, which was executed on the algorithm with synthetic data.
Original languageEnglish
Pages (from-to)1085-1100
Number of pages16
JournalJournal of Signal Processing Systems
Volume93
Issue number9
Early online date21 Jun 2021
DOIs
Publication statusPublished - Sep 2021
MoE publication typeA1 Journal article-refereed

Keywords

  • Internet of things
  • Online computation
  • Percentile estimation
  • Sequential estimation

Fingerprint

Dive into the research topics of 'Online percentile estimation (OPE)'. Together they form a unique fingerprint.

Cite this