An Empricial Study on the Quality of Algorithms for Dimensionality Reduction and Visualization of High-Dimensional Data

The following bachelor thesis explores and evaluates dimensionality reduction and

data visualization algorithms. Their objective is to find low-dimensional, compressed

representation of high-dimensional data sets with minimum information loss, where

analysis of raw data is beyond the capabilities of current software technologies.

As analysis of big data opens up new possibilities and challenges this leads to very

concentrated research efforts and a lot of innovation in the field recently. Therefore

there is a research gap for a very much needed, up-to-date comprehensive overview

of unsupervised dimensionality reduction techniques, which this papers fills.

Evaluation of suchlike techniques is very challenging task since this is an ill-posed

problem and there aren’t currently any good mathematical approaches. However,

humans’ visual system is extremely advanced and sophisticated, much more than any

existing algorithm, which is proven by the fact that identifying faces is something that

we do on daily basis, yet no algorithm can nearly come close to such accuracy. This is

why heuristic approach by visual analysis is generally taken for quality evaluation.

Important to note is that not only metric data has been tested, but a novel attempt

to visualize categorical data with dimensionality reduction techniques has been

successfully made where the user defines mapping function f : String æ Number.

Last but not least, a state-of-the-art web application has been conceptualized and

fully implemented where enduser without any technical knowledge is able to apply

dimensionality reduction and cluster analysis on his own data sets in a very simple,

intuitive way.