There has been extensive research on dimensionality reduction techniques. While these 1 make it possible to present visually the high-dimensional data in 2D or 3D, it remains a challenge 2 for users to make sense of such projected data. Recently, interactive techniques, such as Feature 3 Transformation, have been introduced to address this. This paper describes an user study that was 4 designed to understand how the feature transformation techniques affect user's understanding of 5

more »

... onal data visualisation. It was compared with the traditional dimension reduction 6 techniques, both unsupervised (PCA) and supervised (MCML). Thirty-one participants were 7 recruited to detect visually clusters and outliers using visualisations produced by these techniques. 8 Six different datasets with a range of dimensionality and data size were used in the experiment. Five 9 of these are benchmark datasets, which makes it possible to compare with other studies using the 10 same datasets. Both task accuracy and completion time were recorded for comparison. The results 11 show that there is a strong case for the feature transformation technique. Participants performed best 12 with the visualisations produced with high-level feature transformation, in terms of both accuracy 13 and completion time. The improvements over other techniques are substantial, particularly in the 14 case of the accuracy of the clustering task. However, visualising data with very high dimensionality 15 (i.e., greater than 100 dimensions) remains a challenge. 16 reduction 18 20 to help users better understand the Big Data they have. A large portion of the Big Data is high 21 dimensional and is notoriously difficult for humans to comprehend because of the lack of physical 22 analogy of data with more than three dimensions. Various dimension reduction techniques have been 23 developed to reduce the data dimensions, so they can be visually displayed [1,2]. Dimensionality 24 Reduction (DR) techniques such as Principal Component Analysis (PCA) and Multidimensional 25 Scaling (MDS) allow analysts to project multidimensional data to a lower dimensional (2D or 3D) 26 visual display as scatterplot diagrams where patterns such as groups and outliers can be easily 27 identified. The approach is widely used for explorative analysis of large information spaces. However, most of these techniques are not designed for human perception, but rather optimising 29 for certain metrics such as minimising the distance distortion after the projection. While these 30 techniques have been shown to be very useful, they inadvertently introduced difficulties for data 31 visualisation and sense making in lower dimensions such as visual cluttering that affects the 32 interpretation of a projection. Moreover, with increasing dimensionality and noise in the data, 33 such methods become less effective due to the curse of the dimensionality problem [3]. When 34 the dimensionality is high, the distance measure becomes less meaningful as all objects tend to be 35 similar and dissimilar in many ways, leading to points being projected to similar locations in the 36 projection space (over-plotting problem). Given a particular pattern recognition task, often not all 37 the recorded information is relevant. The irrelevant information will obscure the patterns in the 38 visualization, leading to blurred group boundaries and patterns being hidden behind overlapping 39 group boundaries. A recent study by Etemadpour et al. [4] compared five different DR techniques 40 from the user perception perspective, and the results confirmed the two issues discussed earlier. 41 Recently, there have been a number of works that aim to improve the existing dimension 42 reduction techniques by producing more understandable visualisation or allowing user interaction 43 during the process [5-10]. These are later summarized by Sacha et al. in their survey [11]. Among 44 these, one approach is to use a supervised DR technique that employs class labels to compute the 45 projection. Supervised DR helps improve visual clarity of projections but an uncluttered projection 46 can hardly be guaranteed. On the other hand for explorative analysis, it is important to gain 47 an overview of the data before detailed analysis [12]. Schaefer et al. [8] proposed a feature 48 transformation approach that can be applied in conjunction with any existing DR technique to reduce 49 the over-plotting problem and improve group separation in the visual space. The essential idea is to 50 integrate prior knowledge in the projection process by extending certain features in the original data 51 space before projection to achieve projections that better reveal hidden patterns in the data. Schaefer's 52 work is further extended by Pérez et al. [9,13] where interactive visualizations are proposed to 53 provide analysts with more flexibility and user control over the feature transformation process. 54 Although the feature transformation approach "distorts" the original feature space to a certain 55 degree, testing results in both Schaefer's and Pérez's work demonstrate a good compromise can 56 often be made between maintaining the original characteristics of the data and achieving better visual 57 clarification in the final projection. This was demonstrated through the assessment of the projections 58 using quality measures that showed an improvement of visual overlapping with a small variation 59 of the structural preservation. However, both works do not include user studies that evaluate the 60 effectiveness of the feature transformation approach from the perspective of user perception and 61 comprehension. 62 This paper describes an experiment studying the effectiveness of feature transformation 63 techniques in supporting analysts making sense of high-dimensional data. The participants were 64 asked to perform common analysis tasks, i.e., cluster and outlier identification, using 2D projection 65 (i.e., visualisation) produced by feature transformation and other DR methods. The experiment used 66 a number of benchmark datasets that cover a wide range of size and dimensionality. Both task 67 accuracy and completion time were recorded, and the result analyses show significant difference 68 among these methods. 69 The remainder of the paper is organised as follows: Section 2 provides a more complete and 70 in-depth discussion on the existing work related to the study. The details of the feature transformation 71 are described in Section 3. This is followed by experiment design, hypotheses, data sets and protocol 72 (Section 4). The experiment results are reported in Section 5, followed by in-depth discussions in 73 Section 6. Section 7 concludes the paper. 74 2. Related Work 75 An extensive range of DR techniques exist [1] that estimate the structure of data in a low 76 dimensional space. Classical methods such as Principal Component Analysis (PCA) [14] or 77 Multidimensional Scaling (MDS) [15] are based on linear approaches. Later non-linear techniques 78 were developed, for example Sammon proposed a version of the MDS algorithm [16] to compute 79 a projection that is able to represent non-linear structures in the data. In the beginning of the 80 21 st century, newer non-linear techniques, based on neighbour embedding, were proposed. These 81 algorithms compute a manifold in a low-dimensional space from high dimensional data with an 82 underlying structure. Some of the best known examples are isometric embedding mapping or 83 Isomap [17], Laplacian Eigenmaps (LE) [18], locally linear embedding (LLE) [19], local tangent 84 subspace alignment (LTSA) [20] and t-Distributed Stochastic Neighbour Embedding (t-SNE) [21]. 85 Moreover there are methods that use class information to guide the computation of the 86 projection, that is, supervised dimensionality reduction. Available supervised methods include the 87 Linear Discriminative Analysis (LDA) [22] that extracts the discriminative features to the class labels 88 and uses them to generate embedding, the Neighborhood Components Analysis (NCA) [23] that learns 89 a distance metric by finding a linear transformation of input data such that the average classification 90 performance is maximized in the projection space, and the Maximally Collapsing Metric Learning 91 (MCML) [24] that aims at learning a distance metric that tries to collapse all objects in the same class 92 to a single point and push objects in other classes far away. 93 DR techniques estimate the underlying structure and reveal relationships in multidimensional 94 data. However, due to noise and irrelevant attributes, a satisfactory projection is not always 95 obtained. Feature selection and transformations have been developed to improve performance of 96 many applications in several research fields [25,26]. A recent approach [8] transforms the feature 97 space by extending specific features of selected dimensions. The result can be applied to improve 98 group separation and reduce visual cluttering in the final embedding. 99 Furthermore, with the increasing size and complexity of data, it becomes more difficult to 100 generate meaningful projections in a fully automatic way. This leads to the development of interactive 101 multidimensional data projection techniques that facilitate interactive analysis by integrating the 102 analyst's knowledge about the data with the knowledge gained during the learning process. 103 Examples include the iPCA approach [6] that provides coordinated views for interactive analysis 104 of projections computed by PCA method and the iVisClassifier system [7] which improves data 105 exploration based on a supervised DR technique (LDA). Moreover, the DimStiller framework [27] 106 analyzes dimension reduction techniques with interactive controls that guide the user during the 107 analysis process and Dis-Function [28] provides an interactive visualization to define a distance 108 function. Similarly, AxiSketcher [10] allows user to change the projection dimensions interactively. 109 Perez et al. [9] proposed an interactive framework for feature space extension that allows the user to 110 incorporate class labels into the projection gradually. A hierarchical interpretation can be done using 111 the clusters of the initial projection and the class labels that are revealed by the method. More details 112 of this technique can be seen in Section 3. 113