Advantage of integration in big data: Feature generation in multi-relational databases for imbalanced learning

Farrukh Ahmed, Michele Samorani, Colin Bellinger, Osmar R. Zaiane
2016 2016 IEEE International Conference on Big Data (Big Data)  
Most data mining and machine learning techniques rely on a single flat table and assume balanced training data. However, most real-world applications comprise databases having multiple tables and imbalanced data. It becomes further complicated in the realm of Big Data where related information is spread over different data repositories. This work focuses on the automatic construction of a mining table by aggregating information from multiple local tables and additional data sources as external
more » ... ables in a multi-relational database. Our work extends data aggregation techniques by exploring paths where a single table is traversed multiple times. The existing techniques do not generate attributes that exist on such paths or do not generate them efficiently. However, these paths contain useful past information. Our framework for Generating Attributes with Rolled Paths (GARP) also prevents leakage of the class information by avoiding features built after the knowledge of the class label. While generating new attributes, our system discovers certain patterns that provide useful insights for decision making. Experiments are performed on a transactional dataset from a U.S. consumer electronics retailer to predict product returns and identify reasons behind those returns. In addition, we augmented the retail dataset with Supplier information and Reviews to show the value of data integration. This dataset has the class imbalance problem, since product returns represent only 10% of the complete dataset. The results show that our technique improves classification accuracy and discovers new knowledge even in the presence of the class imbalance. Our scalability analysis shows that our approach can handle an increasing load of data in a linear fashion. ii To my grandfather, Mian Mushtaq Ahmed For making me what I am. iii First and foremost, praises and thanks to the Almighty God, for providing all the guidance, opportunities and abilities to achieve my goals.
doi:10.1109/bigdata.2016.7840644 dblp:conf/bigdataconf/AhmedSBZ16 fatcat:yxwoeguvg5bz7o4muol2ctsa2q