Online Integrated Development Environment for MapReduce Programming

Zhiqiang Ma, Shuangtao Yang, Zhida Shi, Rui Yan
2016 International Journal of u- and e- Service, Science and Technology  
Though MapReduce programming model simplifies the development of parallel program, ordinary users have difficulties in setting up the development environment for MapReduce. The online integrated development environment for MapReduce programming can solve this problem, thus users need not build the environment themselves, only need to focus on the logical design of the parallel program. During the software construction, the problem of independent space setting and naming conflict of the file in
more » ... he multi-user environment, and the problem of online compiling, execution and instant feedback message to client are solved. The software has been deployed and tested in Hadoop cluster, and can meet users' basic requirements for the development of MapReduce. multiple applications; it provides an environment for compiling, building and running MapReduce programs; and it supports to upload and download the data sets of application. So it is flexible enough for developers to configure MapReduce version and data sets. Related Work General integrated development environment includes four parts: a code editor, a compiler, debugging tools and runtime environment. Its working model has desktop model and online model [4] [5] [6] . Desktop model has been widely used to develop desktop applications and server programs by programmer, such as Microsoft visual studio and open source Eclipse. The other model, which adopts Browser/Server design pattern, is online integrated development environment and also a developing tool. We use online model to build IDEMP, so compiling, debugging and running program are working at the server, programmers use the client to code, give commands and receive output messages. Thus programmers need to register the accounts before developing. When they use online environment, they need to open the web page without deploying runtime environment. So development environment of client requires almost nothing to hardware, only PC. For example CodeRun [7] tool based on JavaScipt supports code highlighting, debugging and deployment for C# language; Cloud9IDE [8] tool based on NodeJS includes built-in VIM and supports code highlighting for JS, HTML and CSS ; C-Eclipse [9] tool , whose core function is web services, includes code editing, code promoting, debugging and running for Java language. While the jar program is running on the server, if a client does not get feedback messages from the server in time, then developers think that there is something wrong with their program. Due to the need for a "dynamic execution to obtain information" performance, a real-time data communication model to implement "real-time output" messages can be given, and output messages are stored in the ConsoleTask object, as shown in Figure5
doi:10.14257/ijunesst.2016.9.6.36 fatcat:5rmiumjxxndy7izatmwcw2ayom