A Distributed Cache Management Scheme for Efficient Accesses of Small Files in HDFS
HDFS에서 소형 파일의 효율적인 접근을 위한 분산 캐시 관리 기법

Hyunkyo Oh, Kiyeon Kim, Jae-Min Hwang, Junho Park, Jongtae Lim, Kyoungsoo Bok, Jaesoo Yoo
2014 The Journal of the Korea Contents Association  
In this paper, we propose the distributed cache management scheme to efficiently access small files in Hadoop Distributed File Systems(HDFS). The proposed scheme can reduce the number of metadata managed by a name node since many small files are merged and stored in a chunk. It is also possible to reduce the file access costs, by keeping the information of requested files using the client cache and data node caches. The client cache keeps small files that a user requests and metadata. Each data
more » ... metadata. Each data node cache keeps the small files that are frequently requested by users. It is shown through performance evaluation that the proposed scheme significantly reduces the processing time over the existing scheme. ■ keyword :|Hadoop Distributed File System|Small File|Distributed Cache|Cache Metadata|
doi:10.5392/jkca.2014.14.11.028 fatcat:nzx6ebtfu5farmp5nrrw4nkjuq