Scalable Distributed Reasoning Using MapReduce [chapter]

Jacopo Urbani, Spyros Kotoulas, Eyal Oren, Frank van Harmelen
2009 Lecture Notes in Computer Science  
We address the problem of scalable distributed reasoning, proposing a technique for materialising the closure of an RDF graph based on MapReduce. We have implemented our approach on top of Hadoop and deployed it on a compute cluster of up to 64 commodity machines. We show that a naive implementation on top of MapReduce is straightforward but performs badly and we present several non-trivial optimisations. Our algorithm is scalable and allows us to compute the RDFS closure of 865M triples from
more » ... e Web (producing 30B triples) in less than two hours, faster than any other published approach.
doi:10.1007/978-3-642-04930-9_40 fatcat:djqm7scjibfp5cjbebilb5ualu