WIKIPEDIA REVISION TOOLKIT: EFfiCIENTLY ACCESSING WIKIPEDIA’S EDIT HISTORY

Abstract

We present an open-source toolkit whichallows (i) to reconstruct past states of Wikipedia, and (ii) to efficiently access theedit history of Wikipedia articles. Reconstructing past states of Wikipedia is a prerequisite for reproducing previous experimental work based on Wikipedia. Beyond that,the edit history of Wikipedia articles has been shown to be a valuable knowledge source for NLP, but access is severely impeded by the lack of efficient tools for managing the huge amount of provided data. By using a dedicated storage format, our toolkit massively decreases the data volume to less than 2% of the original size, and at the same time provides an easy-to-use interface to access the revision data. The language-independent design allows to process any language represented in Wikipedia. We expect this work to consolidate NLP research using Wikipedia in general, and to foster research making use of the knowledge encoded in Wikipedia’s edit history.