{{PAPER_TITLE}}

🚀 Introduction

MOLE is a framework that leverages Large Language Models (LLMs) to automatically extract metadata attributes from scientific papers covering datasets of languages other than Arabic. Our schema-driven methodology processes entire documents across multiple input formats and incorporates robust validation mechanisms for consistent output.

📄 Example

Here is an example of extracted metadata from a given sample paper. Note this example is smilplified, the actual paper is more complex and contain more pages.

📹 Demo

You can find the demo on this link to run inference on a given paper.

📚 Dataset

We manually annotated 52 papers. We split it 6 for validation and 46 for testing. The datasets covered span Arabic, English, Japanese, French, Russian, and multi-lingual datasets. The dataset is available on Hugging Face 🤗.

📊 Results

We evaluted the performance of 7 LLMs on the 46 annotated papers spanning datasets of different languages. Our results show that Gemini 2.5 Pro achieves the best scores.

📝 Citation

If you find this work useful, please cite it as follows:

@misc{mole,
    title={MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs}, 
    author={Zaid Alyafeai and Maged S. Al-Shaibani and Bernard Ghanem},
    year={2025},
    eprint={2505.19800},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2505.19800}, 
}