Overview
Marker is a software that converts PDF, EPUB, and MOBI files into markdown format. It is designed to be significantly faster than nougat, provide more accurate conversions, and have a lower risk of hallucinations (incorrect or fabricated content that doesn't exist in the source material).
Key Features
- Support for various PDF documents, particularly optimized for books and scientific papers.
- Capable of removing unwanted artifacts such as headers and footers.
- Can convert many mathematical equations into LaTeX format.
- Formats code blocks and tables effectively.
- Compatible with GPU, CPU, or MPS hardware.
How Marker Works
The Marker operates through a pipeline of deep learning models to process documents:
- Text Extraction: It extracts text and performs Optical Character Recognition (OCR) using heuristics and Tesseract when necessary.
- Layout Segmentation: The layout segmenter analyzes the document's format.
- Column Detection: To handle multi-column documents.
- Nougat Model: Marker utilizes nougat for part of its processing.
- PDF Postprocessor: Cleans up the document after conversion.
Marker encountered a [repetition] in 1.5% of pages during testing, but it outperforms nougat in terms of speed and general-purpose usage, particularly with equation blocks.
Performance Comparison
Marker has been benchmarked against nougat, showing that it is 10x faster and uses less VRAM.
Community and Support
Marker has a community on Discord where users can interact and share their experiences.
Limitations
While Marker is powerful, it does face some challenges:
- Fewer equations converted to LaTeX compared to nougat.
- Inconsistent whitespace and indentation management.
- Not all lines may be correctly joined.
- Better support for languages similar to English; limited support for Asian languages.
- Optimized for digital PDFs, so heavy OCR isn't its forte.
Installation and Setup
For Linux
The installation involves cloning the Marker repository, running a few scripts for dependencies like Tesseract and Ghostscript, and setting up the environment with poetry.
For Mac
The Mac installation process is similar but utilizes Homebrew for installing requirements and then proceeds with setting up poetry and configuring the local environment.
Usage Guidelines
Configuration
Prior to use, certain environment variables must be set, such as TORCH_DEVICE, INFERENCE_RAM, and ENABLE_EDITOR_MODEL, which can be customized within local.env and settings.py.
Converting Files
Marker can convert single files or batch convert multiple files. For batches, one can define several parameters like worker count, RAM usage per task, maximum number of pages, and default language.
Running Benchmarks
Marker provides a benchmark.py script to compare its performance against naive text extraction and nougat.
Commercial Usage
Due to licensing restrictions of underlying models like Layoutlmv3 and nougat, Marker is intended only for non-commercial usage.
For inquiries or issues regarding commercial restrictions, users can contact Marker's support via marker@vikas.sh.
Acknowledgements
Marker's development has been greatly influenced by open-source models and datasets provided by various organizations, including Meta, Microsoft, IBM, and Google.
Conclusion
Marker showcases an advancement in document conversion technology, offering fast, accurate, and reliable conversion of complex documents into markdown format. Nevertheless, it has some current limitations and restrictions concerning commercial use, which are being addressed.
Tags:
- #DocumentConversion
- #MarkerTool
- #PDFtoMarkdown
- #DeepLearningModels
- #OpenSource