# DocLLM: A layout-aware generative language model for multimodal document understanding. (arXiv:2401.00908v1 [cs.CL])


Enterprise documents such as forms, invoices, receipts, reports, contracts, and
other similar records, often carry rich semantics at the intersection of textual
and spatial modalities. The visual cues offered by their complex layouts play a
crucial role in comprehending these documents effectively. In this paper, we
present DocLLM, a lightweight extension to traditional large language models
(LLMs) for reasoning over visual documents, taking into account both textual
semantics and spatial layout. Our model differs from existing multimodal LLMs by
avoiding expensive image encoders and focuses exclusively on bounding box
information to incorporate the spatial layout structure. Specifically, the
cross-alignment between

[Read Full Article](http://arxiv.org/abs/2401.00908)