LexNLP: Natural Language Processing and Information Extraction For Legal and Regulatory Texts

Abstract

LexNLP is an open source Python package focused on natural language processing and machine learning for legal and regulatory text. The package includes functionality to (i) segment documents, (ii) identify key text such as titles and section headings, (iii) extract over eighteen types of structured information like distances and dates, (iv) extract named entities such as companies and geopolitical entities, (v) transform text into features for model training, and (vi) build unsupervised and supervised models such as word embedding or tagging models. LexNLP includes pre-trained models based on thousands of unit tests drawn from real documents available from the SEC EDGAR database as well as various judicial and regulatory proceedings. LexNLP is designed for use in both academic research and industrial applications.

Keywords: natural language processing, legal, regulatory, machine learning, segmentation, extraction, open source, Python

JEL Classification: C19, C53, C55, C38, C45, C63, C88