Building this AI-powered legal document comparison tool was no small feat, and we encountered a number of challenges along the way. Here are some of the key hurdles we faced and how we overcame them:

Extracting Text from Complex PDF Formats

The Issue: PDF documents often contain complex formatting, images, or tables that made extracting text cleanly difficult. When trying to extract content, the tool sometimes failed to capture the correct structure of legal documents, leading to poorly formatted outputs.

Solution: We used the PyMuPDF (fitz) library, which allowed us to extract text more reliably, but we had to implement custom logic to handle edge cases where the layout was complex. We also applied regex to segment text correctly by legal clauses, ensuring a more structured output for comparison.

Handling Large Document Sizes

The Issue: Some of the legal documents we worked with were very large, making the process of comparing them slow and memory-intensive. The model would struggle to handle multiple large PDFs, especially when running on limited resources.

Solution: We implemented a chunking approach, where documents are divided into smaller sections (clauses) before processing them. This approach not only sped up comparison but also allowed us to perform more efficient memory management.

Accuracy of Clause Matching

The Issue: When comparing two versions of a document, we encountered challenges with accurately matching clauses. Even small semantic differences in phrasing (like reworded sentences) caused mismatches in the comparison.

Solution: We utilized Hugging Face Embeddings and cosine similarity to generate embeddings for each clause, which allowed us to compare the semantic meaning of clauses more effectively. However, we had to fine-tune the threshold values for similarity to avoid false positives or negatives, ensuring more accurate detection of modified clauses.