Link prediction Graph Neural Networks for structure recognition of Handwritten Mathematical Expressions

A novel method for recognizing handwritten mathematical expressions has been introduced, leveraging Graph Neural Networks (GNN) to model these expressions as graphs where nodes represent symbols and edges capture spatial relationships. This approach integrates a deep Bidirectional Long Short-Term Memory (BLSTM) network dedicated to symbol recognition with a two-dimensional Context-Free Grammar (2D-CFG) parser that analyzes spatial relations among symbols. The combined system aims to improve the accuracy of mathematical expression recognition by effectively capturing both the identity of individual symbols and their spatial arrangement. By representing expressions structurally as graphs, the method facilitates link prediction tasks that identify the correct connections between symbols. This approach addresses the complex challenge of interpreting handwritten mathematical notation, which requires understanding both symbol shapes and their relative positioning. The integration of deep learning for symbol identification and grammar-based parsing for spatial context represents a comprehensive strategy for structure recognition in this domain. Overall, this method offers a promising advancement in the automated interpretation of handwritten mathematical content.