publications

2024

1. ICDAR 2024

   ChartReformer: Natural Language-Driven Chart Image Editing

   Pengyu Yan, Mahesh Bhosale, Jay Lal, and 2 more authors

   In Document Analysis and Recognition - ICDAR 2024 , 2024

   Abs Bib HTML

   Chart visualizations are essential for data interpretation and communication; however, most charts are only accessible in image format and lack the corresponding data tables and supplementary information, making it difficult to alter their appearance for different scenarios of application. To eliminate the need for original underlying data and information to perform chart editing, we propose ChartReformer, a natural language-driven chart image editing solution that directly edits the charts from the input images with the given instruction prompts. Instead of predicting the plotting code, the key in this method is that we allow the model to comprehend the chart and reason over the prompt to generate the corresponding underlying data table and visual attributes for new charts, enabling a precise and stable editing result. To generalize ChartReformer, we define and standardize the chart editing category and generate the ChartCraft dataset, covering style, layout, format, and data-centric edits. The experiments show promising results for the natural language-driven chart image editing. Our datasets and model are available at: https://github.com/pengyu965/ChartReformer.

   @inproceedings{10.1007/978-3-031-70533-5_26,
     author = {Yan, Pengyu and Bhosale, Mahesh and Lal, Jay and Adhikari, Bikhyat and Doermann, David},
     editor = {Barney Smith, Elisa H. and Liwicki, Marcus and Peng, Liangrui},
     title = {ChartReformer: Natural Language-Driven Chart Image Editing},
     booktitle = {Document Analysis and Recognition - ICDAR 2024},
     year = {2024},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {453--469},
     isbn = {978-3-031-70533-5},
     url = {https://link.springer.com/chapter/10.1007/978-3-031-70533-5_26},
   }

2023

1. ICDAR 2023

   LineFormer: Line Chart Data Extraction Using Instance Segmentation

   Jay Lal, Mahesh Bhosale, Aditya Mitkari, and 1 more author

   In Document Analysis and Recognition - ICDAR 2023 , 2023

   Abs Bib HTML

   Data extraction from line-chart images is an essential component of the automated document understanding process, as line charts are a ubiquitous data visualization format. However, the amount of visual and structural variations in multi-line graphs makes them particularly challenging for automated parsing. Existing works, however, are not robust to all these variations, either taking an all-chart unified approach or relying on auxiliary information such as legends for line data extraction. In this work, we propose LineFormer, a robust approach to line data extraction using instance segmentation. We achieve state-of-the-art performance on several benchmark synthetic and real chart datasets. Our implementation is available at https://github.com/TheJaeLal/LineFormer.

   @inproceedings{10.1007/978-3-031-41734-4_24,
     author = {Lal, Jay and Bhosale, Mahesh and Mitkari, Aditya and Doermann, David},
     editor = {Fink, Gernot A. and Jain, Rajiv and Kise, Koichi and Zanibbi, Richard},
     title = {LineFormer: Line Chart Data Extraction Using Instance Segmentation},
     booktitle = {Document Analysis and Recognition - ICDAR 2023},
     year = {2023},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {387--400},
     isbn = {978-3-031-41734-4},
     url = {https://link.springer.com/chapter/10.1007/978-3-031-41734-4_24},
   }

2. Technical Report

   Player Re-Identification Using Body Part Appearences

   Mahesh Bhosale, Abhishek Kumar, and David Doermann

   arXiv preprint arXiv:2310.14469, 2023

   Abs Bib HTML

   We propose a neural network architecture that learns body part appearances for soccer player re-identification. Our model consists of a two-stream network (one stream for appearance map extraction and the other for body part map extraction) and a bilinear-pooling layer that generates and spatially pools the body part map. Each local feature of the body part map is obtained by a bilinear mapping of the corresponding local appearance and body part descriptors. Our novel representation yields a robust image-matching feature map, which results from combining the local similarities of the relevant body parts with the weighted appearance similarity. Our model does not require any part annotation on the SoccerNet-V3 re-identification dataset to train the network. Instead, we use a sub-network of an existing pose estimation network (OpenPose) to initialize the part substream and then train the entire network to minimize the triplet loss. The appearance stream is pre-trained on the ImageNet dataset, and the part stream is trained from scratch for the SoccerNet-V3 dataset. We demonstrate the validity of our model by showing that it outperforms state-of-the-art models such as OsNet and InceptionNet.

   @article{bhosale2023player,
     title = {Player Re-Identification Using Body Part Appearences},
     author = {Bhosale, Mahesh and Kumar, Abhishek and Doermann, David},
     journal = {arXiv preprint arXiv:2310.14469},
     year = {2023},
   }