Optimized Language-Embedded 3DGS for Realistic Modeling and Information Storage of Historical Buildings

11 pages•Published: August 28, 2025

Zhenyu Liang, Jeff Chak Fu Chan, Jiaying Zhang, Zhaolun Liang, Boyu Wang, Mingzhu Wang and Jack C.P. Cheng

Abstract

A realistic and informative 3D digital model of historical buildings holds significant value for heritage preservation, public education, and cultural dissemination. Traditional digital representations, such as Heritage Building Information Modeling, panoramic images, LiDAR point clouds, photogrammetric mesh models, face limitations in user interaction and engagement. The automatic generation of a semantically enriched 3D model requires advanced scene-understanding capabilities. Pre-trained zero-shot methods struggle with domain-specific knowledge in heritage component semantics, while CNN-based approaches demand extensive manual effort for dataset preparation and model training. Therefore, this study proposes an optimized language-embedded 3DGS framework for the digitalization of historical buildings. It involves three steps: (1) data preparation of on-site images and relevant text; (2) component segmentation by the integration of SAM and MLLM; (3) scene reconstruction using the language-embedded 3DGS. The combination of SAM's localization ability and MLLM's in-context learning achieves 95.6% accuracy in the semantic segmentation of historical building components, requiring only a single annotated sample for each component category. Compared with previous methods, our language-embedded 3DGS model accurately captures complex semantics while providing realistic appearance and convenient navigation. The generated 3D model can be further integrated with an LLM-based chatbot assistant to achieve open-vocabulary and vague searches. This framework was validated on the Shishi Sacred Heart Cathedral in Guangzhou, China, offering a novel digital solution for the protection and sustainment of historical buildings.

Keyphrases: 3d reconstruction, historical buildings, language embedded 3d gaussian splatting, multimodal large language models, scene understanding, segment anything model

In: Jack Cheng and Yu Yantao (editors). Proceedings of The Sixth International Conference on Civil and Building Engineering Informatics, vol 22, pages 601-611.

Links:	https://easychair.org/publications/paper/fDS8
	https://doi.org/10.29007/59dn

BibTeX entry

@inproceedings{ICCBEI2025:Optimized_Language_Embedded_3DGS,
  author    = {Zhenyu Liang and Jeff Chak Fu Chan and Jiaying Zhang and Zhaolun Liang and Boyu Wang and Mingzhu Wang and Jack C.P. Cheng},
  title     = {Optimized Language-Embedded 3DGS for Realistic Modeling and Information Storage of Historical Buildings},
  booktitle = {Proceedings of The Sixth International Conference on Civil and Building Engineering Informatics},
  editor    = {Jack Cheng and Yu Yantao},
  series    = {Kalpa Publications in Computing},
  volume    = {22},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2515-1762},
  url       = {/publications/paper/fDS8},
  doi       = {10.29007/59dn},
  pages     = {601-611},
  year      = {2025}}

Download PDF Open PDF in browser