Implementation of NLP-Based Hybrid Retrieval Using TF-IDF and Sentence-BERT (SBERT) for Spotlight Search in a Lecturer Certification Video
DOI:
https://doi.org/10.54065/artificial.2025Keywords:
Automatic Speech Recognition, Hybrid Retrieval, Sentence-BERT, TF-IDF, Timestamped Video RetrievalAbstract
Locating specific information in long educational videos is inefficient when users must inspect the timeline manually. This study implements and evaluates a Natural Language Processing-based hybrid retrieval pipeline for timestamped spotlight search in one Indonesian Lecturer Certification (Sertifikasi Dosen) video. Faster-Whisper medium produced 748 timestamped ASR segments covering approximately 1,788.21 seconds of spoken content. The transcript was converted into 60 retrieval chunks using a fixed 40-second sliding window with a 10-second overlap. Lexical retrieval used TF-IDF, while semantic retrieval used multilingual Sentence-BERT embeddings searched with FAISS IndexFlatIP after L2 normalization. Candidate scores were min-max normalized and combined through α-weighted score fusion. Transcription quality was evaluated using WER and CER, and retrieval was evaluated on 20 queries with 37 relevant query–chunk pairs using Precision@K, Recall@K, MRR@K, and nDCG@K at K = 1, 3, 5, and 10. The ASR evaluation yielded 14.76% WER and 7.23% CER. At α = 0.50, Precision@1 was 0.6000, meaning 12 of 20 queries retrieved a relevant chunk at rank one; Recall@10, MRR@10, and nDCG@10 were 0.9208, 0.7288, and 0.7150, respectively. Among the five tested α settings, α = 0.50 produced the highest Precision@1 and MRR@10, whereas α = 0.75 produced the highest nDCG@10 (0.7273). These findings support the feasibility of the pipeline within the evaluated video and query set, while broader validation is required before generalization.
References
Ahlawat, H., Aggarwal, N., & Gupta, D. (2025). Automatic speech recognition: A survey of deep learning techniques and approaches. International Journal of Cognitive Computing in Engineering, 6, 201–237. https://doi.org/10.1016/j.ijcce.2024.12.007
Alabbad, M., & Alhoshan, W. (2026). Automatic speech recognition in healthcare in the post-LLM era: A scoping review. Healthcare, 14(10), 1333. https://doi.org/10.3390/healthcare14101333
Amur, Z. H., Kwang Hooi, Y., Bhanbhro, H., Dahri, K., & Soomro, G. M. (2023). Short-text semantic similarity (STSS): Techniques, challenges and future perspectives. Applied Sciences, 13(6), 3911. https://doi.org/10.3390/app13063911
Arazzi, M., Ferretti, M., & Nocera, A. (2023). Semantic hierarchical indexing for online video lessons using natural language processing. Big Data and Cognitive Computing, 7(2), 107. https://doi.org/10.3390/bdcc7020107
Bruch, S., Gai, S., & Ingber, A. (2024). An analysis of fusion functions for hybrid retrieval. ACM Transactions on Information Systems, 42(1), Article 20, 1–35. https://doi.org/10.1145/3596512
Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., & Jégou, H. (2026). The Faiss library. IEEE Transactions on Big Data, 12(2), 346–361. https://doi.org/10.1109/TBDATA.2025.3618474
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022). Pre-training methods in information retrieval. Foundations and Trends in Information Retrieval, 16(3), 178–317. https://doi.org/10.1561/1500000100
Guo, J., Fan, Y., Pang, L., Yang, L., Ai, Q., Zamani, H., Wu, C., Croft, W. B., & Cheng, X. (2020). A deep look into neural ranking models for information retrieval. Information Processing & Management, 57(6), 102067. https://doi.org/10.1016/j.ipm.2019.102067
Gupta, D., Attal, K., & Demner-Fushman, D. (2023). A dataset for medical instructional video classification and question answering. Scientific Data, 10, 158. https://doi.org/10.1038/s41597-023-02036-y
Han, M., Zhang, X., Yuan, X., Jiang, J., Yun, W., & Gao, C. (2021). A survey on the techniques, applications, and performance of short text semantic similarity. Concurrency and Computation: Practice and Experience, 33(5), e5971. https://doi.org/10.1002/cpe.5971
Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4), 422–446. https://doi.org/10.1145/582415.582418
Kiss, C., Nagy, M., & Szilágyi, P. (2025). Max–Min semantic chunking of documents for RAG application. Discover Computing, 28, Article 117. https://doi.org/10.1007/s10791-025-09638-7
Liu, M., Nie, L., Wang, Y., Wang, M., & Rui, Y. (2023). A survey on video moment localization. ACM Computing Surveys, 55(9), Article 188, 1–37. https://doi.org/10.1145/3556537
Maistro, M., Breuer, T., Schaer, P., & Ferro, N. (2023). An in-depth investigation on the behavior of measures to quantify reproducibility. Information Processing & Management, 60(3), 103332. https://doi.org/10.1016/j.ipm.2023.103332
Marchesin, S., Purpura, A., & Silvello, G. (2020). Focal elements of neural information retrieval models: An outlook through a reproducibility study. Information Processing & Management, 57(6), 102109. https://doi.org/10.1016/j.ipm.2019.102109
Masuyama, M., Kawahara, T., & Matsuda, K. (2024). Video retrieval system using automatic speech recognition for the Japanese Diet. In Proceedings of the IV Workshop on Creating, Analysing, and Increasing Accessibility of Parliamentary Corpora (ParlaCLARIN) @ LREC-COLING 2024 (pp. 145–148). ELRA and ICCL. https://aclanthology.org/2024.parlaclarin-1.21/
Moffat, A. (2022). Batch evaluation metrics in information retrieval: Measures, scales, and meaning. IEEE Access, 10, 105564–105577. https://doi.org/10.1109/ACCESS.2022.3211668
Novriansyah, F., Prasti, D., & Uramako, E. (2023). Pembuatan Video Profil Pemodelan Tata Ruang 3 Dimensi Gedung SMAN 15 Luwu . Journal Artificial: Informatika Dan Sistem Informasi, 1(1), 53–59. https://doi.org/10.54065/artificial.250
Polat, H., Turan, A. K., Koçak, C., & Ulaş, H. B. (2024). Implementation of a Whisper architecture-based Turkish automatic speech recognition (ASR) system and evaluation of the effect of fine-tuning with a low-rank adaptation (LoRA) adapter on its performance. Electronics, 13(21), 4227. https://doi.org/10.3390/electronics13214227
Putra, S. J., Wijaya, A., & Alam, R. G. G. (2025). Analysis of MP3 bitrate on the accuracy of academic audio transcription using Whisper large-v3. Jurnal Sistem Cerdas, 8(2), 160–168. https://doi.org/10.37396/jsc.v8i2.528
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning (PMLR 202, pp. 28492–28518). PMLR.
Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (pp. 3982–3992). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-1410
Reimers, N., & Gurevych, I. (2020). Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4512–4525). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.365
Riskawati, R., & Saleh, M. (2026). Implementasi Project-Based Learning Berbantuan Tinkercad untuk Mendukung Pemahaman Konsep Listrik Dinamis Mahasiswa. Journal Artificial: Informatika Dan Sistem Informasi, 4(2), 227–238. https://doi.org/10.54065/artificial.1547
Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523. https://doi.org/10.1016/0306-4573(88)90021-0
Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., & Gurevych, I. (2021). BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks.
Wang, J., Huang, J. X., Tu, X., Wang, J., Huang, A. J., Laskar, M. T. R., & Bhuiyan, A. (2024). Utilizing BERT for information retrieval: Survey, applications, resources, and challenges. ACM Computing Surveys, 56(7), Article 185, 1–33. https://doi.org/10.1145/3648471
Wu, J., Liu, W., Liu, Y., Liu, M., Nie, L., Lin, Z., & Chen, C. W. (2026). A survey on video temporal grounding with multimodal large language model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(2), 1521–1541. https://doi.org/10.1109/TPAMI.2025.3615586
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Evan Januards Sapteno, Luther Alexander Latumakulita, Rillya Arundaa, Eliasta Ketaren, Mahardika Inra Takaendengan, Dodisutarma Lapihu
This work is licensed under a Creative Commons Attribution 4.0 International License.
License and Copyright Agreement
- Authors retain copyright and other proprietary rights related to the article.
- Authors retain the right and are permitted to use the substance of the article in their own future works, including lectures and books.
- Authors grant the journal the right of first publication with the work simultaneously licensed under Creative Commons Attribution License (CC BY 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post or self-archive their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
