Comparative Analysis for Requirement Classification Using Transformer based Pre-trained Models for Digital Governance RFPs
Abstract
Digital Governance deals with vast data. One of the domains in digital governance is tendering, which uses multiple documents, including a Request for Proposal (RFP). These documents contain extensive information, particularly detailing the project's expected requirements. For these RFP documents, the identification and classification of requirements are essential. One of the existing datasets under the software engineering domain is the PROMISE dataset for software requirements; this work takes inspiration from the PROMISE dataset and curates a dataset for digital governance software development-related RFP documents. The curated domain-specific dataset for text classification uses a pre-trained language model to classify functional and non-functional requirements. Experiments were performed to compare the Transformer's model performance with the baseline dataset, the curated DigiGov RFP dataset, and the concatenated PROMISE + DigiGov RFP datasets. The model's statistical performance across the datasets is assessed using an ANOVA test. The work focuses on automating RFP document statement classification using transformer-based pre-trained models through transfer learning, increasing productivity and accuracy in the field of digital governance. The research shows that using state-of-the-art techniques for RFP documents can effectively enhance the quality of the bidding process. This technique can bring automation to requirement analysis in the bidding process, strengthening the digital governance process.References
Hanisch, M., Goldsby, C.M., Fabian, N.E., Oehmichen, J.: Digital governance: A conceptual framework and research agenda. Journal of Business Research 162, 113777 (2023) https://doi.org/10.1016/j.jbusres.2023.113777
Paech, B., Heinrich-Robert, Zorn-Pauli, G., Jung, A., Tadjiky, a.: In: Regnell, B.,
Damian, D. (eds.) Answering a Request for Proposal – Challenges and Proposed Solutions. Springer
Rajbhoj, A., Nistala, P., Kulkarni, V., Ganesan, G.: A rfp system for generating response to a request for proposal. In: Proceedings of the 12th Innovations in Software Engineering Conference (Formerly Known as India Software Engineering Conference). ISEC '19. Association for Computing Machinery, New York, NY, USA (2019).https://doi.org/10.1145/3299771.3299779
Rajbhoj, A., Nistala, P., Batra, P., Kulkarni, V.: Ai-enabled project initiation:
An approach based on rfp response document. In: Proceedings of the 15th Innovations in Software Engineering Conference. ISEC '22. Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3511430. 3511450 .
Knuth,D.:Knuth:Computers and Typesetting.
https://www.mckinsey.com/industries/public-sector/our-insights/transforming-government-through-digitization
Saha, B.K., Haab, L., Tandur, D.: A natural language understanding approach toward extraction of specifications from request for proposals. In: International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2023, Bali, Indonesia, February 20-23, 2023, pp. 205– 210. IEEE,(2023). https://doi.org/10.1109/ICAIIC57133.2023.10067032 .
Winkler, J., Vogelsang, A.: Automatic classification of requirements based on convolutional neural networks. In: 2016 IEEE 24th International Requirements Engineering Conference Workshops (REW), pp. 39–45 (2016). https://doi.org/ 10.1109/REW.2016.021
Navarro-Almanza, R., Juarez-Ramirez, R., Licea, G.: Towards supporting software engineering using deep learning: A case of software requirements classification. In: 2017 5th International Conference in Software Engineering Research and Innovation (CONISOFT), pp. 116–120 (2017). https://doi.org/10.1109/ CONISOFT.2017.00021
Hey, T., Keim, J., Koziolek, A., Tichy, W.F.: Norbert: Transfer learning for requirements classification. In: 2020 IEEE 28th International Requirements Engineering Conference (RE), pp. 169–179 (2020). https://doi.org/10.1109/RE48521.2020.00028
Sainani, A., Anish, P.R., Joshi, V., Ghaisas, S.: Extracting and classifying requirements from software engineering contracts. In: 2020 IEEE 28th International Requirements Engineering Conference (RE), pp. 147–157 (2020). https: //doi.org/10.1109/RE48521.2020.00026
Tiun, S., Mokhtar, U.A., Bakar, S.H., Saad, S.: Classification of functional and non-functional requirement in software requirement using word2vec and fast text. Journal of Physics: Conference Series 1529(4), 042077 (2020) https://doi.org/10. 1088/1742-6596/1529/4/042077
Luo, X., Xue, Y., Xing, Z., Sun, J.: Prcbert: Prompt learning for requirement classification using bert-based pretrained language models. In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. ASE '22. Association for Computing Machinery, New York, NY, USA (2023). https://doi.org/10.1145/3551349.3560417 . https://doi.org/10.1145/3551349.3560417
Kaur, K., Kaur, P.: Improving bert model for requirements classification by bidirectional lstm-cnn deep model. Computers and Electrical Engineering 108, 108699 (2023) https://doi.org/10.1016/j.compeleceng.2023.108699
Kaur, K., Kaur, P.: Mnor-bert: multi-label classification of non-functional requirements using bert. Neural Comput. Appl. 35(30), 22487–22509 (2023) https: //doi.org/10.1007/s00521-023-08833-1
Sonawane, P. S.N.: Classification of functional and non-functional requirements based on convolutional neural network with flower pollination optimiser. Innovations in Systems and Software Engineering (2024) https://doi.org/10.1007/ s11334-024-00592-z
Garc´ıa, F.-y.-F.C.A..P.E.G. S.E.: Classification of non-functional requirements using convolutional neural networks. Innovations in Systems and Software Engineering Program Comput Soft 49 705–711 (2023). (2023) https://doi.org/ 10.1007/s11334-024-00592-z
Saqib, M.M.J.M.S.e.a. M.: Deep-transfer learning inspired natural language processing system for software requirements classification. knowl inf syst (2024). https://doi.org/10.1007/s10115-024-02248-7. Knowledge and Information Systems (2024) https://doi.org/10.1007/s10115-024-02248-7
W'ojcicki, B., Dabrowski, R.: Applying machine learning to software fault prediction. e-Informatica Software Engineering Journal 12(1), 199–216 (2018)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018) 1810.04805
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: A Robustly Optimised BERT Pretraining Approach (2019). https://arxiv.org/abs/1907.11692
huggingface: huggingface.co. https://huggingface.co/docs/transformers/en/ model doc/roberta. Accessed: 2025-01-25 (2025)
medium: medium. https://medium.com/huggingface/distilbert-8cf3380435b5. Accessed: 2025-01-25 (2025)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019) 1910.01108
huggingface: huggingface. https://huggingface.co/docs/transformers/en/model doc/distilbert. Accessed: 2025-01-25 (2025)
Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: Xlnet: Generalised autoregressive pretraining for language understanding. CoRR abs/1906.08237 (2019) 1906.08237
huggingface: huggingface. https://huggingface.co/docs/transformers/en/model doc/xlnet. Accessed: 2025-01-25 (2
DOI:
https://doi.org/10.31449/inf.v50i13.9522Downloads
Published
How to Cite
Issue
Section
License
Authors retain copyright in their work. By submitting to and publishing with Informatica, authors grant the publisher (Slovene Society Informatika) the non-exclusive right to publish, reproduce, and distribute the article and to identify itself as the original publisher.
All articles are published under the Creative Commons Attribution license CC BY 3.0. Under this license, others may share and adapt the work for any purpose, provided appropriate credit is given and changes (if any) are indicated.
Authors may deposit and share the submitted version, accepted manuscript, and published version, provided the original publication in Informatica is properly cited.
