Complaints with Target Scope Identification on Social Media
Abstract
A complaint is uttered when reality fails to meet one's expectations.
Research on complaints, which contributes to our understanding of basic human behavior, has been conducted in the fields of psychology, linguistics, and marketing.
Although several approaches have been implemented to the study of complaints, studies have yet focused on a target scope of complaints.
Examination of a target scope of complaints is crusial because the functions of complaints, such as evocation of emotion, use of grammar, and intention, are different depending on the target scope.
We first tackle the construction and release of a complaint dataset of 6,418 tweets by annotating Japanese texts collected from Twitter with labels of the target scope.
Our dataset is available at \url{https://github.com/sociocom/JaGUCHI}.
We then benchmark the annotated dataset with several machine learning baselines and obtain the best performance of 90.4 F1-score in detecting whether a text was a complaint or not, and a micro-F1 score of 72.2 in identifying the target scope label.
Finally, we conducted case studies using our model to demonstrate that identifying a target scope of complaints is useful for sociological analysis.
Full Text:
PDFReferences
Alicke, M., Braun, J. C., Glor, J., Klotz, M. L., Magee, J., Sederhoim, H., and Siegel, R. (1992). Complaining behavior in social interaction. Personality and Social Psychology Bulletin, 18:286 – 295.
Artstein, R. and Poesio, M. (2008). Survey article: Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596.
Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent dirichlet allocation. the Journal of machine Learning research, 3:993–1022.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
Fang, M., Zong, S., Li, J., Dai, X., Huang, S., and Chen, J. (2022). Analyzing the intensity of complaints on social media. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1742–1754, Seattle, United States. Association for Computational Linguistics.
Filgueiras, J., Barbosa, L., Rocha, G., Lopes Cardoso, H., Reis, L. P., Machado, J. P., and Oliveira, A. M. (2019). Complaint analysis and classification for economic and food safety. In Proceedings of the Second Workshop on Economics and Natural Language Processing, pages 51–60, Hong Kong. Association for Computational Linguistics.
Gautam, A., Mahata, D., Gosangi, R., and Shah, R. R. (2020). Semi-supervised iterative approach for domain-specific complaint detection in social media. In Proceedings of the 3rd Workshop on e-Commerce and NLP, pages 46–53, Seattle, WA, USA. Association for Computational Linguistics.
Hochreiter, S. and Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8):1735–1780.
Jin, M. and Aletras, N. (2020). Complaint identification in social media with transformer networks. In Proceedings of the 28th International
Conference on Computational Linguistics, pages 1765–1771, Barcelona, Spain (Online). International Committee on Computational
Linguistics.
Jin, M. and Aletras, N. (2021). Modeling the severity of complaints in social media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2264–2274, Online. Association for Computational Linguistics.
Kowalski, R. M. (1996). Complaints and complaining: functions, antecedents, and consequences. Psychological bulletin, 119 2:179–96.
Lee, D.-H. (2013). Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks.
Louviere, J. J., Flynn, T. N., and Marley, A. A. J. (2015). Best-worst scaling: Theory, methods and applications. Cambridge University Press.
min Kim, C., Kim, S., Im, S., and Shin, C. (2003). The effect of attitude and perception on consumer complaint intentions. Journal of Consumer Marketing, 20:352–371.
Mitsuzawa, K., Tauchi, M., Domoulin, M., Nakashima, M., and Mizumoto, T. (2016). Fkc corpus : a japanese corpus from new opinion survey service. In proceedings of the Novel Incentives for Collecting Data and Annotation from People: types, implementation, tasking requirements, workflow and results, pages 11–18,
Portoroˇz, Slovenia. Olshtain, E. and Weinbach, L. (1987). 10. complaints: A study of speech act behavior among native and non-native speakers of hebrew.
Plutchik, R. (1980). Chapter 1 - a general psychoevolutionary theory of emotion. In Plutchik, R. and Kellerman, H., editors, Theories of Emotion, pages 3–33. Academic Press.
Preot¸iuc-Pietro, D., Gaman, M., and Aletras, N. (2019). Automatically identifying complaints in social media. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5008–5019, Florence, Italy. Association for Computational Linguistics.
R, S. (2010). A paradise built in hell: The extraordinary communities disaster. Penguin.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ”why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1135–1144, New York, NY, USA. Association for Computing Machinery.
Trosborg, A. (2011). Interlanguage Pragmatics: Requests, Complaints, and Apologies. De Gruyter Mouton.
V´asquez, C. (2011). Complaints online: The case of tripadvisor. Journal of Pragmatics, 43(6):1707–1717. Postcolonial pragmatics.
Zhou, G. and Ganesan, K. (2016). Linguistic understanding of complaints and praises in user reviews. pages 109–114.
DOI: https://doi.org/10.31449/inf.v47i3.4758
This work is licensed under a Creative Commons Attribution 3.0 License.