Systematic review of training methods for conversational systems: the potential of datasets validated with user experience

Authors

  • Carolina Abrantes GoTelecom, S.A., Aveiro
  • Juliana Camargo University of Aveiro
  • José Nunes Department of Communication and Art/DigiMedia, Aveiro University
  • Maria João Antunes Department of Communication and Art/DigiMedia, Aveiro University 
  • Óscar Mealha Department of Communication and Art/DigiMedia, Aveiro University 
  • Luís Nóbrega

DOI:

https://doi.org/10.15847/obsOBS17220232263

Abstract

The increasing maturity of artificial intelligence technologies such as Machine Learning algorithms, Natural Language Processing (NLP), Automatic Speech Recognition (ASR) and Natural Language generation are changing the way users interact with technology. Specifically, as voice interactions are becoming commonplace, it is important to understand how such systems are being trained. This systematic review investigates how human data is collected for training conversational agents, with specific interest on data sets directly obtained from human participation in real contexts of need and use. The work reported in this article was supported by PRISMA guidelines and search procedures were led in Scopus, Web of Science and ProQuest, in English and within the last 15-years (2005-2020), with pre-defined criteria to get a detailed holistic perspective of practices published until July 2020. From both search iterations, a total of 22 papers were considered for this review. The main contributions from these papers reveal a common use of learning from demonstration/observation and crowdsourcing methods, in system training and dataset cataloguing, alongside handwriting and sentence labelling and Wizard-of-Oz based studies.

Author Biographies

Carolina Abrantes, GoTelecom, S.A., Aveiro

GoTelecom, S.A., Aveiro

 

Systematic review of training methods for conversational systems: the potential of datasets validated with user experience

José Nunes, Department of Communication and Art/DigiMedia, Aveiro University

Department of Communication and Art/DigiMedia, Aveiro University

Maria João Antunes, Department of Communication and Art/DigiMedia, Aveiro University 

Department of Communication and Art/DigiMedia, Aveiro University 

 

Systematic review of training methods for conversational systems: the potential of datasets validated with user experience

Óscar Mealha, Department of Communication and Art/DigiMedia, Aveiro University 

Department of Communication and Art/DigiMedia, Aveiro University 

"Systematic review of training methods for conversational systems: the potential of datasets validated with user experience".

Luís Nóbrega

 GoTelecom, S.A., Aveiro ()

Downloads

Published

2023-06-14

How to Cite

Abrantes, C., Camargo, J., Nunes, J., Antunes, M. J., Mealha, Óscar, & Nóbrega, L. (2023). Systematic review of training methods for conversational systems: the potential of datasets validated with user experience. Observatorio (OBS*), 17(2). https://doi.org/10.15847/obsOBS17220232263

Issue

Vol. 17 No. 2 (2023)

Section

Articles

License

Copyright (c) 2023 Juliana Camargo Observatorio (OBS*) has an academic journal profile. As such, it is dedicated to the open exchange of its contents, and freely available to individuals and institutions without charge or formal permission. Copies of this journal or articles in this journal may be distributed for research or educational purposes free of charge and without permission. Authors will not be charged for publication or submission to this journal. Authors who publish in Observatorio (OBS*) will release their contributions under the Creative Commons Attribution-NonCommercial-2.0 generic. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given - being the author the copyright holder. This journal utilizes the LOCKSS system to create a distributed archiving system among participating libraries and permits those libraries to create permanent archives of the journal for purposes of preservation and restoration. The publisher perpetually authorizes participants in the LOCKSS system to archive and restore our publication through the LOCKSS System for the benefit of all LOCKSS System participants. Specifically participating libraries may: • Collect and preserve currently accessible materials; • Use material consistent with original license terms; • Provide copies to other LOCKSS appliances for purposes of audit and repair. Fair use of a copyrighted work, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include: 1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes; 2. the nature of the copyrighted work; 3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; & 4. the effect of the use upon the potential market for or value of the copyrighted work. The fact that a work is unpublished shall not itself bar a finding of fair use if such finding is made upon consideration of all the above factors. In accord with these provisions, the OBS journal believes in the vigorous assertion and defense of Fair Use by scholars engaged in academic research, teaching and non-commercial publishing. Thus, we view the inclusion of “quotations” from existing print, visual, audio and audio-visual texts to be appropriate examples of Fair Use, as are reproductions of visual images for the purpose of scholarly analysis. We encourage authors to obtain appropriate permissions to use materials originally produced by others, but do not require such permissions as long as the usage of such materials falls within the boundaries of Fair Use.