Member-only story

Influencing Factors in Speech Quality Assessment Using Crowdsourcing

Ivan Turgenev

·10.8k Followers· Follow

Published in Kindle Edition

5 min read · 3 weeks before

631 View Claps

38 Respond

Save

Listen

Influencing Factors in Speech Quality Assessment using Crowdsourcing

by Rafael Zequeira Jiménez

5 out of 5

Language	:	English
File size	:	6619 KB
Text-to-Speech	:	Enabled
Screen Reader	:	Supported
Enhanced typesetting	:	Enabled
Print length	:	239 pages

Abstract

This article provides a comprehensive overview of the factors that influence the accuracy and reliability of crowdsourced speech quality assessments. We discuss the impact of participant characteristics such as age, gender, and expertise, as well as the importance of task design elements like task difficulty and task duration. We also explore the role of technology, including the use of different audio codecs and playback devices. We conclude by providing recommendations for researchers and practitioners working in this area.

Speech quality assessment is a critical component of the development and refinement of speech communication systems. Traditional methods for speech quality assessment have relied on subjective testing, in which a group of human listeners are asked to rate the quality of a speech sample. However, this method can be time-consuming and expensive. Crowdsourcing has emerged as a promising alternative to traditional methods, offering the potential for faster and more cost-effective speech quality assessment.

However, a number of factors can influence the accuracy and reliability of crowdsourced speech quality assessments. In this article, we discuss the most important factors that researchers and practitioners need to consider when designing and conducting crowdsourced speech quality assessments.

Factors Influencing Speech Quality Assessment

Participant Characteristics

The characteristics of the participants in a crowdsourced speech quality assessment can significantly influence the results. Factors such as age, gender, and expertise can all have an impact on the way that participants perceive and rate speech quality.

Age: Older adults tend to have more difficulty hearing high-frequency sounds, which can affect their perception of speech quality.
Gender: There is some evidence that women are more sensitive to speech quality than men.
Expertise: Participants who are more familiar with speech quality assessment are likely to provide more accurate and reliable ratings.

Task Design

The design of the task that participants are asked to complete can also affect the results of a crowdsourced speech quality assessment. Factors such as task difficulty, task duration, and the instructions given to participants can all influence the quality of the data collected.

Task difficulty: Tasks that are too difficult or too easy can lead to inaccurate or unreliable ratings.
Task duration: Tasks that are too long can lead to fatigue, which can affect the quality of the data collected.
Instructions: The instructions given to participants should be clear and concise, and should provide all of the information that participants need to complete the task accurately.

Technology

The technology used to conduct a crowdsourced speech quality assessment can also have an impact on the results. Factors such as the audio codec used, the playback device used, and the environment in which the assessment is conducted can all affect the way that participants perceive and rate speech quality.

Audio codec: The audio codec used can affect the quality of the speech sample, which can in turn affect the way that participants rate the speech quality.
Playback device: The playback device used can also affect the quality of the speech sample, as well as the way that participants perceive the speech quality.
Environment: The environment in which the assessment is conducted can also affect the way that participants perceive and rate speech quality.

Recommendations

Based on our discussion of the factors that influence speech quality assessment using crowdsourcing, we offer the following recommendations for researchers and practitioners:

Use a representative sample of participants.
Design tasks that are appropriate for the target population.
Provide clear and concise instructions to participants.
Use high-quality audio codecs and playback devices.
Conduct the assessment in a quiet and controlled environment.

Crowdsourcing is a powerful tool that can be used to conduct speech quality assessments quickly and cost-effectively. However, it is important to be aware of the factors that can influence the accuracy and reliability of the results. By carefully considering the factors discussed in this article, researchers and practitioners can design and conduct crowdsourced speech quality assessments that produce high-quality data.

Influencing Factors in Speech Quality Assessment using Crowdsourcing

by Rafael Zequeira Jiménez

5 out of 5