Although advances in artificial intelligence and machine learning have steadily improved the quality of online voice cloning, there are several difficulties related to this process. The first big challenge is the quality and realism of the cloned voice, especially with sophisticated vocal features. Although platforms such as Descript or Lyrebird can replicate broad speech patterns, accommodating emotions, changes in tone and the more nuanced aspects of human inflections is another matter. Still, trying to synthesize a natural voice after processing only 5 minutes of your recording will also make it hard to reproduce the connotations (“the colors”) of how you speak when emotional, affecting this too.
Data needs are also a big challenge. A good quality voice clone generally requires a larger amount of speech data in order to generate a more authentic replica. For example, a 30 second clone may sound super robotic with only a small fraction of your actual voice print, but the longer the sample – for instance 5 minutes – you will get results that much more fitting. Yet, collecting long voice recordings can be difficult or inconvenient, especially in professional environments where time and resources are limited.
A major issue when it comes to voice cloning is privacy as well. The increasing availability of the technology means it can be used by people who do not have a good understanding of its limitations or have access to technical expertise. We also heard echo examples of deepfake audio —regarding cloned voices, deployed to impersonate people in fraud or misinformation. Moreover, in one particular case a CEO was voice-cloned and the cyber criminals managed to alter the audio file so as to prompt a fake order for €220.000. The more readily accessible this technology is, the more important it becomes to develop ethical protocols and legal parameters in order to prevent abuse.
Moreover, high-quality voice cloning also has great computational cost and takes a lot longer. Powerful algorithms and models are required to synthesize realistic human-like voice replicas using advanced cloning platforms which makes it time-intensive and costly for the users. As you can see, price varies considerably for platforms that range from basic voice cloning up to $20-$200 a month or more for extras such as proprietary renditions of emotions and customization of vocal traits. That cost is one that a lot of small businesses or regular old people, at which this product is aimed squarely, might find hard to swallow for what amounts to pro voice cloning
But industry watchers say navigating the divide between quality and accessibility continues to be a challenge. As one AI researcher told me, ”The harder part of voice cloning is creating affordable fake voices that are good enough to fool listeners, but not too good and stripped co-operation in down-stream limits”.
They also bring regulatory challenges as governments struggle to cope with the fast pace of technological innovation. There are many platforms available for people who want to learn more about clone voice online, but navigating these technical, ethical and legal challenges will require careful though by the user.