Voice Morphing System for Impersonating in Karaoke Applications using Matlab and generating a run time version

The objective is to develop a real-time system for morphing two voices in the context of a karaoke application, using Matlab or Mathematica. As the user sings a pre-established song, his pitch, timbre, vibrato and articulation can be modified to resemble those of a pre-recorded and pre-analyzed recording of the same melody sang by another person.

The underlying analysis/synthesis technique is based on SMS, to which many changes have been done to better adapt it to the singing voice and the real-time constrains of the system. Also a recognition and alignment module has been added for the needed synchronization of the user’s voice with the target’s voice before the morph is done.

In other word, a form of karaoke but with the background of a professional reciter instead of music and with some voice morphing between the user voice and the professional singer voice and with some basic comparison between the user recital and reference(Professional) user singing. This comparison uses a form of voice recognition technique As detailed in the reference paper.

Detailed functions:

The goal is to come up with a karaoke-type application for Android in which the user can sing like his/her favorite singer and also correct the User singing/glitches by increasing the morphing ration between the user voice and the reference reciting according to how near or far the user singing is compliant to the reference singer.

This will enable the user to enjoy a mix between His favorite singer and his own voice according to his own pace and have a rough correction of his singing by giving the user a feedback of the correct singing as he reads.

. If the reciting is not very far away, the program will alert the user, but will channel the original singer.

There are various Settings to adjust The compliance percentage.

In order to incorporate to the user ’s voice the corresponding characteristics of the “target ” voice, the “ target” voice is recorded and analyzed beforehand. Then the system performs a real-time morphing between the user’s voice and the pre-stored analysis of the target voice. The user is able to control the degree of morphing, thus being able to choose the level of “impersonation ” that he/she wants to accomplish.

The application Should relies on two main algorithms that define and constrict the architecture: the Spectral Modeling Synthesis (SMS) And a Speech Recognizer.

The referenced paper presents the practical implementation aspects of such an application. The theoretical background can be found in other papers Included in the original paper attached.

Notes:

I will provide the complete set of sound files that is needed for the application.

I will contribute to the ongoing testing

A flowchart is required before actual programming.

A run time version is required as well.

A quick prototype is required before the final version

During the preprocessing stage of the original song file, a split between voice and music must be done with both saved into different files.

It is acceptable to split the work between two teams. One for algorithm/Digital Signal processing and the other Android implementation.

The winning candidate should demonstrate knowledge in speech recognition, Digital signal analysis and android programming.

Please check the attachments as they have detailed framework and implementation/algorithm proposal for a similar apps.

Algoritma Matlab ve Mathematica

Proje NO: #7588873

Proje hakkında