Develop a synthesizer that outputs a periodic voltage signal based on a hand-drawn template waveform in real time.
Project Description
The objective of this project is to develop a real-time waveform generator / synthesizer that outputs a periodic voltage signal based on a template waveform that the user draws by hand on paper. The system could also synthesize waveforms based on pre-printed templates. The input to the system will be a real-time video signal, captured using a simple USB camera that is pointed at the template waveform. Custom software (developed as part of the project) will use image processing techniques to extract the shape of the waveform from the incoming video frames. The system output will be a synthesized voltage waveform that is a quasi-periodic extension of the template waveform shown in the video signal.
A system of this type has multiple possible applications:
Waveform generator: The system could be used as a laboratory waveform generator, especially in educational settings, allowing students to explore the frequency content and other characteristics of different waveform shapes by sketching them on paper.
Music synthesizer: The system could be used as an electronic music synthsizer that allows a performer to craft the timbre of an instrumental sound by drawing different waveforms. Other parameters such as pitch and intensity could also be controlled by changing the position and/or size of the template waveform in the input video stream, allowing the system to be played as a standalone instrument using the video input stream as a user interface.
Sonification: The word sonification refers to the conversion of visual input into audio form so that it can be experienced by means of listening rather than seeing. This has sometimes been proposed as a form of assistive technology for users with visual impairments. In this case, the conversion of a visual shape into a waveform output that can be played as audio could allow a user to recognise different visual forms based on their characteristic sounds.
Key Outcomes
Capture video frames from a camera in real time.
Analyse the pixel content of the incoming video frames to extract a waveform shape from the image background and store it as a series of samples in an array.
Generate a periodic output waveform (e.g. as a voltage signal, or as an audio signal played through a computer's audio hardware) that continuously reflects whatever template shape appears in front of the input camera.
Thoroughly test the system using a broad selection of example inputs.
Fully document the design, implementation and testing of the system in the project report.
Produce a compelling system demonstration for the final project assessment.
Additional project information
Software development: The technical implementation aspect of this project will mainly consist of software development (preferably in C, C++ or Python).
Hardware: It is envisaged that mostly off-the-shelf hardware will be sufficient for this project. The custom software will run on a laptop, desktop PC, Raspberry Pi, or similar. The camera will be a regular USB webcam or similar. Waveform output could either be via the computer's audio output channel, or using an externally connected digital-to-analog converter (e.g. via an Arduino).
Location: Since the bulk of the implementation work involves software development and the system could be designed to be highly portable, there is a reasonable degree of flexibility in where the work is carried out. However, the primary setting will be in the electrical / electronic laboratories in Central Quad and the system (as it currently stands) should be available for inspection at each weekly meeting.