Speech Synthesis From A Spectrographic Trace

Abstract

A voice synthesis system using a sound spectrogram, showing the spectrum in the form of a plot of frequency against time with intensity being represented by the variable density of the plot. The spectrogram is scanned and analog signals are produced the amplitude of which is a function of the density of the spectrogram plot. Synchronously with the production of the analog signals oscillation signals are produced at the respective scanning frequencies and are amplitude modulated by the analog signal. The amplitude modulated oscillation signals are stored and summed and subsequently reproduced thereby synthesizing an acoustic signal.

Description

FIELD OF THE INVENTION

My present invention relates to sound analysis and synthesis and, more particularly, to the analysis or synthesis of voice signals as well as acoustical signals from other sources using a form of spectrographic recordal and breakdown.

Sonic analysis can be carried out by recording an acoustic signal or an acoustical waveform to enable subsequent analysis of the recorded waveform, based upon Fourier transforms, into a spectrum consisting of a series of component waves which are commonly shown in graphic form as a printed diagram or sound spectrogram.

A sound spectrum, according to the principles to be applied here, therefore, is a representation of the breakdown or analysis of a composite acoustical signal, from whatever source, into its frequency components, the frequency increments of the spectrum being chosen in accordance with the desired resolution of the spectral analysis.

The spectrogram, then, is a diagram of the spectrum in which time may be plotted along the abscissa while frequency is plotted along the ordinate. Furthermore, the amplitude or relative intensity of the respective frequency components is indicated by the relative density of the plot so that, for example, the absence of a trace represents lack of a signal of the particular frequency whereas a light trace represents a low-amplitude component of the particular frequency and a relatively dark trace represents a relatively high amplitude of the frequency component under discussion.

Spectrographic techniques of this type, wherein acoustical signals are "analyzed" or dissected to yield the components making up the composite signal, provide valuable information on the sound for various purposes. For example, vocal sounds may be analyzed in speech research to identify characteristic patterns of component frequencies, known as formants, or phonemes, and are also concerned with formant changes as a result of context, intonation and stress of spoken words or sentences. These analyses are highly desirable in the evaluation of defective speech patterns, in the identification of the oral actions leading to particular formant characteristics, and even in the standardization of speech.

The resultant body of information on vocal sounds, therefore, is extremely valuable since it enables an understanding of the complicated phenomena of speech production and enables the design of new modes of speech transmission so as, for example, to provide automatic speech recognition, automatic speech transmission and automatic sound synthesis. All of these automatic results are most desirable to facilitate the interaction of man and machine at so-called "man/machine interfaces," e.g., as computer inputs and readouts.

Another function of sound spectrograms of the type described above and especially voice spectrograms, is identification of the speaker, it having been found that considerable differences exist in the spectrograms produced by different individuals speaking the same or similar words, these differences being a consequence not only of different speaking habits and accents but also to significantly distinct functional and structural differences in the vocal organs of these individuals. It thus has been proposed to provide so-called "voice prints" or "voice signatures" and analysis thereof to aid in criminal investigations or otherwise enable mechanized identification of selected individuals by comparing a present input with a previously recorded or stored reference.

Spectrograph analysis is also recognized as a source of information permitting evaluation of various functional disorders in any organism generating or responding to acoustic waves or signals which can be converted to such waves without introducing additional variables. Thus, for example, a diseased heart or a malfunctioning engine can be spectrographically evaluated and the existence of the defect and/or the nature of the defect diagnosed spectrographically.

In all of the foregoing applications and in diverse other fields making use of acoustic analysis and acoustic synthesis, it is convenient to synthesize or recompose an acoustic signal from the spectrographically reproduced data. Thus acoustic synthesis may be significant when the analyst wishes to work with an audible signal or when modification of the recorded data is desired in, say, systems analysis. In the latter case it may be desired to subtract, to add or to modify a portion of the data recorded in a spectrogram to eliminate background noise, or even to isolate a particular signal. Furthermore, this reproduced acoustical signal may even form the input of an analysis system following the basic step previously set forth of recording the amplitudes of the various frequency components against time.

OBJECTS OF THE INVENTION

It is the principal object of the present invention to provide a simple, economical and yet reliable method of and system for the synthesis of acoustical signals or the reproduction thereof from spectrographic data resulting from analysis.

Another object of the invention is to provide an improved sonic-analysis system enabling a wider range of investigation of acoustical inputs than has been possible heretofore and capable of producing results in sonic analysis which are of greater practical and theoretical significance than earlier systems.

Still another object of the invention is to provide a sonic analysis and synthesis system capable of making available more information as to the nature of analyzed sound signals than has been obtainable from prior analysis systems.

It is yet another object of my invention to provide a synthesis system for generating artificial yet recognizable speech sounds for use at man/machine interfaces of the character described and for communication purposes.

It is also an object of the invention to provide a general-purpose sonic-analysis system yielding significant results on objective and subjective levels which is capable of providing objectively significant engineering and like data, but which can consider subjective factors as well.

Still another object of the instant invention is to provide an improved system for breaking down sound waves into components thereof.

These objects and others which will become apparent hereinafter, are attained, in accordance with this invention, with a novel sound-synthesis and sound-analysis system and method, based upon the recognition that it is possible to further derive, from a spectrographic breakdown of an acoustical signal, a further signal in which incremental frequencies are scanned and an analog output is formed, based upon the amplitudes of each frequency increment such that the analog signals are combined, composed or synthesized into an acoustical output and/or an electrical or other signal representing same.

Thus, the present invention in generic terms makes use of a spectrographic trace of any acoustic input, as will be prepared by conventional techniques or special apparatus described hereinafter, in which the component waves of the acoustic signal are plotted as incremental frequency bands (i.e., individual frequency components), against time, with the amplitude of the particular function being represented by varying density of the particular plot. If the frequency distribution of the acoustic signal is considered to be a function of time and overall amplitude is a function of time, it will be seen immediately that each component frequency may also be viewed as a function of time.

In the breakdown of the input signal, the value of the signal for each of a multiplicity of component frequencies or frequency bands is ascertained and plotted against time in a spectrographic trace which generally makes use of visible indicia, although invisible traces (e.g., magnetic recordings) may also be employed. The intensity (amplitude) of the component frequency or component frequency band is therefore indicated by some parameter of the trace, generally a passage from relatively light to relatively dark as amplitude increases when optical traces are considered. Any form or registry using the same principle is, however, contemplated within the terminology used here of high density and low density, which may identify optical lightness and darkness, spreading or narrowing of a recorded component signal, greater recording depth or shallowness, etc.

The expression "spectrographic trace" is used to denote any trace or recordal of spectrographically analyzed signals in which a composite or mixed sonic input is broken down, dissected or analyzed to yield the respective frequencies and the relative amplitudes of these component frequencies can be ascertained, preferably in an instantaneous manner or as a function of time. The spectrographic trace may be created from an acoustical signal of any form and any source and practically any machine, individual or system capable of producing sound material constitute an input from which a spectrographic trace is taken. The input may be used for investigation or for any of the other purposes mentioned earlier and may be derived for voice analysis, speech sound analysis, musical sound analysis, noise analysis, acoustical signals associated with the detection, diagnosis and evaluation of dysfunction of organic or mechanical systems, or acoustical signals from individuals, machines and systems for purposes of identification.

The term "spectrographic trace" is used herein not only to refer to an original trace or spectrogram of the raw acoustic input, but also to refer to modified traces, traces which have been copied, traces which have been partly canceled or selectively evaluated or traces which have been produced by a previous sonic synthesis using, for example, the techniques described in greater detail hereinafter.

According to the principal feature of the present invention, a spectrographic trace of the character described is scanned successively for incremental frequencies to derive analog outputs or signals whose amplitudes vary in proportion to the intensities or densities of the respective plots of the component or analyzed frequencies.

It must be noted here, to avoid confusion, that many acoustical signals will provide a continuum of component frequency, while other acoustical signals will consist of relatively pure tones, relatively pure or simple overtones and mixtures of such tones, overtones and harmonics. Still other signals will be revealed as having one or more continuum-type frequency bands together with one or more pure tones, pure overtones, etc. Thus, when reference is made herein to "component frequencies," it should be understood that not necessarily every frequency of a particular sound signal is to be recorded individually and thereafter scanned. It has been found that it is frequently practical to record frequency bands of a band width which will depend upon the purpose of analysis. On the other hand, for some purposes it may be desirable to make use of extremely narrow bands or practically pure component frequencies or to space the component frequencies to be derived more or less from one another. When voice analysis is contemplated, it may be sufficient to record substantially contiguous component frequency bands of a band width between 10 and 200 Hz, for example.

Synchronously with the respective scanning steps, isolation signals are generated at the respective scanning frequencies and amplitude modulated by the analog signals, the amplitude-modulated signals being recorded for subsequent playback in the synthesis stage of the invention.

In accordance with one aspect of this invention, the amplitude-modulated signals are recorded in parallel but coincident and co-ordinated tracks of a single recording medium and, upon complete recording, a synthesized output is produced by combining the recorded signals as they are reproduced as a function of time.

It has also been found to be possible to reduce the number of tracks and therefore the densities of the recording medium by a system which may be described in terms of a "seesaw" effect. In accordance with this technique, each incrementally scanned signal is recorded as scanned but each subsequent scan and recordal combines the previously recorded signal so that the signal recorded at the second scan will contain two components while that recorded at the third scan will contain three components, etc. In other words, the scanning step is combined with a mixing step so that only a single track is finally required to record the synthesized signal. In this arrangement, therefore, a pair of narrow recording media, e.g., magnetic tapes, can be used. On a first of these tapes, I record the amplitude-modulated oscillation signal corresponding to one of the incremental frequency scans. This signal is then reproduced and mixed with the amplitude-modulated oscillation signal of the next scan to provide a resultant or mixed signal which is recorded upon the second tape while the first tape is erased. During the next scan, the mixed signal of a second tape is combined with a fresh amplitude-modulated oscillation signal of the third scan while the second tape is erased and the tripartite mixed signal is applied to the first tape. In this manner a mixed signal of increasing complexity is recorded first on one and then on the other tape and may be played back as the completely synthesized signal from the final tape.

According to another aspect of this invention, the spectrograph is produced by an improved and highly sensitive recordal system having pickup means with a mechanical resonance element, preferably a tuning fork and means for continuously varying the resonant frequency of this device. The resonant system thus serves as a variable-frequency monochromatic pass filter or regulator capable of isolating the particular frequency to which the resonance element has been set and thereby enabling a recording device to be energized with an input corresponding to the time-variation of the amplitude of this particular frequency. More specifically, the variable-frequency monochromatic signal isolator comprises a tuning fork having a tuning member movable by a lead screw or the like between the shanks of the tuning fork, which is energized at one of its shanks by an electric signal derived from an acoustical signal by mechanical-electric transducer. The other shank co-operates with a mechanical-electrical transducer feeding an amplifier or output device to generate an electrical signal which is proportional to the amplitude of the resonant frequency of the tuning fork at the particular instant and the specific setting thereof.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects, features and advantages of the present invention will become more readily apparent from the following description, reference being made to the accompanying drawing in which:

FIG. 1 is a diagrammatic representation, partly in block form, of a spectrographic, recording system for use with the system of the present invention;

FIG. 2 is a sound spectrogram (or trace simulating a sound spectrogram) or visible recording as made of the speech sound or word ("hallow") by the system of FIG. 1, in which the frequency (f) is plotted along the ordinate and time (t) is plotted along the abscissa with the intensity (I) being indicated by visually ascertainable plots;

FIG. 3 is a graph showing the frequency-amplitude relationship of the component waves taken at a certain instant of said speech sound (corresponding to the double-L sound of "Hallow");

FIG. 4 is a partly perspective view, in diagrammatic form, of a sound-synthesis system embodying the principles of the present invention;

FIG. 4a is a circuit diagram of an amplitude-modulation and gate system which can be incorporated in the synthesis system of the present invention;

FIG. 4b is a graph representing the signal of FIG. 4a;

FIG. 5 is a diagrammatic view of another system according to this invention generally similar to the system of FIG. 4;

FIG. 5a is a block diagram of an input device with a conventional electrical heterodyne system;

FIG. 6 is a circuit diagram of a self-intensifying optical scanner according to this invention;

FIG. 7 is a view of another circuit useful for control of a switching device according to this invention;

FIG. 8 is a diagram generally similar to that of FIG. 2 but illustrating a seesaw recording method, according to the invention; and

FIG. 9 is a block diagram illustrating the switching of the recording and play-back devices of the latter system.

SPECIFIC DESCRIPTION

In FIG. 1, I show a spectrograph or registering system for preparing a spectrogram of a recorded acoustic input in which component waves are plotted incrementally in coordinate form of which ordinate represents frequency and abscissa represents time, the intensity of the component waves being indicated by relative darkness of the plots.

In this system, an audio input is picked up by a microphone, stethoscope, electroacoustic transducer, strain gauge, geophone or other sonic pickup 11 and recorded in a tape recorder 10. The recorded sound signal is repeatedly iteratively reproduced as an electrical signal which is transmitted via the usual amplifier 12 to an electromechanical transducer 13 of the electromagnetic type.

The system 13, 14, 15, described in detail hereinafter, constitutes a mechanical filter adapted to sweep the frequency spectrum of any acoustic source and register the intensity (generally time-varying) of the component waves individual to each of these frequencies. To this end, the scanning means includes a tunable vibratile element such as a tuning fork whose legs 14a and 14b tend to vibrate upon electromagnetic energization at the tuned frequency with an amplitude proportional to the electromagnetic energization amplitude.

Thus, the electromechanical vibrator 13 can include a magnetic yoke 13b energizable by a coil 13a connected across the amplifier 12 and juxtaposed with the leg 14a of the tuning fork. The other leg 14b of the tuning fork is juxtaposed with an electromagnetic pickup 15 whose yoke 15a, of the magnetic-carriage meter type, cooperates with a coil 15b to respond to vibration of the leg 14b. The flux changes in the core 15a, which are proportional to the amplitude and velocity of vibration of the leg 14b, induce an electrical output which is amplified at 16 and delivered at 17 to the recording unit.

While substantially any pen, stylus or electrode recorder may be used for the present purposes, the description with respect to FIGS. 1-4 will be confined to stylus-type recording devices adapted to produce an electric discharge between the stylus 22 and a substrate in the form of a recording paper or sheet 21, wound around an electrically conductive drum 18, to plot the component wave with darkness proportional to the intensity. A suitable recording sheet may compose of carbon substrate coated with a thin layer of a metal oxide such as titanium oxide, the amplitude-responsive discharge effecting breakdown of this dielectric layer to expose the carbon substrate with intensity or density proportional to the amplitude or intensity contribution of the scanned frequency.

In the system of FIG. 1, the recorder consists of the drum 18 carried by a shaft 19 and driven at a constant speed by motor 20 for each frequency scanning. The output from the recorder circuit 17 to the stylus 22 is applied between the latter and the drum 18 via a brush 18a bearing on the shaft 19 of this drum.

The stylus assembly 22 is shiftable upon its carriage 23 by a leadscrew 24 driven via a transmission 25 and a motor 26 synchronously with the tuning means to provide incremental frequency scans as represented at f in FIG. 1.

The incremental frequency-scanning shift is effected via an output shaft 27 of motor 26 and a transmission 28, 29 which drives a leadscrew 14c adapted to shift a tuning block 14d in the direction of arrow 14e between the legs 14a and 14b of the tuning fork. In the system illustrated, the frequency spectrum is scanned from the lower frequency end to the higher frequency end, although the reverse scanning may be desired from time to time. It will be understood that the stylus 22 generates spark discharge on the recording sheet 21 with intensity of breakdown which represents the intensity of the particular frequency to which the fork is tuned and at the particular instant. The overall result of the scan is represented by the spectrogram in FIG. 2. FIG. 3 represents the frequency-intensity relationship or weighted frequency distribution of the component waves at a selected instant of time.

In accordance with the principles of this invention, an acoustic output is synthesized or reproduced from a spectrogram as shown in FIG. 2 or its modified or copied form. The system of FIG. 4 thus can comprise a drum 50 carrying a spectrographic trace 51 as indicated. The drum 50 is mechanically coupled, via a clutch 52, with the drum 53 carrying a recording sheet such as a magnetic, electrostatic or optical recording sheet adapted to cooperate with a recording head 54 and a multiplicity of pickup heads or a unitary head 55 whose function will be described in greater detail hereinbelow.

The drums 50 and 53 are jointly driven at an identical angular velocity by a transmission 68 from a constant-speed motor 71. Another output of the latter drives a leadscrew 56 whose carriage 57 supports the head 54 for movement along a generatrix of the drum 53. A further leadscrew 58 is coupled with leadscrew 56 via the slip clutch 59 and shifts a scanning head 60 upon the carriage 61 along a generatrix of drum 50. The leadscrew 58 also displaces a frequency-shift member 62 of a variable-frequency oscillator 63 (e.g., the movable member of a tuning condenser of a Hartley oscillator) adapted to generate the corresponding frequency oscillation signal to be fed to an amplitude-modulation gate 64. The latter receives from the pickup head 60, which can be of the optical type illustrated in FIG. 6 or FIG. 7, a control signal (analog signal) 66 as a function of the intensity of the component wave registered on the spectrogram 51, thereby delivering the amplitude-modulated oscillation signal upon amplification by amplifier 65 to the recording head 54.

In operation, the spectrographic trace 51 is applied to the drum 50 and this drum, synchronized with the recording drum 53, is set in rotation at the peripheral displacement rate at which the original sound was recorded or the synthesis sound is to be produced. The pickup head 50 adapted to effect step movement synchronous with the recording head, scans each incremental frequency of the trace 51 and produces an output at 66 whose amplitude is a function of the time-variable intensity of the respective component wave represented by darkness of the time-frequency plots and provides at the output of the gate 64 the amplitude-modulation signal to be amplified by the amplifier 65 and energizes the head 54 to record this amplified signal upon the drum 53 for reproducible manner.

Upon completion of one frequency scan, the pickup head 50, the frequency-shift member 62 and the recording head 54 are set to the next incremental frequency position; the above-mentioned scanning and recording operation is repeated until the entire frequency range is translated from the visual spectrographic trace to the reproducible recording media. It will be understood that, depending on the purpose of the analysis and synthesis, a selected portion of the spectrographic trace may be canceled from the record of the data; in this case a corresponding blocking signal may be applied to the gate 64. The synthesized sound will not have the eliminated portion of the spectrum.

The translated record upon recording drum 53 is synthesized into a sound signal, in accordance with this invention, by reproduction head 55 which can be composed of a multiplicity of pickup heads as shown arrayed along a common generatrix of the drum or an elongated unitary pickup head spanning the generatrix, and connected to an amplifier 69 whose output energizes a loudspeaker 70 or otherwise is magnetically recorded for direct acoustic reproduction.

When the synthesized output is to be modified by suppression or augmentation of the selected frequencies at this stage, the head 55 which in this case is a multiplicity corresponding in number to the incremental frequencies are connected via respective variable-gain amplifiers 55a to the common output amplifier 69 and the individual amplifiers adjusted accordingly. Furthermore, the relative weights of the respective frequencies or of a portion thereof can be evaluated by tapping the magnetic heads 55 selectively. Inasmuch as the head or heads 55 pick up the entire frequency spectrum simultaneously, the output at 70 corresponds essentially to the input at 11 (FIG. 1) when no modification is entered in the spectrogram or the scanning stage and when the peripheral speed of drum 53 is identical to the peripheral speed of the recording drum 18.

In FIG. 4a, there is shown a circuit specifically designed for use as amplitude-modulation gating means 64 of FIG. 4. This circuit includes an input transformer T.sub.i whose primary winding is energized by the constant-voltage, variable-frequency oscillator 63 whose output frequency is regulated by the shift member 62 as previously described.

The transformer T.sub.i has its secondary winding divided into two sections by a center tap c, the end terminals of this secondary winding being connected across a high-impedance primary winding of an output transformer T.sub.o via high-ohmic resistors R.sub.1 and R.sub.2, respectively, as shown. Across the secondary winding of the output transformer T.sub.o is provided a load resistor RL at which an amplitude-modulated oscillation appears as will be described. The circuit further includes a pair of NPN transistors Tr.sub.1 and Tr.sub.2 having their respective emitter electrodes interconnected at a junction connected via a diode D of the indicated conduction orientation to the centertap c of the secondary winding of the input transformer T.sub.i. Collector electrodes of these transistors are tied to the line connecting the high-ohmic resistor R.sub.1 to the one terminal of the primary winding of the output transformer T.sub.o and the line connecting the other high-ohmic resistor R.sub.2 to the other terminal, respectively. The resistors R.sub.1 and R.sub.2 are of an identical ohmic value. Base electrodes of the transistors Tr.sub.1 and Tr.sub.2 are connected via their respective base resistors R.sub.3 and R.sub.4 commonly to one terminal E of an analog input while the common emitter junction of these transistors is connected to the other terminal of this analog input.

The input comprises a battery G of a fixed bias voltage and an input resistor R.sub.5 across which is applied an analog signal 6 of the opposing polarity to the fixed voltage G and detected by the pickup head 60 as described earlier.

Let it be assumed that no signal appears across the input resistor R.sub.5 ; then the transistors Tr.sub.1 and Tr.sub.3 are in conduction because of the fixed bias voltage G. Since the primary winding of the output transformer T.sub.o is designed to be high-impedance as mentioned earlier, the positive and negative cycles of a transformed input alternating current V.sub.ab (see FIG. 4b) appearing across the terminals A-B are both short-circuited through the conducting transistors Tr.sub.1 and Tr.sub.2 and the resistors R.sub.1 and R.sub.2, respectively, and through the common line of the diode D with the result that substantially no signal appears across the output transformer T.sub.o, hence at the output resistor RL.

In scanning operation, let it be assumed that an input signal of waveform V66 shown in the graph 2 appears at the resistor R.sub.5 ; then terminal voltage V.sub.ef will assume the voltage waveform of the graph 3 of FIG. 4b as the difference between the fixed bias voltage V.sub.g and the input voltage waveform V.sub.66 which causes the collector-emitter impedance of the respective transistor Tr.sub.1 and Tr.sub.3 to vary in inverse proportionality to the net voltage V.sub.ef across the terminals E-F. As the result, it will be readily appreciated that the amplitude-modulated oscillation signal as shown V.sub.jk in the graph 4 appears at the terminal J-K of the output transformer T.sub.o and the corresponding waveform signal at the output resistor RL.

In FIG. 5, I show a system generally similar to that of FIG. 4 wherein, however, the spectrographic trace 51 of frequency/amplitude against time is generated by a stylus such as that shown at 22 in FIG. 1 and here represented at 122. The block 113, of course, represents the audio input and circuitry of the recording means of FIG. 1 or the similar device which may have the conventional construction with an electrical heterodyne system as shown in FIG. 5a and is energized by a microphone, stethoscope or like pickup means 107.

In this embodiment, the recording drum is replaced by a roller 150 around which the belt 151a passes, an idler roller 150a being spaced from the roller 150. A magnetic recording belt 152 likewise passes over the rollers 150 and 150a for recording the frequency/amplitude vs time scan. In this system, the recording and translating means are provided in one apparatus, the stylus assembly 122 being step-shifted in the direction of arrow f accomplishing each incremental frequency scan by a leadscrew assembly of the type illustrated at 23-28 of FIG. 1.

In this case, the leadscrew of the stylus assembly 122 may be coupled with the motor 180. The motor 180 drives a leadscrew 56, 156, 158 along which the optical pickup head 160 is shiftable jointly with the magnetic recording head 154. An array of closely spaced narrow-band playback heads 155 (or a single elongated head) is provided along a generatrix of the belt 153 and connected by independent variable-gain amplifier (not shown) to the common amplifier 161 and loudspeaker 170.

In this system, the variable-frequency oscillator 163 is regulated by a control member 162 driven by the lead-screw 156, 158 so that the frequency supplied to mixer 164 is tuned to the frequency band with which the pickup head 160 is aligned. A constant-frequency oscillator 165, for generating the carrier frequency for the recording head 154, is also connected to the mixer 164 which feeds an amplifier 181 and an electric switch 182. A synchronizer synchronizes the output signal of the variable-frequency oscillator 163 with that of the constant-frequency oscillator 165.

The electronic switch or gate 182 which may be that shown in FIG. 4a is controlled by the output from head 160 as represented at 167 and illustrated in FIG. 7. The system of FIG. 5, of course, operates essentially in the manner described with reference to FIG. 4.

The input system illustrated in FIG. 5a and energized by a mechanical-electrical transducer, includes the microphone 311 which feeds a magnetic tape recorder 311a. The variable frequency-oscillator 311b provides the carrier frequency which is modulated by the taped input at a mixer 311c before passing via a filter 311d to the amplifier 312. Recording is done at 321 by stylus or the like, energized via the amplifier 316 which is fed by an amplitude-modulation network 316a to which a base signal is supplied by the constant frequency oscillator 316b.

In FIG. 6, I show a pickup circuit with a self-intensifying feedback for use at 60 or 160 in the systems of FIGS. 4 and 5, for example, and for general use in analyzing optically sensed traces of the present recording system. The system of FIG. 6 comprises a light source 260a which focuses a beam 260b of illumination upon the recording medium 251 via an optical system 260c. A reflected beam 260d is picked up by an optical system 260e and focuses via an iris-type diaphragm 260f upon the audio-optically sensitive transducer 260q. The latter may be a phototransistor connected to an amplifier circuit 260h with the reflected beam 260d forming the control element or base of the transducer.

In a simple version of this circuit, the phototransistor 260q is connected in series with a source of electric current, a load and a current transformer 260i whose output is applied across a resistor 260j. The latter is connected in series with a d-c source 260k shunted by an a-c passing capacitor 260l in series with the filament of the lamp 260a.

The output of amplifier 260h, upon phase reversal, is delivered to the recording system 266 etc. as described in connection with FIGS. 4 and 5 (i.e., to the gate 64, 182). Thus the output received at 260q is proportional to the optical recording upon layer 251 and delivers the appropriate control signal to the recorder 266, while simultaneously intensifying this signal by applying an augmented current to the lamp 260a. This positive feedback ensures heightened contrast even with relatively light traces upon the recording medium 251.

In the system of FIG. 7, a nonintensifying arrangement is illustrated and here the lamp 360a produces a beam 360b which is directed at the recording medium 361 via an optical system 360c, the reflected beam 360d being picked up by the optical system 360e and focused upon a photoresistor 360q connected in shunt across the output side of the variable frequency oscillator or the mixer 364 as described earlier. The switching system is represented diagrammatically at 382 and is here designed to short-circuit the frequency output 364 so that an output is delivered to the recording head 54 or 154 only when the reflected beam is of diminished intensity because of absorption at the spectrographic plots and, yet with an amplitude of the output oscillation signal proportional to the relative darkness of the plots.

In FIG. 8, I have shown a sound-synthesis system in which a motor 520 drives a shaft 519 on which is mounted a first drum 521, via a clutch 519a. The drum 521 carries a record sheet, e.g., a visible sound spectrogram, which is scanned by the optical scanner 522.

The drum is mechanically coupled with a second drum 552 carrying a pair of magnetic tapes 552a and 552b, the drums being rotated in synchronism and at a constant speed via the shaft 529b and a magnetic clutch 519c. The record sheet is provided with a spectrographic trace 504a of a given sound in which component frequency waves are plotted incrementally against time with the intensity of the component waves being represented by optically ascertainable conditions, preferably relative darkness of the plots. A spectrogram of this type has been illustrated in FIG. 2.

The drum 521 is rotated to sweep each component frequency past the scanner in a particular "time position " of the latter. Parallel to the generatrix and axis of the spectrogram trace, I provide a shaft 524 which is driven by the motor 520 via a transmission represented at 508. The carriage of the optical scanner 522 is mounted upon the leadscrew formed by the shaft 524 so that the optical scanner can be shifted continuously or stepped in the direction of the arrow shown in FIG. 8 (to the left) upon each rotation of the drum 521 or upon the completion of each complete frequency scan in the direction of the time axis.

A variable-frequency oscillator is provided at 510 and has an output frequency tuned to the frequency scanned by the optical scanner 522, the tuning being effected by a frequency-control lever 510a shifted by the leadscrew 524 synchronously with the optical sensor 522.

Adjacent the drum 552 and along a generatrix thereof, I provide a pair of magnetic tape heads 554a and 554b confronting the respective magnetic tapes 552a and 552b, the tapes extending around the drum 552. The tape heads 554a and 554b each operate alternately as a recording and reproducing head as will be apparent hereinafter. In juxtaposition with tapes 552a and 552b, I also provide respective erasers or cancelation heads 512a and 512b designed to clear previously recorded signals from each of the tapes.

At 513 there is provided a gate-type modulator at which the signal from the oscillator 510 may be amplitude modulated by a control or switching signal received from the optical sensor 522 and representing the instantaneous intensity of the frequency-time plots of the trace 504a. An amplitude modulated signal is then fed to a mixer 514 which may be supplied, via the reproducing stage 515 of the tape play-back system, with a reproduced signal from one of the magnetic tapes 552-a or 554b as derived by the head 554a or 554b which operates in the reproduced or play-back mode. The other head, of course, simultaneously operates in the record mode. A filter 516 (See FIG. 9) may be provided to remove background noise or selectively modify the acoustical signal delivered to the mixer from the tape 552a or 552b. The mixed signal is magnetically recorded on the previously erased tape 552a, 552b, by a recorder 517. An oscillator 518 is provided to supply to the recorder 517 an alternating bias current for magnetic recording and also to supply a canceling signal to the erase head 512a, 512b via a switch 619. A switch 620 selectively connects the output of the recorder 517 to one of the recorder circuits of the respective heads 554a.sub.1, 554b.sub.1 while a switch 621 selectively connects one of the reproducing circuits 554a.sub.2 and 554b.sub.2 of the heads with the reproducer 515. A further switch 622 can switch the output of a filter 516 into an electrical-acoustic transducer 623 via an amplifier 624 for reproduction of an acoustic output.

In operation, let it be assumed that the switches 619, 620, 621 and 622 are in the respective positions indicated in FIG. 9 for scanning of frequency F1. Then, the output frequency of the oscillator 510 is set in F1 by the lever 510a and this output signal is amplitude-modulated at the gate 513 in the manner previously described. The amplitude-modulated oscillation signal is fed to the mixer 514 and, in the absence of any input from the reproducer 515, directly to the recorder 517 which then energizes the recording head 554a.sub.1 to magnetically record said signal on the recording tape 552a on which any previous recording if present has been canceled by the erase head 512a.

When the optical head 522 completes the frequency F1 scan of the spectrographic trace as the drum 518 accomplishes one rotation, the drive 520 is temporarily stopped or continuously shifts the head 522 to the next frequency-scanning position F2, while shifting the output frequency of the oscillator 510 to F2 via the lever 510a. At the same time, switches 519, 520 and 521 are thrown into positions 512b, 554b.sub.1 and 554a.sub.2, respectively. In this step, the prior recording of frequency F1 registered on the tape 552a in the preceding step is erased by head 512a, but the signal is reproduced via the head 554a serving now as a reproducing or playback head, the reproducer 515 and the filter 516, and is fed to the mixer 514 while the tape 554b receives a recording signal from the head 554b serving in this step as a recording head.

Thus, upon commencing scan, an amplitude-modulated signal of frequency F2 is derived at the output of the gate circuit 513 and combined at the mixer 514 with the reproduced signal of frequency F1, the resultant mixed signal F1+F2 being recorded on the tape 552b via the recorder 517 and the recording head 554b as the optical scanner proceeds in the scanning of component frequency F2 on the spectrographic trace.

Subsequently, for scanning the next frequency component F3, the switches 519, 520 and 521 are returned to the indicated positions and the frequency of the oscillator 510 is shifted to F3 with the scanning head 522 shifted to the position corresponding to F3. An amplitude-modulated oscillation signal of frequency F3 is generated under control of the optical scanner 522 and fed to the mixer 514 which also receives the reproduced signal F1+F2 from the tape 552b.

The resultant mixed signal F1+F2+F3 is recorded on the tape 552a while the previously recorded signal F1+F2, being presently reproduced, is erased from the tape 522b by the head 512b.

With the above-mentioned steps repeated, the overall frequency spectrum or any interesting portion thereof is recorded finally either on the tape 552a or 552b for reproduction into an acoustic wave. For reproduction, the switch 522 is thrown to connect the reproducing head 554a.sub.2 or 554b.sub.2, depending on which has the final recording, to the audio amplifier 624 and the speaker 623.

Claims

1. A method of synthesizing an acoustical signal from a spectrograph trace in which component frequency portions of sound are incrementally plotted against time over a predetermined time period with the intensity of the components being represented by varying density of the plots at corresponding frequency-time points, said method comprising the steps of: scanning said trace successively for said incremental frequency portions over said period and deriving analog signals of amplitude as a function of the density of the plot of the respective frequencies; synchronously with the scanning of the trace generating oscillation signals at the respective scanning frequencies during said period and amplitude-modulating said oscillation signals by the analog signals of amplitude to produce amplitude-modulated oscillation signals; recording said amplitude-modulated oscillation signals upon a medium in a multiplicity of tracks; scanning said medium and generating automatically in response to the scanning of the medium a reproduction of the recorded signals to produce a synthesized acoustic signal corresponding to a summation of the

2. The method defined in claim 1 wherein said signals are additively combined by simultaneously reproducing signals from all of the recorded

3. The method defined in claim 1 wherein the acoustical signal reproduced from the recording of said amplitude-modulated oscillation signals is substantially identical to the acoustical input forming said

4. The method defining in claim 3, further comprising the step of subtracting a part of the acoustical signal represented by said spectrographic trace prior to reproducing the recorded amplitude-modulated

5. The method defined in claim 4 wherein said spectrographic trace is a

6. The method defined in claim 1, further comprising the steps of preparing said spectrographic trace prior to the scanning thereof, by plotting component waves of an acoustical input along incremental frequency bands against time over said period while recording the intensity of the component waves as relative lightness and darkness of the respective plots at the corresponding frequency-time points; and modifying at least a portion of the plotted information prior to the scanning of the spectrographic and trace acoustically comparing the

7. The method defined in claim 6 wherein a further spectrographic trace is produced from the spectrographic trace as first prepared by modifying information relating to at least some of the component frequencies, said

8. The method defined in claim 6 wherein the component waves of the acoustic input are plotted successively in said incremental frequency

9. A method of synthesizing an acoustical signal from a spectrographic trace in which component frequency portions of sound are incrementally plotted against time over a predetermined time period with the intensity of the components being represented by varying density of the plots at corresponding frequency-time points, said method comprising the steps of: scanning said trace successively for said incremental frequency portions over said period and deriving analog signals of amplitude as a function of the density of the plot of the respective frequencies; synchronously with the scanning of the trace generating oscillation signals at the respective scanning frequencies during said period and amplitude-modulating said oscillation signals by the analog signals of amplitude to produce amplitude-modulated oscillation signals; recording said amplitude-modulated oscillation signals upon a medium; and scanning said medium and generating automatically in response to the scanning of the medium a reproduction of the recorded signals to produce a synthesized acoustic signal corresponding to a summation of the amplitude-modulated signals, each amplitude-modulated oscillation signal but the first being combined with a previously recorded amplitude-modulated oscillation signal during a previous scan whereby the final recording represents a summation of all of said amplitude-modulated

10. The method defined in claim 9 wherein said trace is a visible spectrogram and said density is the relative darkness of parts of the

11. The method defined in claim 9 wherein said spectrographic trace is

12. The method defined in claim 9 wherein said spectrographic trace is

13. A method of analyzing an acoustic input, comprising the steps of: a. detecting successive incremental frequency bands of the acoustic input and resonating a mechanical element at the band frequency with an oscillation amplitude representing the intensity of the corresponding frequency band; b. recording for each incremental frequency band detected in step (a), a visible plot with the amplitude of the corresponding frequency band at each instant of time during a time interval represented by a corresponding degree of darkness of the plot, thereby forming a spectrographic trace in which component frequency waves of the acoustic input are incrementally plotted against time over a given time period with the intensity of the component waves indicated by relative darkness of the plots at the corresponding frequency-time points; c. optically scanning said trace successively for said incremental frequencies and deriving an electrical output of an amplitude varying with time in accordance with the relative darkness of the plot of the corresponding frequency; d. generating an electrical oscillation of a frequency corresponding to each of the scanned frequencies in step (c) and amplitude modulating the electrical oscillation with the corresponding electrical output of step (c) to form amplitude-modulated oscillation signals; e. electrically combining the amplitude-modulated oscillation signals of step (d) for the incremental frequencies; and f. automatically converting the combined amplitude-modulated oscillation

14. The method defined in claim 13 wherein said amplitude-modulated oscillation signals are combined in step (e) by recording the first amplitude-modulated oscillation signal upon one recording track, recording the next amplitude-modulated oscillation signal and the previously recorded amplitude-modulated oscillation signal upon another recording track while cancelling the previously made recording, and repeating the recordal and cancellation steps until one of said tracks carries a

15. The method defined in claim 13 wherein said amplitude-modulated oscillation signals are combined by recording them individually upon a

16. An apparatus for the synthesis of an acoustic output from a spectrographic trace in which component frequency portions of an original sound are incrementally plotted against time over a predetermined time period with the intensity of the components being represented by varying density of the plots at corresponding frequency-time points, said apparatus comprising: scanning means for repetitively scanning said trace successively for said incremental frequencies and deriving for each of the scanned frequencies analog signals of an amplitude varying with the time in proportion to the density of the plots of the respective component frequencies; oscillator means operatively connected with said scanning means for producing respective oscillation signal corresponding to each of the scanning frequencies; amplitude-modulating means connected with said scanning means and with said oscillator means for producing an amplitude-modulated oscillation signal as a function of time for each of the scanned frequencies; means for combining at least some of said amplitude-modulated oscillation signals; and electroacoustic transducer means for producing an acoustical output corresponding to the combined amplitude-modulated oscillation signals.

17. The apparatus defined in claim 16 wherein said trace is visible and the varying density of said plots is represented by different degrees of darkness, said scanning means including an optical scanner adapted to sweep said trace and provided with a source of illumination trained at said plots successively and optically sensitive pickup means receiving reflected light from said plots for detecting the degree of darkness

18. The apparatus defined in claim 16 wherein said means for combining said amplitude-modulated oscillation signals includes a single recording medium and means for recording said amplitude-modulated oscillation signals

19. The apparatus defined in claim 16 wherein said scanning means includes a drum carrying said trace and rotatable about an axis parallel to the time axis of said trace and a scanning head displaceable parallel to a generatrix of said drum, said oscillator means comprising a variable-frequency oscillator having a frequency-selection member and

20. An apparatus for the synthesis of an acoustic output from a spectrographic trace in which component frequency portions of an original sound are incrementally plotted against time over a predetermined time period with the intensity of the components being represented by varying density of the plots at corresponding frequency-time points, said apparatus comprising: scanning means for repetitively scanning said trace successively for said incremental frequencies and deriving for each of the scanned frequencies analog signals of an amplitude varying with the time in proportion to the density of the plots of the respective component frequencies; oscillator means operatively connected with said scanning means for producing respective oscillation signal corresponding to each of the scanning frequencies; amplitude-modulating means connected with said scanning means and with said oscillator means for producing an amplitude-modulated oscillation signal as a function of time for each of the scanned frequencies; means for combining at least some of said amplitude-modulated oscillation signals; and electroacoustic transducer means for producing an acoustical output corresponding to the combined amplitude-modulated oscillation signals, said means for combining said amplitude-modulated oscillation signals including a pair of synchronously driven recording tracks, recording means associated with each of said tracks, playback means associated with each of said tracks, cancellation means associated with each of said tracks, and switch means operable to connect the recording means of one of said tracks to said amplitude-modulating means and the playback means of the other of said tracks to said recording means of said one of said tracks to record the amplitude-modulated oscillation signal of the frequency currently scanned by said scanning means upon said one of said tracks concurrently with a previously recorded amplitude-modulated oscillation signal from said other track, while cancelling the recorded signal on said other

21. An apparatus for the synthesis of an acoustic output from a spectrographic trace in which component frequency portions of an original sound are incrementally plotted against time over a predetermined time period with the intensity of the components being represented by varying density of the plots at corresponding frequency-time points, said apparatus comprising: scanning means for repetitively scanning said trace successively for said incremental frequencies and deriving for each of the scanned frequencies analog signals of an amplitude varying with the time in proportion to the density of the plots of the respective component frequencies; oscillator means operatively connected with said scanning means for producing respective oscillation signals corresponding to each of the scanning frequencies; amplitude-modulating means connected with said scanning means and with said oscillator means for producing an amplitude-modulated oscillation signal as a function of time for each of the scanned frequencies; means for combining at least some of said amplitude-modulated oscillation signals; electroacoustic transducer means for producing an acoustical output corresponding to the combined amplitude-modulated oscillation signals; sound-analyzing means for producing said trace and including vibratory means having a resonance-shifting member, means responsive to an acoustical input for energizing said vibratory means; and means responsive to the amplitude of vibration of said vibratory means at

22. The apparatus defined in claim 21 wherein said sound analyzing means includes a recording head adapted to sweep a recording medium for

23. The apparatus defined in claim 21 wherein said vibratory means includes a tuning fork, said member being shiftable between the arms of said fork, said input means constituting a first electromagnetic transducer co-operating with one of said arms and said output means constituting a second electromagnetic transducer co-operating with the other of said arms.