EP0953969A1 - Method for rendering speech with silence regulation - Google Patents

Abstract

The silent periods between speech signals are detected and the length of restoration gap of the signals regulated using a single measure to produce the speech signals.

Description

In the switched telephone network, speech is transmitted in the form a continuous stream of code words representing the instantaneous amplitude of the voice signal, cyclically sampled and digitized for this purpose. A communication has a permanent transmission channel, or circuit, to flow this continuous flow.

On the other hand, on a computer network of the Internet type, or on some packet radio networks, the code words representing the voices are transmitted in packets, on a channel offering sufficient speed to be able to share the channel in time between several terminals.

Packet transmission over a network, however, generates problems.

In fact, the crossing time of the network is essentially variable, because each packet can follow a variable path and we cannot take count the packets as soon as they are received, because a packet can arrive before the end of the sound reproduction of the previous one, or can, on the contrary, happen after this one. In the first case, there would be, if taken into account, packet crushing, in the second case, dead time.

We therefore interpose a buffer memory in which we store the packets at random rate of their arrivals and we read the packets, for the sound reproduction, at the fixed rhythm of the transmitter thereof, with a delay, relative to their transmission times, which is fixed and sufficiently large, in relation to fluctuations in the durations of transmission, so that the packets were received.

It may even happen that packets are received in order different from their broadcast, so you have to number them at the issue to restore the order in restitution.

All of this requires the buffer to be relatively large big. However, this delay in sound reproduction becomes perceptible and troublesome for the dialogue between the two interlocutors.

The present invention aims to remedy these drawbacks.

To this end, the invention relates to a method of sound reproduction of speech signals, received in successive packets representing slices successive speech times and temporarily stored before to be restored with sound, a process characterized by the fact that detects the presence of silences, in the slices received, and that regulates the duration of restitution to reproduce, in a single piece, the signals of speech other than silences.

Thus, we modulate the time, at the level of the restitution, for, in practice, compensate for the fluctuation in the times of arrival of the packets. We thus temporally reunites the parts of an original sound sequence which have been separated physically, and therefore temporally, by their put in different packages. As time modulation carries on silences, it is in practice without disadvantage for the understanding the lyrics.

Advantageously, the restitution of any silence that is followed is shortened of a complete sound sequence to be reproduced.

• la figure 1 est un schéma par blocs illustrant un dispositif de mise en oeuvre du procédé de l'invention,
• la figure 2, formée des figures 2A, 2B, 2C, 2D et 2E, illustre le découpage par paquets d'un signal de parole en fonction du temps t, et
• la figure 3 est un diagramme de cheminement illustrant le procédé.

The invention will be better understood using the following description of a preferred embodiment of the method of the invention, with reference to the appended drawing, in which:

• FIG. 1 is a block diagram illustrating a device for implementing the method of the invention,
• FIG. 2, formed by FIGS. 2A, 2B, 2C, 2D and 2E, illustrates the cutting up of a speech signal in packets as a function of time t , and
• Figure 3 is a flow diagram illustrating the process.

The device of FIG. 1 comprises a microprocessor-based controller 1 controlling the operation of a reception and restitution chain sound of speech signals. The chain, connected as input to line 2 of a packet communication network, has as input a circuit 3 for reading the received packets, which identifies the sound segments and the rests and reconstructs their temporal position. According to the type of coding used to represent the voice, explicit reference points may exist that distinguish the segments and indicate their dates, or else such explicit marks do not exist and circuit 3 reconstructs them from the serial number of packets and their content decoded into voice signal whose form it analyzes. Circuit 3 is followed by a buffer memory 4 of storage of the packets, which puts them back in their order of transmission and transmits to a circuit 5 of reproduction or sound reproduction commanding a headset 6.

The operation of the above device will now be explained.

Conventionally, a transmitting terminal, in communication with that of figure 1, analyzes the signal of its microphone by a vocoder for the code in compressed form.

FIG. 2A represents the amplitude S of the voice signal as a function of time t .

In an explicit variant, a vocoder seeks to delimit segments with a voice signal corresponding to a normalized sound in library, such as a vowel, a consonant or a quasi-silence. For the transmission of information representing the signal S, this is then replaced by a series of code words representing the sounds recognized there. The volume of information data is thus very reduced. In reception, the consultation of a similar library allows the original signal S to be restored.

In an implicit variant, the signal is not analyzed so finely for the coding, and it is circuit 3 of the receiver which analyzes the reconstructed signal to locate the segments of silence. In all cases, the information coded is transmitted in packets P1, P2, P3, P4 each carrying a more or less long section of the speech.

In the example of FIG. 2A, the signal S comprises, in the packet P1, the end S0 of a silence, a block of speech energy signal 11 followed of a silence S1 and of another block 12. In the packet P2, the block 12 is continues, with reference 13, and is followed by two blocks 14 and 15 with rests S2 and S3 interposed. The package P3 includes the end of block 15, referenced 16, a silence S4, a block 17, a silence S5 and the beginning 18 of a block followed, packet P4, of an end 19 of the block then of the start S6 of a silence.

Blocks 11, 12-13, 14, 15-16, 17, 18 here represent the six sequences respective: "and" "the invention" "is" "new" "and" "inventive" (figure 2B).

On reception, block 12-13 (just like 15-16 and 18-19), spread over the two packets, P1 and P2, may be temporarily separated in two when of its sound reproduction. We therefore seek here to avoid it.

The instants t0, t1, t2, t3 (fig. 2A) delimit the slices of the initial signal assigned to successive packets P1, P2, P3, P4. References t'0, t'1, t'2, t'3 (fig. 2C), uniformly translated with respect to the instants respective t0, t1, t2, t3, mark the corresponding theoretical dates of playback in the earpiece 6. Due to the fluctuation in the delay transmission, packets may arrive early or late. In this example, packet P2 arrives after a dead time or delay R following the instant t'1 of end of restitution of the packet P1. On the contrary, the P3 package, although arriving after t'2, arrives ahead of the end of the restitution of the P2 package.

"et..l'invention...est...nouvelle...et...inventive",

dans laquelle "..." représente un silence naturel comme S1, devient :

"et...l'inven□ □ □ tion...est...nouv/elle...et...inventive"

où "□ □ □" représente un silence parasite qui s'interpose (de durée quelconque), coupant le mot "invention".

et "/" représente une superposition additive, ou un écrasement, entre le début et la fin (qui arrive trop tôt) du mot "nouvelle", le mot "inventive" étant de même déformé.

Figure 2D illustrates a restitution which would be immediate. The phrase :

"and..the invention ... is ... new ... and ... inventive",

in which "..." represents a natural silence like S1, becomes:

"and ... the invention □ □ □ tion ... is ... new ... and ... inventive"

where "□ □ □" represents an interfering parasitic silence (of any length), cutting the word "invention".

and "/" represents an additive superimposition, or overwriting, between the beginning and the end (which arrives too early) of the word "new", the word "inventive" being likewise distorted.

"." représente un silence d'origine qui a été abrégé et

"....." représente un silence d'origine qui a été allongé.

FIG. 2E illustrates the method of the invention. On this one,

"." represents an original silence which has been abbreviated and

"....." represents an original silence which has been lengthened.

We notice first of all that the parasitic silence □ □ □ of figure 2D has disappeared, as well as the overlay "/".

When packet P1 arrives, the entire sound block 11 ("and") is restored here immediately (we assume that we were in a phase of restitution of the start of the S0 silence transmitted in the preceding packet). However, the sequence 12 ("the inven") of block 12-13 is not restored at this time, because it is only a portion of a sequence. S1 silence is elongated (.....) until reception of the P2 packet, for which the sequences 12-13 and 14, with S2 silence, are returned. When arrives packet P3, in relative advance with respect to packet P2, silence S3 is shortened (.) to restore, without notable delay, the sequence: "new.and.". The rests S2 and S4 are here shortened, or abbreviated (.) to empty buffer memory 4 as quickly as possible in order to better tolerate early arrivals of following packages. This is of particular interest in the case for which the duration of restitution of sound sequence (s) triggered by the arrival of a packet is greater than the theoretical period. Indeed an arriving packet can complete an earlier sound block representing an uninterrupted signal duration spanning several time slices, which will then be returned.

Figure 3 illustrates the management for this purpose of the sound reproduction of packets received.

We start here from a state 21 in which the circuit 5 restores a sequence sound like 12 - 13 or 14. When the end of it arrives, so the beginning of a silence detected by circuit 3, we test, in a step 22, the presence in buffer 4 of the continuous description of the signal until at least the start of the next segment or sound block. In if so, we test, in a step 23, if the delay exceeds a high threshold A. The total duration of the signal to be reproduced in buffer; the buffer filling rate can be one convenient approximation. We can choose a threshold A zero, that is to say then go from step 22 to step 24, indicated below, back to state 21.

If yes in step 23, we shorten, in step 24, the silence in course or even delete it and play back the above sound sequence by transition to state 21. If not, in steps 22 and 23, we pass to a state 26 of silence reproduction.

We can also go to state 26, from state 21, by inserting a silence additional, if step 25 is detected as a negative overflow of the memory 4 (no information available in memory, relating to next segment to be reproduced).

At state 26, it is also detected if the memory 4 is empty of the segment next to reproduce and we decide in such a case, step 27, to extend the silence beyond its normal duration, i.e. we insert a silence additional.

On the contrary, step 28, if a contiguous sound sequence, like 12 - 13, is present and therefore available (packets P1 and P2 received) and represents a duration exceeding a threshold B, the silence in progress in state 26 (S1) will be abbreviated. In this example, we do the same if a block (12) of sequence sound has a delay in restitution exceeding threshold A, even in the absence of reception of the end (13) of the block. If, on the other hand, threshold B is not reached, an additional silence is inserted.

We leave state 26 when all the silence that was to be reproduced has been and that the following sound sequence should be reproduced. We delay it however by passing temporarily or durably through a state 29, emission of background noise, or background noise, sounding or tone like "uhh", indicating that the speaker will speak again, which avoids being cut off by replacing an extended silence, or additional, by background noise. The two conditions of step 28 (thresholds B and A) are also sought in state 29 and we pass, in yes, in state 21 of sound reproduction.

We test there (step 20) if the delay in restitution exceeds a threshold C, by example of 1.5 times the value of threshold A, which indicates an overflow memory positive 4. In such cases, the oldest rests, restore first, are reduced and possibly almost deleted. It can also be planned to delete the sound sequences the older or simply time slices thereof, which is equivalent to modulating, here accelerating, the speed of reproduction of the sound signal.

Claims (8)

Method for sound reproduction of speech signals (11, S1, 12, 13, S2) received in successive packets (P1, P2, P3) representing slices successive speech times and temporarily memorized (4) before to be restored by sound (5, 6), process characterized by the fact that the presence of rests (S1, S2) is detected in the slices received, and that we regulate the duration of restitution to restore, in one piece, the speech signals (11, 12) other than rests. Method according to claim 1, in which the (24, 28) is shortened restitution of all silence (S1) including at least the beginning of the sequence next sound is available. Method according to claim 1, in which the (23, 28) is shortened restitution of silence (S1) if the delay in the restitution of the speech signals exceeds a threshold (A). Method according to one of claims 1 to 3, in which a additional silence (25, 27) when no next sequence (11) is not available in memory. Method according to one of claims 1 to 4, in which, when silence (S1) is not followed in memory by a sound sequence of duration above a threshold (B), an additional silence is inserted. Method according to one of claims 4 and 5, in which it is replaced additional silence by background noise (29). The method of claim 6, wherein the background noise is voice tone. Method according to one of claims 1 to 7, in which it is eliminated (20), of the restitution, the oldest memorized signals when their delay for return exceeds a threshold (C).

Images