WO2003088669A1 - Data transfer using television video signal - Google Patents

Data transfer using television video signal Download PDF

Info

Publication number
WO2003088669A1
WO2003088669A1 PCT/US2002/011181 US0211181W WO03088669A1 WO 2003088669 A1 WO2003088669 A1 WO 2003088669A1 US 0211181 W US0211181 W US 0211181W WO 03088669 A1 WO03088669 A1 WO 03088669A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
data
data transfer
transfer method
modulated
Prior art date
Application number
PCT/US2002/011181
Other languages
French (fr)
Inventor
Jeng-Jye Shau
Original Assignee
Jeng-Jye Shau
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jeng-Jye Shau filed Critical Jeng-Jye Shau
Priority to PCT/US2002/011181 priority Critical patent/WO2003088669A1/en
Publication of WO2003088669A1 publication Critical patent/WO2003088669A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8166Monomedia components thereof involving executable data, e.g. software
    • H04N21/8186Monomedia components thereof involving executable data, e.g. software specially adapted to be executed by a peripheral of the client device, e.g. by a reprogrammable remote control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781Games
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782Web browsing, e.g. WebTV
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8126Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/025Systems for the transmission of digital non-picture data, e.g. of text during the active part of a television frame
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/08Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N2007/1739Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal the upstream communication being transmitted via a separate link, e.g. telephone line

Definitions

  • the present invention relates to information transfer methods, and more particularly to transfer methods using television (TV) video signals.
  • TV television
  • the proposal is to utilize existing TN networks to transfer data. Combining all the TN channels, the total bandwidth of TV signal is about one million times higher than a telephone line.
  • the TV network already reaches nearly everyone in the world; it requires no new investment to implement. Since TV system is a one-way broadcast system, we will still need the telephone system to transfer low bandwidth tasks, while using TV to transfer most of data from providers to users. This combination of TV and telephone networks has enough capability to solve most of existing problems.
  • the major challenge for this proposal is that almost all the bandwidth of TV channels has been used to transfer images to TN. Watching TV has been an important part of modern human life; any change in existing TV system will certainly encounter strong resistance. It is therefore strongly desirable to provide methods to transfer high bandwidth data using TV system without influencing TV viewers.
  • TV signal contains timing and color information to control the scanning electron beam hitting on a TV screen.
  • Figs. l(a-g) shows the relationship between TV image and TV signal.
  • Each picture is divided into a plurality of horizontal lines. The picture is created line by line with a scanning electron beam. Each line is composed of a plurality of picture elements (pixel). The size of each pixel is defined by the resolution of the image.
  • a pixel is actually composed of three dots of three primary colors (red, green, and blue). The light density and color of each pixel is determined by the strength and location of the scanning electron beam in TV tube, which is controlled by the TV signal.
  • Nearby lines belong to two separated frames. Half of the lines are scanned on the screen first, while the electron beam goes back to the upper left corner to scan the other half of interleaved lines. Display of 30 pictures or 60 frames in every second creates a motion picture.
  • FIG. 1(b) shows a snapshot of a typical TV signal waveform after it is demodulated.
  • V sync vertical synchronization
  • a vertical blank interval (112)
  • H sync horizontal synchronization
  • the duration of the horizontal sync signal is the time when the scanning beam move back from the right side of the screen to the left side to start on a new line.
  • Video signal for one line of image is transferred between horizontal sync signals.
  • the amplitude of video signal varies between 0.3 to 1 volts.
  • a voltage at 1 volt represents the brightest white color, while a voltage at 0.3 volts (black level) represents totally black.
  • the voltages for both the vertical sync and the horizontal sync are at zero volts, which is called the "blank level”.
  • the 0.3 volts difference between black level and blank level is designed to avoid false image during vertical and horizontal blank intervals.
  • the amplitude of the video signal (115) represents the light intensity along one horizontal line. It also includes frequency modulated (FM) audio signals.
  • the video signal for color TV is more complex. Besides the FM audio signal, the color video signal contains three sets of signals as
  • V 0.877(R-Y)q (3)
  • R the red light intensity
  • G the green light intensity
  • B the blue light intensity
  • Y called the "luminance signal” that is equivalent to light density adjusted by color sensitivity of human eyes
  • U is the blue color differential signal
  • p is a phase factor representing a phase shift and a 4.43 MHZ carrier frequency shift
  • V is the red color difference signals
  • q is equal to p plus 90 degree phase shift.
  • the audio signal is carrier by a narrow side band 6 MHZ (million cycles per second) above basic carrier frequency (CF).
  • the luminance signal (Y) occupies spectrum between CF and audio side band.
  • the color difference signals (U, V) occupies a side band centered at 4.43 MHZ above CF.
  • the spectrum peaks of the color difference signals is carefully inserted between that of the luminance signals to minimize interference. This is possible only because the amplitudes of color difference signals (U, V) often follow that of luminance signal (Y).
  • the color TV also uses another timing signal called "color burst" (117).
  • the color burst (117) is placed at the back porch between the horizontal sync pulse and the start of video signal as shown in FIG. 1(b).
  • the color TV signals are defined in this way in order to be compatible with black and white TV.
  • other types of information have been transferred through the TN signals taking advantage that part of those signals are not displaced on IN screen.
  • special binary signals are inserted into the "spare" time during the vertical blank interval to carry text.
  • the video image near the edge of the TN screen is usually not displayed. It is therefore possible to transfer data through those "unused" lines.
  • TN signal line 7-18 and 320-331 are used to carry text signals that are only recognizable with special decoding circuits.
  • TN decoder circuits replace Lines 22-24 and their companions lines 334-336 by special signals used for automatic gray scale compensation.
  • All these video, audio, timing, and special signals are all transferred by modulating high frequency carrier signals within a pre-defined standard bandwidth ( ⁇ 8 MHZ). Signals from hundreds of TN channels are transferred in parallel using carefully defined carrier frequency; each channel occupies a well- defined bandwidth to avoid interference.
  • the primary objective of this invention is, therefore, to provide practical methods to transfer data using TV signals without disturbing TV viewers.
  • the other objective is to provide effective methods to find available bandwidth for data transfer methods of the present invention.
  • Another objective is to provide methods to compensate the distortion caused by such data transfer.
  • Another important objective is to provide methods to improve tolerance in noise. It is also a major objective of the present invention to provide efficient methods to work with other data transfer methods.
  • Figs, l(a-g) show relationships between current art TV signal and TV display
  • FIG. 2(a) is an example of a TV video signal with low contrast areas
  • FIG. 2(b) is an example of the spectrum for a low contrast area
  • FIG. 2(c) illustrates the spectrum after frequency modulated data signal has been inserted into the low contrast area in FIG. 2(b);
  • FIG. 2(d) illustrates the spectrum after multiple frequency data signal has been inserted into the low contrast area in FIG. 2(b);
  • FIG. 2(e) shows an example when amplitude modulated data signal is inserted into a low contrast area in FIG. 2(a);
  • FIG. 2(f) shows an example when compensated amplitude modulated data signal is inserted into a line next to the line in FIG. 2(e);
  • FIG. 2(g) shows an example when compensated amplitude modulated data signal is inserted into a low contrast area in FIG. 2(a);
  • FIG. 2(h) is an example of differential amplitude data signal
  • FIG. 3(a) is the block diagram for a low contrast area data transfer system of the present invention.
  • Figs. 3(b-d) are examples for the video signals in FIG. 3(a);
  • Figs. 4(a-c) are examples for the video signals of black level data transfer method of the present invention.
  • Figs. 5(a-c) are examples for the video signals of white level data transfer method of the present invention.
  • Figs. 6(a-c) are examples for the video signals of blank level data transfer method of the present invention.
  • Figs. 7(a-c) are the float charts for color table data transfer, pre-defined object data transfer, and invisible frame data transfer procedures;
  • FIG. 8(a) is a high level block diagram for a communication system of the present invention.
  • FIG. 8(b) is a block diagram for the layer structures of a communication system of the present invention.
  • FIG. 9(a) shows the block diagram for a video game controller of the present invention
  • FIG. 9(b) shows the float chart of the communication procedure for the controller in FIG. 9(a); and FIG. 10 illustrate a real-time stock price update system of the present invention
  • FIG. 1(d) shows the video signal waveform for one line (121) of the TV image displayed in FIG. 1(a).
  • an H sync (125) indicated the end of one line, and the electron beam is move back to the left side of the screen during the horizontal blank interval.
  • the value of the TV signal during this interval is at blank level (0 volt).
  • Each line begins when the signal returns from blank level to black level (0.3 volt).
  • a color burst (127) composed of several pulses between blank and black levels is used to indicate the timing relationships between color (U, V) signals and luminance signal (Y).
  • a video signal at 0.3 volts means total black; there should be no light emitted from corresponding pixel.
  • a video signal at 1 volt means white color at the highest luminance level of TV screen.
  • Video signals between 0.3 and 1 volt represent various kinds of color and luminance. Areas where the video signal changes slowly along horizontal direction are called “horizontal low contrast” area. Areas where the video signal changes dramatically along horizontal direction are called “horizontal high contrast” areas (129) as marked by dashed lines in FIG. 1(d).
  • the V sync signal (111) contains a plurality of pulses varying between blank and black levels. Sometimes, these pulses are also used to carry special information such as binary code for text display.
  • FIG. 1(e) shows the video signal for a line (122) that is right next to the line in FIG. 1(d). This line belongs to the same picture at a different frame.
  • the video signals for nearby lines are very similar; the signal only changes at positions where vertical image is changing. Areas where the video signal changes slowly along vertical direction are called “vertical low contrast” area. Areas where the video signal changes dramatically along vertical direction are called “vertical high contrast” areas (139) as marked by dashed lines in FIG. 1(d). Note that horizontal high contrast area is not necessary a vertical high contrast area. Similarly, a horizontal low contrast area can be a vertical high contrast area.
  • FIG. 1(f) shows the video signal for the same line (121) as the line in FIG. 1(d) but for the next picture.
  • the video signal of nearby pictures does not change much; the signal only changes at positions where image is changing with time.
  • the video signal only changes around the bat. Areas where the video signal changes slowly with time are called “low moving contrast” area. Areas where the video signal changes dramatically with time are called “high moving contrast” area (149) as marked by dashed lines in FIG. 1(f).
  • FIG. 1(g) shows the video signal of one line (123) that displays the text box (107) on FIG. 1(a).
  • text characters are displayed by using two monochromatic colors: the foreground color and the background color.
  • the amplitude of the video signal in FIG. 1(f) stays constant until it hits the edges of character frames.
  • the only high contrast area is therefore at the edges of character frames.
  • characters are recognized correctly as soon as the edges of character frames are marked with high contrast area. Changes in the foreground and background areas will not change the information passing to us.
  • the present invention provides methods to replace TV video signal with data signal while the resulting broadcast signal can be used for both TV display and data transfer.
  • One data transfer method of the present invention is to use this available bandwidth in the low contrast areas to transfer data. This method is called "Low Contrast area Data Transfer" (LCDT).
  • FIG. 2(a) shows an example of a video signal with three low contrast areas (201) marked with dashed lines.
  • the whole spectrum reserved for the channel would be crowded by the video signal, as shown in the example in FIG. 1(c).
  • FIG. 2(b) shows an example of the spectrum for the video signal within a low contrast area. For color signals, there are two narrow spectrum peaks, one peak for luminance signal (211) and one peak for color difference signal (212); most bandwidth reserved for the TV channel is not used.
  • a data transfer method of this invention preserves original video information. That means the data signal must (1) stay within the bandwidth of the TN channel, (2) minimize the changes in video information. Data transfer that satisfies those requirements is described hereafter.
  • the color of a pixel is determined by the light intensity of three primary colors red, green and blue, i.e., R, G, B.
  • This set of three light intensity is translated into a set of amplitude modulated (AM) video signal (Y, U, N). It is possible to represent the same video signal (Y, U, N) using different carry frequency side bands, as soon as the amplitude the resulting signal detected by TN receiver is correct.
  • frequency modulated (FM) signal can be embedded in the video signal without changing the meaning of video signal. This method is usually not useful because the result of frequency modulation will expand the bandwidth occupied by the video signal, causing distortion. In a low contrast area, we have available bandwidth to carry FM data signal without causing problem.
  • FIG. 2(c) shows the spectrum when the video signal in FIG. 2(b) is carrying FM data.
  • the effect of FM data modulation widens the spectrum peaks (221, 222).
  • the effect of the data signal will not cause any change in video information as soon as the spectrum peaks are not wide enough to interfere with one another.
  • the luminance peak (221) and the color peak (222) can carry different sets of FM data separately.
  • This frequency modulation to carry data signal does not influence the FM audio signal because the audio signal is carried by CF + 6 MHZ side band.
  • a variation of FM format is the Multiple Frequency (MF) format.
  • FIG. 2(d) shows the spectrum for one example of MF data. There are 4 luminance peaks centered at four frequencies (FO, FI, F2, F3).
  • Data are represented by carriers of different frequencies within the bandwidth of the TV channel.
  • carrier frequency FO represents binary data '00'
  • FI represents '01'
  • F2 represents '10'
  • F3 represents '11'.
  • the amplitude of MF signal still can follow the original video signal.
  • the color signal also can carry a different set of MF format data. For example, color signal centered at F4 carries binary data '0'; color signal centered at F5 carries binary data 'V.
  • the data may be transferred by modulating the phase of the data carrier. This method is called Phase Modulated (PM) data transfer. Data are represented by the phase change in the carrier signal. A variation of PM data is the multiple phase (MP) data format. Data are represented by video signals with discrete phases.
  • PM Phase Modulated
  • MP multiple phase
  • Low contrast-area data transfer in frequency modulated formats such as transferred in FM, MF, PF, or MP format has the advantage that the amplitude of the original video signal remains the same. It is therefor possible to transfer data in the low contrast areas without any change in the video information. On the other hand, human eyes are not sensitive to small changes in a low contrast area. It is therefore possible to insert amplitude modulated (AM) signal in the low contrast area.
  • FIG. 2(e) shows an example when an AM data signal (253) is inserted into one of the low contrast areas (251) of the video signal in FIG. 2(a).
  • One problem for inserting data into video signal is that it may create a random background color fluctuation on the TV image.
  • NC compensated formats
  • FIG. 2(f) shows a line of video signal that is carrying VC or TC data signal (254) compensating for the AM data (253) in FIG. 2(e).
  • the third CF method is to carry the compensated data in nearby pixels on the same line.
  • This format is called horizontal compensation (HC).
  • FIG. 2(g) shows an example HC data (255).
  • the compensated format data (253-255) shown in Figs. 2(e-g) are simplified for clarity. There are actually three degrees of freedom to represent and to compensate the modulated data. Data compensation can be implemented on one or more of the color components. The amplitude of the video signal is also not necessary linear proportional to the amplitude of light. Therefore, a proper compensation signal also needs to take alpha correction into consideration.
  • FIG. 2(h) shows an example of data signal in differential amplitude (DA) format.
  • Binary data ⁇ is represented by a change in amplitude in either up or down direction, and binary data '0' is represented by no change. If a parity bit is carried in every
  • the differential format can be employed to carry data signals with an FM, MF, PM, or MP formats.
  • combinations of multiple data formats e.g., combinations of FM, MF, PM, MP, AM, CA, DA, can be employed to achieve higher data transfer rate when necessary.
  • FIG. 3(a) shows the block diagram for a hardware system executing a low contrast area data transfer (LCDT) process.
  • a video signal analyzer (VSA) examines the outgoing video signal (OVS) looking for available bandwidth. This video signal analyzer detects changing rate of the out going video signal (OVS). Whenever low contrast areas are detected, the VSA sends signals to a signal , processor. In the mean time, the data providers also send data to the signal processor for data transfer.
  • This signal processor inserts outgoing data signal (ODS) into the outgoing video signal (OVS) to create broadcast video signal (BVS).
  • ODS outgoing data signal
  • VVS broadcast video signal
  • Both TN users and data users receive and process the broadcast video signal. Data transfer methods of the present invention preserve the quality of video signal.
  • the TN receivers process the BNS the same way as before, not affected by the data signal carried along with the broadcast video signal, to display high quality pictures with the processed BNS.
  • the data receiver has a data signal analyzer (DSA) and data decoder.
  • DSA examines BNS and sends out a signal to the data decoder whenever data signal is found.
  • the data decoder filters the right signal out of BNS, and extracts the correct data for the data user.
  • the output generated by a low contrast area data transfer (LCDT) NSA would look like FIG. 3(c).
  • the output of the level sensor is binary '1', otherwise it is '0', as shown in FIG. 3(c).
  • the areas allowed for timing control signals e.g., vertical sync, horizontal sync, and color burst, automatically meets the requirements for a low contrast area. Therefore, the signal in FIG. 3(c) also goes high during the horizontal blank interval.
  • data signal is inserted to create the BNS as shown in FIG. 3(d).
  • the data can be transferred using any one or any combination of the formats (FM, PM, MF, MP, DA, CA, CP) of the present invention.
  • the LCDT data signal analyzer also has the capability to detect low contrast area. Whenever a low contrast area is found, the data decoder will look for overlapped data signal, and extract the data by demodulation procedures.
  • the optimum data transfer rate of LCDT is strongly dependent on the video signal. Higher data rate can be achieved for special types of TN signals. For example, special data transfer rate can be achieved when the low contrast area is at black level.
  • a Black Level Data Transfer (BLDT) method is disclosed in the present invention to take advantage of the special data transfer rate achievable in the black-level low contrast area.
  • this invention also applies a White Level Data Transfer (WLDT) method by taking advantage of the higher data transfer rate in a white-level low contrast area.
  • WLDT White Level Data Transfer
  • this invention also discloses a Blank Level Data Transfer (KLDT) method carry the data signals in a blank-level low contrast area.
  • FIG. 3(d) shows example for the situations when BLDT (331), WLDT (332), common LCDT (333), KLDT (334) are used for the example video signal in FIG. 3(b).
  • BLDT BLDT
  • WLDT WLDT
  • KLDT KLDT
  • Video signals with a voltage level representing the black color are one of the signals most frequently transmitted to the TV receivers. Black as a color is frequently displayed on TV screen. Also, the signal's voltage level is applied as the upper level for video timing signals (H sync, V sync, color burst). In optical terms, black means no light. When the video signal is at black level, the corresponding picture element (pixel) should be totally black on the TN screen. According to optical concept, it is impossible to have a color darker than black.
  • Black level is represented by an amplitude-modulated (AM) signal at 0.3 volts. The black level is set at 30% of full-scale amplitude because there is a need to have enough margin to define the "blank level" used for timing signals. Ideally, a video signal should never have a value between black level and blank level because nothing can be darker than black while it is not a timing signal.
  • AM amplitude-modulated
  • a video signal lower than black level will be processed by the receiver as black, immediately after the receiver circuits detect a video signal level lower than the black level.
  • a spot on the TN screen can not be totally black; the TN screen may reflect lights from nearby light source even when the screen itself is not emitting light. Therefore, a video signal slightly higher than black level can be treated as black in practical conditions.
  • the concept of "black” is therefore not strictly defined as represented by one- and-only signal level.
  • the TN signals with amplitudes between lower black level and upper black level represent the same signal as far as TN display is concerned. It is therefore very convenient to carry along data signals with a signal representing black spots for TN display. The only limitation is to have the overlapped signals remain within the black level range.
  • the VSA and DSA need to have a level sensor.
  • This level sensor examines the video signal, and sends out a control signal whenever the video signal is within black level range.
  • the BLDT signal analyzer output would look like FIG.4(b).
  • the output of the level sensor is binary 'V , otherwise it is '0', as shown in FIG. 4(b).
  • data signal is inserted into the original video signal to create the video output signal as shown in FIG. 4(c).
  • One obvious problem for the AM data signal in this example is noise sensitivity.
  • the black level range is only a fraction of full-scale amplitude.
  • the signal to noise ratio is therefore much smaller than TV signals.
  • One solution to solve the noise problem is to use frequency modulation methods. Since black level does not have color information, we can change phase and frequency without influencing the quality of video display. Using FM methods we can keep the amplitude of the black level signal at 0.3 volts, while carrying high bandwidth data signal with FM, MF, PM, or MP methods described in previous sections. Because the amplitude of data signal is 0.3 volts, excellent signal to noise ratio can be achieved.
  • a narrow side band within the TV channel to carry the AM signal can be employed to improve noise tolerance for amplitude modulated black level data transfer (AMBLDT).
  • a filter can also used to filter out most of noise at other frequencies and a compensated or differential format is also used for AMBLDT.
  • two or more of the data transfer formats of the present invention such as FM, PM, MF, DA, CA. CP described above can be combined to achieve higher data transfer rate.
  • white for TV display is not the same as natural white color.
  • "white” is a signal with white level amplitude, that is, at 100% of full-scale amplitude. In reality, there is also an upper white level and lower white level. TV signals with amplitudes between upper and lower white level (white level range) will display “white” on TV screen. TV viewers can not distinguish any difference as soon as the signal is within white level range. Because "white level” is not strictly defined, we can use it to transfer more data in similar ways as BLDT. However, there is a major difference between WLDT and BLDT. For WLDT, if we change Y without changing color difference signals U and V, there will be differences in color. The color difference signals (U, V) occupies a side band centered at 4.43 MHZ above CF. The need to have balanced color reduced the degree of freedom in choosing frequency side bands for WLDT. On the other hand, WLDT signals are at full- scale amplitude, so that its noise tolerance is better than BLDT.
  • the VSA and DSA need to have a level sensor.
  • This level sensor examines the video signal, and sends out a control signal whenever the video signal is within white level range.
  • the WLDT signal analyzer output would look like FIG.5(b).
  • the output of the level sensor is binary ⁇ , otherwise it is '0', as shown in FIG. 5(b).
  • data signal is inserted into the original video signal to create the video output signal as shown in FIG. 5(c).
  • the data can be transferred using any one or any combination of the formats (FM, PM, MF, DA, CA. CP) described in previous sections.
  • Unlike BLDT for WLDT selection of the phase and frequency of the carrier signal need to take color into consideration.
  • Blank level is used for timing signals such as horizontal sync, vertical sync, and color burst. Blank signal should have zero amplitude. In reality, timing circuits are most sensitive to the falling and rising edges of the timing signals. Other than those edges, timing circuits can tolerate signals with amplitude smaller than the blank level limit as blank signal. Therefore, we can insert data signals to replace blank signals as soon as the amplitude of the inserted signal is lower than the blank level limit.
  • the VSA and DSA need to have a level sensor. This level sensor examines the video signal, and sends out a control signal whenever the video signal is below blank level limit. Using the video signal in FIG. 6(a) as an example, the KLDT signal analyzer output would look like FIG. 6(b).
  • the output of the level sensor is binary 'V, otherwise it is '0', as shown in FIG. 4(b).
  • data signal is inserted into the original video signal to create the video output signal as shown in FIG. 6(c).
  • the data can be transferred using any one or any combination of the formats (FM, PM, MF, DA, CA. CP) of the present invention. Similar to BLDT, we have total freedom to select the phase and frequency of the carrier signal because blank level does not have color.
  • the data transfer methods using the available bandwidth in the low contrast areas have been disclosed in the above sections. Those methods provide data transfer bandwidth whenever the video signal is in low contrast areas.
  • the average data transfer rate is therefore dependent on the property of TV image.
  • Color table is commonly used for computer display as a method to reduce the size of graphic files.
  • a color table defines a finite number (16, 64, or 256) of colors. The color of each pixel in a picture is represented by one of the color in the color table that is closest to the original. For most cases, a 256-color table is adequate to display high quality pictures, especially when the content of the color table can be changed to adapt for different pictures. Definition of the colors in the color table is not unique. We can replace every entry of a color table with similar but different colors to create another table. The new table will still be able to represent high quality pictures. This property is used by a data transfer method of the present invention called color table data transfer (CTDT).
  • CTT color table data transfer
  • FIG. 7(a) shows the flow chart for CCDT procedures.
  • a video signal analyzer determines the color for each pixel in the original video signal (ONS). For transmitting a binary number '0', the color of the pixel is replace by the best fit in table TO, and for transmitting a binary number ⁇ , the color of the pixel is replace by the best fit in table TI.
  • the resulting picture (BNS) is broadcast through TN network.
  • a data receiver uses a data signal analyzer (DSA) to examine the BVS. When the color of a pixel is found in TO, a binary number '0' is received. When the color of a pixel is found in TI, a binary number ⁇ is received.
  • DSA data signal analyzer
  • CTDT has the advantage that a data binary bit may be into every pixel of the video signal. It is not necessary to consider possible interference to the quality and color variations of the video signals due to the insertion of data. It is therefore not necessary to first determine the low contrast areas.
  • CCDT data transmission can be flexibly applied to different portions of the display signals and does not have to be applied to the whole screen. It is usually advantageous to transfer data on part of the screen where a small color table may be used (e.g. 16 or 64 colors) to simplify supporting circuits.
  • a pre-defined object is an object that is known to both the data sender and the data receiver.
  • pre-defined objects are logo (102), score board (104), and caption frame (107), as shown in FIG. 1(a).
  • Pre-defined objects usually are simple graphic figures with simple color pattern so that multiple methods of this invention (LCDT, BLDT, WLDT, CTDT) can be used to achieve high data transfer rate. Because we know exactly the color of each pixel, further data transfer methods are available. For example, an area in the PO can be designated to fill with any video signal. In this way, there will be no constraint on the data signal format within that area. Maximum bandwidth can be achieved within such area.
  • Both the data sender and the data receive should have a PO library.
  • FIG. 7(b) shows the flowchart as one example for showing the processing steps carried out to insert data into the pixels of the pre-defined object data for transferring data using the PODT procedure.
  • SODT Small Object data transfer
  • IFDT Invisible frame data transfer
  • the video signal analyzer (VSA) and the data signal analyzer (DSA) need to have a timing circuit.
  • This timing circuit uses the video timing signals (V sync and H sync) and an internal timer to determine if the signal is within the invisible frame.
  • V sync and H sync video timing signals
  • an internal timer to determine if the signal is within the invisible frame.
  • data signal is inserted into the original video signal to create the video output signal.
  • the data can be transferred using any one or any combination of the formats described in previous sections.
  • FIG. 7(c) shows the float chart for one example of IFDT procedure.
  • DODT Dedicated object data transfer
  • the present invention provides effective methods to utilize the TV network as a parallel path for Internet communication.
  • Combining the data transfer methods (BLDT, WLDT, KLDT, VSDT, CTDT, PODT, SODT, IFDT, DODT) of the present invention more than 90% of the TV bandwidth will be available for data transfer.
  • the bandwidth for each pixel on a TV screen is about equal to 6 phone lines. If all the available TV channels are fully utilized, more than one billion bits per second (Gbps) is transmissible to every user in the world.
  • Gbps bits per second
  • FIG. 8(a) shows a broad view of a communication system of the present invention.
  • Information users (801) and information providers (803) are connected through Internet.
  • the backbone of Internet is built on telephone systems and computer networks.
  • TV stations (805) broadcast video signals to TV viewers.
  • Information users (801) send requests, commands, selections, and simple e-mails through the Internet. Usually those activities require little bandwidth; a simple telephone line is more than enough to handle them.
  • the information providers (803) also use Internet for data transfer. The bottle neck is the bandwidth requirement for a popular Web site to send large data files to users.
  • the present invention provides an alternative data path.
  • Information provider can send data to a TV station with a data encoder of the present invention.
  • This data encoder contains a video signal analyzer (VSA) and a video signal processor (VSP).
  • VSA video signal analyzer
  • VSP video signal processor
  • the video signal analyzer (VSA) examines the outgoing video signal (OVS) looking for available bandwidth.
  • ODS outgoing video signal
  • the output of VSA is sent to VSP to select proper data transfer methods.
  • This signal processor (VSP) inserts outgoing data signal (ODS) into the outgoing video signal (OVS) to create broadcast video signal (BVS).
  • the BVS is broadcast through TV systems, and it is processed by both TV receivers and data users (801).
  • Data transfer methods of the present invention preserve the quality of video signal, so that the TN viewer still can receive high quality video display using BVS.
  • the data receiver has a data signal analyzer (DSA) and data decoder (807).
  • DSA examines BVS and sends out a signal to the data decoder (807) whenever data signal is found.
  • Control information such as the type of data transfer and the encoding keys are provide by the information providers (803) either through Internet or transferred as part of ODS.
  • the data decoder (807) filters the right signal out of BVS, and extract the correct data for the data user.
  • This alternative data path through TN network has much wider bandwidth then the telephone networks. It is therefore possible to improve overall system performance dramatically.
  • FIG. 8(b) shows the layer structure of the system in FIG. 8(a). Most of the layer structures and communication protocols are the same as those of current art communication systems. There are no changes on application, presentation, session, and transport layers. Therefore, a communication system of the present invention can use most of existing software and hardware.
  • the present invention provides one way parallel paths in the lower level layers.
  • FIG. 9(a) shows the block diagram for a video game controller of the present invention.
  • This video game controller is equipped with a TV signal interface (901) for receiving television signals.
  • Typical examples for this TN signal interface (901) are connections to TV antenna or cable TN box.
  • TN signal received by the TN interface (901) is sent to a data decoder (903).
  • the data decoder (903) is used to extract data from TN signal.
  • This video game controller is also connected to an Internet interface (905).
  • a typical example of an Internet interface is a computer equipped with modem.
  • This Internet interface also can be placed inside of the video game control box. Both the Internet interface (905) and the data decoder (903) are connected to a storage unit (907) and a video game control unit (909).
  • the storage unit (907) is a memory device used to store data. Typical examples of the storage unit are hard disk or tape.
  • the video game controller (909) is the same as current art video game controllers except that it has programmable firmware to allow re-configuration for different games. Video game players can play different games by programming the controller firmware.
  • a player need to scan the web side of a video game provider, then load the whole set of a video game program into the game controller (909) in order to play a new game. If there are 1 million players wanting the same game, the same procedures will be repeated one million times. Most likely the web site will be jammed by requests for popular games. Even if the web side has enough bandwidth to handle the request, it is still a tremendous waste in resource.
  • the video game player uses the Internet interface (905) to select a new game.
  • the game provider sends a "decoder key” to the player.
  • This "decoder key” tells the data decoder (903) when and how to down load data from the TV interface (901).
  • High volume data such as the video images of web pages and the game programs are transferred through the TN interface using data transfer methods of the present invention.
  • the Internet interface (905) only handle slow activities such as selection of game or transfer of the decoder key.
  • the same decoder key can be given to multiple users, so that when many users are requesting for the same data simultaneously, the provider only need to send one copy through the TN interface.
  • RT data transfer A data transfer that is initiated immediately after a request from the user is called a real time (RT) data transfer.
  • RT data transfer One problem for RT data transfer is that players usually send out their requests at different time. If the provider always send out the data from the very beginning whenever a request is received, the same data will need to be sent many times.
  • One way to solve the problem is to delay the data transfer, accumulate many requests, then send one copy out to satisfy all the requests. This method is called delayed data transfer.
  • the other way to solve the problem is to break a large file into small packages. The game players do not need to receive a large data file from the beginning. Small packages can be received out of sequence. The final data file is established after all packages are received. This method is called package data transfer.
  • the game provide simply keep on sending out packages of requested games as soon as there are requests for that game. All the players requesting the same game are given the same key. Whenever a player has collected all the necessary packages, a signal is sent back to the provider to notify end of request. The game providers stop the procedure when all the requests are satisfied.
  • Another method to solve the problem is to schedule the TN data transfer ahead of time. This method is especially useful for introduction of a brand new game. All the players wanted the new game are given a key to access the data. Data for the new game is sent out at a pre-defined time to all players. In this way, the provider only needs to send one copy once. Another method is for the provider to send data to players who are likely to want the data before the player actually request for the game.
  • FIG. 9(b) shows a float chart for the above communication procedures.
  • the data transfer methods of the present invention are so efficient that they can support thousands of people playing the same game simultaneously. To support such a large scale game, each video game system should store the game map. Players send commands through conventional Internet connections, while a central system update the results through TN data transfer methods of the present invention. The TN signal updates all the game maps in all the involved individual systems with a single broadcast. In this way, thousands of people can play the same game without jamming the system.
  • FIG. 10 shows an example of stock market data update system of the present invention.
  • This system is identical to current art stock market data update system except for a TV signal interface (991) that can obtain data from TN signals.
  • the system is still connected to internet. Users still send their requests through Internet.
  • the software programs used to display stock market information are the same as current art software programs. The only difference is that there is a parallel data path from the stock information provider to all users through TN data transfer methods of the present invention.
  • the most updated stock prices are sent to millions of users through Internet. That means millions of duplicated copies are sent to individual users.
  • the stock information providers only need to send out one copy of the latest stock data. Updating latest stock price only requires a few thousand bits per second. Any one data transfer method of the present invention will easily handle the bandwidth requirement, while all the users can obtain real-time stock prices simultaneously with minimum delay. Note that the system does not have to use a computer.
  • a video game controller described in FIG. 9(a) can be programmed to have stock update capability.
  • Data transfer system of the present invention uses existing TV broadcast systems to send data. It will satisfy the bandwidth requirement for many applications without any changes to existing system.
  • the system requires little resource to implement. It is the most cost efficient method to solve the bandwidth problem, and the system can be established in a short time.

Abstract

Data transfer system of the present invention examines TV video signal for unused bandwidth. Data signals are inserted into TV video signal when unused bandwidth is found. The resulting video signal is still used for TV display without sacrificing video quality. These data transfer methods provide an alternative high-bandwidth data path to Internet. It will satisfy the bandwidth requirement for many applications without any changes to existing system. The system requires little resource to implement. It is the most cost efficient method to solve the bandwidth problem, and the system can be established in a short time.

Description

DATA TRANSFER USING TELEVISION VIDEO SIGNAL
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to information transfer methods, and more particularly to transfer methods using television (TV) video signals.
2. Description of the Reference Art
It is very clear that Internet is bringing revolutionary changes to human life. The world- wide-web allow easy individual access to information at anywhere in the world. It brings tremendous opportunities in business, revolutionary changes in education, and it will certainly change every aspect of our life style.
As Internet access gets more and more popular, data transfer bandwidth becomes a major problem. Today, telephone lines are still the major media for individual Internet connections. The 2.4 KHZ (thousand cycles per second) bandwidth for an end-user telephone line was adequate for voice transfer, but it is not designed to transfer large amount of data. Computer modem devices have been upgraded to 56 Kbps (thousand bits per second), but it is still far too slow. Many solutions have been proposed to solve the bandwidth problem. Among them, Integrated Services Digital Network (ISDN), fiber to home, and cable to home have been implemented at selected areas. However, those proposed new methods require tremendous amount of resource to implement, and it will take many years before it can reach individual users. Those new methods also represent tremendous wastes due to the burst nature of Internet access. An individual user would like to have a large bandwidth while accessing data, but most of time the individual line is not in use. An optical fiber to home is therefore a waste in bandwidth for most of time. Another important fact is that the bandwidth requirement from the user to the provider is usually very low. A human being can send out just a few commands per second. High bandwidth is often needed after the user request large amount of data from the provider. Providing the same bandwidth for both directions is therefore a waste. Further more, those new methods do not really solve the bandwidth problem for popular data providers. When thousands or millions of users request a popular web page at the same time, the provider does not have enough bandwidth to send out so many copies of data even if it is equipped with optical fiber. The present invention provides a solution that can solve most of the bandwidth problems now. The proposal is to utilize existing TN networks to transfer data. Combining all the TN channels, the total bandwidth of TV signal is about one million times higher than a telephone line. The TV network already reaches nearly everyone in the world; it requires no new investment to implement. Since TV system is a one-way broadcast system, we will still need the telephone system to transfer low bandwidth tasks, while using TV to transfer most of data from providers to users. This combination of TV and telephone networks has enough capability to solve most of existing problems. The major challenge for this proposal is that almost all the bandwidth of TV channels has been used to transfer images to TN. Watching TV has been an important part of modern human life; any change in existing TV system will certainly encounter strong resistance. It is therefore strongly desirable to provide methods to transfer high bandwidth data using TV system without influencing TV viewers.
Existing television (TV) signal transfer methods are first reviewed to facilitate understanding of the issues. TV signal contains timing and color information to control the scanning electron beam hitting on a TV screen. Figs. l(a-g) shows the relationship between TV image and TV signal. Each picture is divided into a plurality of horizontal lines. The picture is created line by line with a scanning electron beam. Each line is composed of a plurality of picture elements (pixel). The size of each pixel is defined by the resolution of the image. For a color TV, a pixel is actually composed of three dots of three primary colors (red, green, and blue). The light density and color of each pixel is determined by the strength and location of the scanning electron beam in TV tube, which is controlled by the TV signal. Nearby lines belong to two separated frames. Half of the lines are scanned on the screen first, while the electron beam goes back to the upper left corner to scan the other half of interleaved lines. Display of 30 pictures or 60 frames in every second creates a motion picture.
FIG. 1(b) shows a snapshot of a typical TV signal waveform after it is demodulated. For every 1/60 seconds, a vertical synchronization (V sync) signal (111) marks the time when the scanning beam need to go back to upper left corner of the screen. The duration of this vertical sync signal, called "vertical blank interval" (112), is the time when the scanning beam needed to move from lower right corner back to upper left corner. There is a 15.75 KHZ horizontal synchronization (H sync) signal (113) that determines the time when the scanning beam should start on a new line. The duration of the horizontal sync signal called "horizontal blank interval" (114) is the time when the scanning beam move back from the right side of the screen to the left side to start on a new line. Video signal for one line of image (115) is transferred between horizontal sync signals. The amplitude of video signal varies between 0.3 to 1 volts. A voltage at 1 volt (white level) represents the brightest white color, while a voltage at 0.3 volts (black level) represents totally black. The voltages for both the vertical sync and the horizontal sync are at zero volts, which is called the "blank level". The 0.3 volts difference between black level and blank level is designed to avoid false image during vertical and horizontal blank intervals.
For a black and white TV, the amplitude of the video signal (115) represents the light intensity along one horizontal line. It also includes frequency modulated (FM) audio signals. The video signal for color TV is more complex. Besides the FM audio signal, the color video signal contains three sets of signals as
Y = 0.3R + 0.59G + 0.11B (1) U = 0.493(B-Y)p (2)
V = 0.877(R-Y)q (3) where R is the red light intensity, G is the green light intensity, B is the blue light intensity, Y is called the "luminance signal" that is equivalent to light density adjusted by color sensitivity of human eyes, U is the blue color differential signal, p is a phase factor representing a phase shift and a 4.43 MHZ carrier frequency shift, V is the red color difference signals, q is equal to p plus 90 degree phase shift. These signals are merged into the same bandwidth originally designed for black and white TN signals. The Y signal defines contrast of the image, so it occupies wider bandwidth, while U and N signals occupying narrower bandwidth. FIG. 1(c) shows the spectrum of one channel of TV signal. The audio signal is carrier by a narrow side band 6 MHZ (million cycles per second) above basic carrier frequency (CF). The luminance signal (Y) occupies spectrum between CF and audio side band. The color difference signals (U, V) occupies a side band centered at 4.43 MHZ above CF. The spectrum peaks of the color difference signals is carefully inserted between that of the luminance signals to minimize interference. This is possible only because the amplitudes of color difference signals (U, V) often follow that of luminance signal (Y). The color TV also uses another timing signal called "color burst" (117). The color burst (117) is placed at the back porch between the horizontal sync pulse and the start of video signal as shown in FIG. 1(b). The color TV signals are defined in this way in order to be compatible with black and white TV. Besides sound and image, other types of information have been transferred through the TN signals taking advantage that part of those signals are not displaced on IN screen. For example, special binary signals are inserted into the "spare" time during the vertical blank interval to carry text. The video image near the edge of the TN screen is usually not displayed. It is therefore possible to transfer data through those "unused" lines. For example, TN signal line 7-18 and 320-331 are used to carry text signals that are only recognizable with special decoding circuits. For another example, TN decoder circuits replace Lines 22-24 and their companions lines 334-336 by special signals used for automatic gray scale compensation.
All these video, audio, timing, and special signals are all transferred by modulating high frequency carrier signals within a pre-defined standard bandwidth (~8 MHZ). Signals from hundreds of TN channels are transferred in parallel using carefully defined carrier frequency; each channel occupies a well- defined bandwidth to avoid interference.
From the above descriptions, it is clear that all available bandwidth of TN system has been fully occupied. People already explored all kinds of methods to insert more information into the limited TN bandwidth. Using conventional methods to insert data into TN signals is therefore likely to cause interference. It is therefore highly desirable to invent novel methods to transfer high bandwidth data using TN signals without influencing the programs displayed for TN viewers.
SUMMARY OF THE INVENTION
The primary objective of this invention is, therefore, to provide practical methods to transfer data using TV signals without disturbing TV viewers. The other objective is to provide effective methods to find available bandwidth for data transfer methods of the present invention. Another objective is to provide methods to compensate the distortion caused by such data transfer. Another important objective is to provide methods to improve tolerance in noise. It is also a major objective of the present invention to provide efficient methods to work with other data transfer methods.
These and other objectives are accomplished by novel methods in overlapping data signals with TV signals without causing sensible disturbs in TV image displays.
While the novel features of the invention are set forth with particularly in the appended claims, the invention, both as to organization and content, will be better understood and appreciated, along with other objects and features thereof, from the following detailed descriptions taken in conjunction with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Figs, l(a-g) show relationships between current art TV signal and TV display;
FIG. 2(a) is an example of a TV video signal with low contrast areas;
FIG. 2(b) is an example of the spectrum for a low contrast area;
FIG. 2(c) illustrates the spectrum after frequency modulated data signal has been inserted into the low contrast area in FIG. 2(b);
FIG. 2(d) illustrates the spectrum after multiple frequency data signal has been inserted into the low contrast area in FIG. 2(b);
FIG. 2(e) shows an example when amplitude modulated data signal is inserted into a low contrast area in FIG. 2(a);
FIG. 2(f) shows an example when compensated amplitude modulated data signal is inserted into a line next to the line in FIG. 2(e);
FIG. 2(g) shows an example when compensated amplitude modulated data signal is inserted into a low contrast area in FIG. 2(a);
FIG. 2(h) is an example of differential amplitude data signal;
FIG. 3(a) is the block diagram for a low contrast area data transfer system of the present invention;
Figs. 3(b-d) are examples for the video signals in FIG. 3(a);
Figs. 4(a-c) are examples for the video signals of black level data transfer method of the present invention;
Figs. 5(a-c) are examples for the video signals of white level data transfer method of the present invention;
Figs. 6(a-c) are examples for the video signals of blank level data transfer method of the present invention;
Figs. 7(a-c) are the float charts for color table data transfer, pre-defined object data transfer, and invisible frame data transfer procedures;
FIG. 8(a) is a high level block diagram for a communication system of the present invention;
FIG. 8(b) is a block diagram for the layer structures of a communication system of the present invention; and
FIG. 9(a) shows the block diagram for a video game controller of the present invention;
FIG. 9(b) shows the float chart of the communication procedure for the controller in FIG. 9(a); and FIG. 10 illustrate a real-time stock price update system of the present invention
DETAILED DESCRIPTION OF THE INVENTION
The relationship between TV image and TV signal is described in further details to facilitate understanding of the present invention. FIG. 1(d) shows the video signal waveform for one line (121) of the TV image displayed in FIG. 1(a). At the end of each line, an H sync (125) indicated the end of one line, and the electron beam is move back to the left side of the screen during the horizontal blank interval. The value of the TV signal during this interval is at blank level (0 volt). Each line begins when the signal returns from blank level to black level (0.3 volt). For a color TV signal, a color burst (127) composed of several pulses between blank and black levels is used to indicate the timing relationships between color (U, V) signals and luminance signal (Y). The signal stays at black level, then starts to vary from 0.3 to 1 volts to transfer the video signal on the line. A video signal at 0.3 volts means total black; there should be no light emitted from corresponding pixel. A video signal at 1 volt means white color at the highest luminance level of TV screen. Video signals between 0.3 and 1 volt represent various kinds of color and luminance. Areas where the video signal changes slowly along horizontal direction are called "horizontal low contrast" area. Areas where the video signal changes dramatically along horizontal direction are called "horizontal high contrast" areas (129) as marked by dashed lines in FIG. 1(d).
After the electron beam scans to the lower right corner, the beam is moved back to the upper left corner during the V sync interval. This motion is synchronized by the V sync signal (111) as shown in FIG. 1(b). The V sync signal (111) contains a plurality of pulses varying between blank and black levels. Sometimes, these pulses are also used to carry special information such as binary code for text display.
After the V sync interval, the electron beam starts to scan lines on a new frame, which are interleaved lines on the same picture of previous frame. FIG. 1(e) shows the video signal for a line (122) that is right next to the line in FIG. 1(d). This line belongs to the same picture at a different frame. Typically, the video signals for nearby lines are very similar; the signal only changes at positions where vertical image is changing. Areas where the video signal changes slowly along vertical direction are called "vertical low contrast" area. Areas where the video signal changes dramatically along vertical direction are called "vertical high contrast" areas (139) as marked by dashed lines in FIG. 1(d). Note that horizontal high contrast area is not necessary a vertical high contrast area. Similarly, a horizontal low contrast area can be a vertical high contrast area.
After the second frame is scanned, the electron beam starts to scan the image of the next picture after the V sync interval. FIG. 1(f) shows the video signal for the same line (121) as the line in FIG. 1(d) but for the next picture. Typically, the video signal of nearby pictures does not change much; the signal only changes at positions where image is changing with time. For the example in FIG. 1(a), only the bat of the baseball player is moving. Therefore, the video signal only changes around the bat. Areas where the video signal changes slowly with time are called "low moving contrast" area. Areas where the video signal changes dramatically with time are called "high moving contrast" area (149) as marked by dashed lines in FIG. 1(f).
Human eyes are sensitive to high contrast (horizontal, vertical, and moving) areas. Details in the low contrast areas are usually ignored. One example to illustrate the importance of the high contrast areas is text. FIG. 1(g) shows the video signal of one line (123) that displays the text box (107) on FIG. 1(a). In this example, text characters are displayed by using two monochromatic colors: the foreground color and the background color. Within the text box (107), the amplitude of the video signal in FIG. 1(f) stays constant until it hits the edges of character frames. The only high contrast area is therefore at the edges of character frames. For texts, characters are recognized correctly as soon as the edges of character frames are marked with high contrast area. Changes in the foreground and background areas will not change the information passing to us. Another important fact is the describing a two-color picture only needs two kind of signals; that means we do not need a lot of bandwidth to display simple objects like text. Text is just an example easy to understand. It is generally true that small changes in the low contrast area will not influence the information received by viewers. From signal processing point of view, the bandwidth reserved for each TV channel is defined to be enough to describe high contrast objects, while the low contrast areas do not require the same bandwidth. That means there are available bandwidth to transfer other information whenever the image has low contrast areas. The present invention provides methods to replace TV video signal with data signal while the resulting broadcast signal can be used for both TV display and data transfer. One data transfer method of the present invention is to use this available bandwidth in the low contrast areas to transfer data. This method is called "Low Contrast area Data Transfer" (LCDT).
Low Contrast area Data Transfer (LCDT)
A low contrast area is an area where the video signal does not change rapidly. FIG. 2(a) shows an example of a video signal with three low contrast areas (201) marked with dashed lines. When a video signal has high contrast areas, the whole spectrum reserved for the channel would be crowded by the video signal, as shown in the example in FIG. 1(c). During the period of time when the video signal is in the low contrast areas (201), a small bandwidth is enough to carry the video information because the colors of all the pixels in the area are about the same. FIG. 2(b) shows an example of the spectrum for the video signal within a low contrast area. For color signals, there are two narrow spectrum peaks, one peak for luminance signal (211) and one peak for color difference signal (212); most bandwidth reserved for the TV channel is not used.
A data transfer method of this invention preserves original video information. That means the data signal must (1) stay within the bandwidth of the TN channel, (2) minimize the changes in video information. Data transfer that satisfies those requirements is described hereafter.
The color of a pixel is determined by the light intensity of three primary colors red, green and blue, i.e., R, G, B. This set of three light intensity is translated into a set of amplitude modulated (AM) video signal (Y, U, N). It is possible to represent the same video signal (Y, U, N) using different carry frequency side bands, as soon as the amplitude the resulting signal detected by TN receiver is correct. On the other word, frequency modulated (FM) signal can be embedded in the video signal without changing the meaning of video signal. This method is usually not useful because the result of frequency modulation will expand the bandwidth occupied by the video signal, causing distortion. In a low contrast area, we have available bandwidth to carry FM data signal without causing problem. FIG. 2(c) shows the spectrum when the video signal in FIG. 2(b) is carrying FM data. The effect of FM data modulation widens the spectrum peaks (221, 222). The effect of the data signal will not cause any change in video information as soon as the spectrum peaks are not wide enough to interfere with one another. Note that the luminance peak (221) and the color peak (222) can carry different sets of FM data separately. This frequency modulation to carry data signal does not influence the FM audio signal because the audio signal is carried by CF + 6 MHZ side band. A variation of FM format is the Multiple Frequency (MF) format. FIG. 2(d) shows the spectrum for one example of MF data. There are 4 luminance peaks centered at four frequencies (FO, FI, F2, F3). Data are represented by carriers of different frequencies within the bandwidth of the TV channel. For example, carrier frequency FO represents binary data '00', FI represents '01', F2 represents '10', and F3 represents '11'. The amplitude of MF signal still can follow the original video signal. Based on the same principle, the color signal also can carry a different set of MF format data. For example, color signal centered at F4 carries binary data '0'; color signal centered at F5 carries binary data 'V.
Similar to the concept of FM signal, the data may be transferred by modulating the phase of the data carrier. This method is called Phase Modulated (PM) data transfer. Data are represented by the phase change in the carrier signal. A variation of PM data is the multiple phase (MP) data format. Data are represented by video signals with discrete phases.
Low contrast-area data transfer (LCDT) in frequency modulated formats such as transferred in FM, MF, PF, or MP format has the advantage that the amplitude of the original video signal remains the same. It is therefor possible to transfer data in the low contrast areas without any change in the video information. On the other hand, human eyes are not sensitive to small changes in a low contrast area. It is therefore possible to insert amplitude modulated (AM) signal in the low contrast area. FIG. 2(e) shows an example when an AM data signal (253) is inserted into one of the low contrast areas (251) of the video signal in FIG. 2(a). One problem for inserting data into video signal is that it may create a random background color fluctuation on the TV image. Although a TN viewer will have no problem recognizing the original image, this background fluctuation can be annoying. The present invention provides methods to solve this problem. One solution is to transmit data in compensated formats (CF). For each data modulation done on one pixel, opposite modulation is done on a nearby pixel to compensate it. Since human eyes automatically takes averaged images of fine objects, the resulting image of compensated format will be very close to the original image. There are three ways to transmit CF format data. The first way is to put the AM data of opposite amplitude in a nearby line. When the CF data is implemented on a nearby line, it is called "vertical compensation" (NC). The pre-requirement for NC is that both lines need to have vertical low contrast areas. The same method can be implemented on nearby pictures. When the compensating data is carried by the same line at a different picture, it is called "time compensation" (TC). The pre-requirement for TC is that both lines need to have low moving contrast areas. FIG. 2(f) shows a line of video signal that is carrying VC or TC data signal (254) compensating for the AM data (253) in FIG. 2(e). The third CF method is to carry the compensated data in nearby pixels on the same line. This format is called horizontal compensation (HC). FIG. 2(g) shows an example HC data (255). Naturally, we can use a combination of those three compensation methods (HC, VC, TC) to represent data. Also note that each point on the screen actually has three colors (R, G, B). The compensated format data (253-255) shown in Figs. 2(e-g) are simplified for clarity. There are actually three degrees of freedom to represent and to compensate the modulated data. Data compensation can be implemented on one or more of the color components. The amplitude of the video signal is also not necessary linear proportional to the amplitude of light. Therefore, a proper compensation signal also needs to take alpha correction into consideration.
These compensation techniques (HC, VC, TC) not only improve picture quality, but also improve noise tolerance. When data is carried by compensated format, the data is determined by the changes in nearby points. Since most noise will have the same effects on nearby points, the resulting data signal has much better signal to noise ratio. This improvement in signal to noise ratio can allow us to carry more data in each point. The net result in data carrying capability is therefore not necessary worse than uncompensated data format.
Using compensated format, at least two pixels are required to carry each data point. In case it is not desirable to use multiple pixels to carry one data point, a differential format (DF) can be applied to solve the problem. FIG. 2(h) shows an example of data signal in differential amplitude (DA) format. Binary data Υ is represented by a change in amplitude in either up or down direction, and binary data '0' is represented by no change. If a parity bit is carried in every
8 data bits, it can be arranged to have the average amplitude to be zero for every
9 bits. The resulting disturbance in the original signal is therefore smooth and negligible. Similarly, the differential format can be employed to carry data signals with an FM, MF, PM, or MP formats. Naturally, combinations of multiple data formats, e.g., combinations of FM, MF, PM, MP, AM, CA, DA, can be employed to achieve higher data transfer rate when necessary.
FIG. 3(a) shows the block diagram for a hardware system executing a low contrast area data transfer (LCDT) process. A video signal analyzer (VSA) examines the outgoing video signal (OVS) looking for available bandwidth. This video signal analyzer detects changing rate of the out going video signal (OVS). Whenever low contrast areas are detected, the VSA sends signals to a signal , processor. In the mean time, the data providers also send data to the signal processor for data transfer. This signal processor inserts outgoing data signal (ODS) into the outgoing video signal (OVS) to create broadcast video signal (BVS). The BVS is broadcast through TN systems. Both TN users and data users receive and process the broadcast video signal. Data transfer methods of the present invention preserve the quality of video signal. The TN receivers process the BNS the same way as before, not affected by the data signal carried along with the broadcast video signal, to display high quality pictures with the processed BNS. The data receiver has a data signal analyzer (DSA) and data decoder. The DSA examines BNS and sends out a signal to the data decoder whenever data signal is found. The data decoder filters the right signal out of BNS, and extracts the correct data for the data user.
As an example, after processing the video signal shown in FIG. 3(b), the output generated by a low contrast area data transfer (LCDT) NSA would look like FIG. 3(c). Whenever the signal in FIG. 3(b) has a changing rate less than a pre-defined limit, the output of the level sensor is binary '1', otherwise it is '0', as shown in FIG. 3(c). There are three low contrast areas (311-313) in the video signal of the example in FIG. 3(b). In the mean time, the areas allowed for timing control signals, e.g., vertical sync, horizontal sync, and color burst, automatically meets the requirements for a low contrast area. Therefore, the signal in FIG. 3(c) also goes high during the horizontal blank interval. When ONS stay in low contrast area long enough to carry data, data signal is inserted to create the BNS as shown in FIG. 3(d). The data can be transferred using any one or any combination of the formats (FM, PM, MF, MP, DA, CA, CP) of the present invention. The LCDT data signal analyzer (DSA) also has the capability to detect low contrast area. Whenever a low contrast area is found, the data decoder will look for overlapped data signal, and extract the data by demodulation procedures.
The optimum data transfer rate of LCDT is strongly dependent on the video signal. Higher data rate can be achieved for special types of TN signals. For example, special data transfer rate can be achieved when the low contrast area is at black level. A Black Level Data Transfer (BLDT) method is disclosed in the present invention to take advantage of the special data transfer rate achievable in the black-level low contrast area. Similarly this invention also applies a White Level Data Transfer (WLDT) method by taking advantage of the higher data transfer rate in a white-level low contrast area. Furthermore, this invention also discloses a Blank Level Data Transfer (KLDT) method carry the data signals in a blank-level low contrast area. FIG. 3(d) shows example for the situations when BLDT (331), WLDT (332), common LCDT (333), KLDT (334) are used for the example video signal in FIG. 3(b). These data transfer methods at extreme cases (BLDT, WLDT, KLDT) are described in further details hereafter.
Black level data transfer (BLDT)
Video signals with a voltage level representing the black color are one of the signals most frequently transmitted to the TV receivers. Black as a color is frequently displayed on TV screen. Also, the signal's voltage level is applied as the upper level for video timing signals (H sync, V sync, color burst). In optical terms, black means no light. When the video signal is at black level, the corresponding picture element (pixel) should be totally black on the TN screen. According to optical concept, it is impossible to have a color darker than black. For TN signals, Black level is represented by an amplitude-modulated (AM) signal at 0.3 volts. The black level is set at 30% of full-scale amplitude because there is a need to have enough margin to define the "blank level" used for timing signals. Ideally, a video signal should never have a value between black level and blank level because nothing can be darker than black while it is not a timing signal.
In reality, a video signal lower than black level will be processed by the receiver as black, immediately after the receiver circuits detect a video signal level lower than the black level. In practical conditions, a spot on the TN screen can not be totally black; the TN screen may reflect lights from nearby light source even when the screen itself is not emitting light. Therefore, a video signal slightly higher than black level can be treated as black in practical conditions. The concept of "black" is therefore not strictly defined as represented by one- and-only signal level. The TN signals with amplitudes between lower black level and upper black level represent the same signal as far as TN display is concerned. It is therefore very convenient to carry along data signals with a signal representing black spots for TN display. The only limitation is to have the overlapped signals remain within the black level range. It should also be noted that the black level range is not a fixed signal range. At areas right next to a timing signal, the black level need to be accurate; at other areas, black level range can be very wide. The exact value for black level range is also dependent on the design of TV receiver circuits. Another important factor is that black is not processes as a color signal and mathematically black signal means R=G=B=Y=U=V=0. Therefore, there are full freedom to represent black using different carrier signals, as soon as the resulting amplitude fall between the upper and lower black levels.
To support BLDT, the VSA and DSA need to have a level sensor. This level sensor examines the video signal, and sends out a control signal whenever the video signal is within black level range. Using the video signal in FIG.4(a) as an example, the BLDT signal analyzer output would look like FIG.4(b). Whenever the signal in FIG.4(a) is within black level range, the output of the level sensor is binary 'V , otherwise it is '0', as shown in FIG. 4(b). When the video signal is found to stay within black level range long enough to carry data, data signal is inserted into the original video signal to create the video output signal as shown in FIG. 4(c). One obvious problem for the AM data signal in this example is noise sensitivity. The black level range is only a fraction of full-scale amplitude. The signal to noise ratio is therefore much smaller than TV signals. One solution to solve the noise problem is to use frequency modulation methods. Since black level does not have color information, we can change phase and frequency without influencing the quality of video display. Using FM methods we can keep the amplitude of the black level signal at 0.3 volts, while carrying high bandwidth data signal with FM, MF, PM, or MP methods described in previous sections. Because the amplitude of data signal is 0.3 volts, excellent signal to noise ratio can be achieved.
A narrow side band within the TV channel to carry the AM signal can be employed to improve noise tolerance for amplitude modulated black level data transfer (AMBLDT). A filter can also used to filter out most of noise at other frequencies and a compensated or differential format is also used for AMBLDT. Naturally, two or more of the data transfer formats of the present invention such as FM, PM, MF, DA, CA. CP described above can be combined to achieve higher data transfer rate.
White level data transfer (WLDT)
The concept of "white" for TV display is not the same as natural white color. In optical terms, white means a color with balanced color components as R=G=B. There is no limitation on the density of natural white light. For a TV display, the density of light emitted from the screen has a physical limit. Therefore, the TV signal has an upper limit on its amplitude. The concept of white for TV display means a light spot reaching the luminance limit of the TV screen with balanced color components as Y=R=G=B=1. Note that R=G=B is not enough to be defined as "white" for TV display; it is "gray" if Y is not at full scale. From electrical signal point of view, "white" is a signal with white level amplitude, that is, at 100% of full-scale amplitude. In reality, there is also an upper white level and lower white level. TV signals with amplitudes between upper and lower white level (white level range) will display "white" on TV screen. TV viewers can not distinguish any difference as soon as the signal is within white level range. Because "white level" is not strictly defined, we can use it to transfer more data in similar ways as BLDT. However, there is a major difference between WLDT and BLDT. For WLDT, if we change Y without changing color difference signals U and V, there will be differences in color. The color difference signals (U, V) occupies a side band centered at 4.43 MHZ above CF. The need to have balanced color reduced the degree of freedom in choosing frequency side bands for WLDT. On the other hand, WLDT signals are at full- scale amplitude, so that its noise tolerance is better than BLDT.
To support WLDT, the VSA and DSA need to have a level sensor. This level sensor examines the video signal, and sends out a control signal whenever the video signal is within white level range. Using the video signal in FIG.5(a) as an example, the WLDT signal analyzer output would look like FIG.5(b). Whenever the signal in FIG. 5(a) is within black level range, the output of the level sensor is binary Υ , otherwise it is '0', as shown in FIG. 5(b). When the video signal is found to stay within black level range long enough to carry data, data signal is inserted into the original video signal to create the video output signal as shown in FIG. 5(c). The data can be transferred using any one or any combination of the formats (FM, PM, MF, DA, CA. CP) described in previous sections. Unlike BLDT, for WLDT selection of the phase and frequency of the carrier signal need to take color into consideration.
Blank level data transfer (KLDT)
Blank level is used for timing signals such as horizontal sync, vertical sync, and color burst. Blank signal should have zero amplitude. In reality, timing circuits are most sensitive to the falling and rising edges of the timing signals. Other than those edges, timing circuits can tolerate signals with amplitude smaller than the blank level limit as blank signal. Therefore, we can insert data signals to replace blank signals as soon as the amplitude of the inserted signal is lower than the blank level limit. To support KLDT, the VSA and DSA need to have a level sensor. This level sensor examines the video signal, and sends out a control signal whenever the video signal is below blank level limit. Using the video signal in FIG. 6(a) as an example, the KLDT signal analyzer output would look like FIG. 6(b). Whenever the signal in FIG. 6(a) is below blank level limit, the output of the level sensor is binary 'V, otherwise it is '0', as shown in FIG. 4(b). When the video signal is found to stay within blank level range long enough to carry data, data signal is inserted into the original video signal to create the video output signal as shown in FIG. 6(c). The data can be transferred using any one or any combination of the formats (FM, PM, MF, DA, CA. CP) of the present invention. Similar to BLDT, we have total freedom to select the phase and frequency of the carrier signal because blank level does not have color.
The data transfer methods using the available bandwidth in the low contrast areas (LCDT, BLDT, WLDT, KLDT) have been disclosed in the above sections. Those methods provide data transfer bandwidth whenever the video signal is in low contrast areas. The average data transfer rate is therefore dependent on the property of TV image. For many types of applications, it is desirable to have a steady data transfer rate. Therefore, the present invention provides data transfer methods with transfer rate independent of the TN image, as disclosed hereafter.
Color Table data transfer (CTDT)
Color table is commonly used for computer display as a method to reduce the size of graphic files. A color table defines a finite number (16, 64, or 256) of colors. The color of each pixel in a picture is represented by one of the color in the color table that is closest to the original. For most cases, a 256-color table is adequate to display high quality pictures, especially when the content of the color table can be changed to adapt for different pictures. Definition of the colors in the color table is not unique. We can replace every entry of a color table with similar but different colors to create another table. The new table will still be able to represent high quality pictures. This property is used by a data transfer method of the present invention called color table data transfer (CTDT).
To support CTDT, both the data provider and data receiver need to agree upon two or more pre-defined color tables, e.g., TO and TI where TO represents the first color table and TI represents the second color table. These color tables can be changed but all the tables need to be coherent. FIG. 7(a) shows the flow chart for CCDT procedures. A video signal analyzer (NSA) determines the color for each pixel in the original video signal (ONS). For transmitting a binary number '0', the color of the pixel is replace by the best fit in table TO, and for transmitting a binary number Υ , the color of the pixel is replace by the best fit in table TI. The resulting picture (BNS) is broadcast through TN network. Since both color tables TI, TO are provided for producing high quality pictures, the resulting mixed video signal will be able to provide high quality display for TN viewers. A data receiver uses a data signal analyzer (DSA) to examine the BVS. When the color of a pixel is found in TO, a binary number '0' is received. When the color of a pixel is found in TI, a binary number Υ is received.
CTDT has the advantage that a data binary bit may be into every pixel of the video signal. It is not necessary to consider possible interference to the quality and color variations of the video signals due to the insertion of data. It is therefore not necessary to first determine the low contrast areas. CCDT data transmission can be flexibly applied to different portions of the display signals and does not have to be applied to the whole screen. It is usually advantageous to transfer data on part of the screen where a small color table may be used (e.g. 16 or 64 colors) to simplify supporting circuits.
Pre-defined object data transfer (PODT)
A pre-defined object (PO) is an object that is known to both the data sender and the data receiver. Examples of pre-defined objects are logo (102), score board (104), and caption frame (107), as shown in FIG. 1(a). Pre-defined objects usually are simple graphic figures with simple color pattern so that multiple methods of this invention (LCDT, BLDT, WLDT, CTDT) can be used to achieve high data transfer rate. Because we know exactly the color of each pixel, further data transfer methods are available. For example, an area in the PO can be designated to fill with any video signal. In this way, there will be no constraint on the data signal format within that area. Maximum bandwidth can be achieved within such area. Both the data sender and the data receive should have a PO library. For each PO in the library, the original color and the data transfer methods of every pixel is defined. To start a data transfer using PODT, the data provider simply notifies the data user when, where, and which PO is going to be used. FIG. 7(b) shows the flowchart as one example for showing the processing steps carried out to insert data into the pixels of the pre-defined object data for transferring data using the PODT procedure. Small Object data transfer (SODT)
Human eyes are not sensitive to a small object on a large picture. If wc select a few pixels on the screen to carry data, the effect of the data won't be visible as soon as the selected area is small enough. These small objects can be placed at a fixed place on the screen. It also can be a moving small object. The location even can be randomly selected as soon as both the sender and the receiver knows which pixels are carrying data. We have total freedom to use any combination of data formats of the present invention within the small object. Any combinations of the data formats of the present invention can be used for SODT. The data transfer procedures for SODT is the same as PODT as shown in the float chart in FIG. 7(b).
Invisible frame data transfer (IFDT)
Not all the video signals are displayed on TV screen. The first and last few lines of each frame are not displayed. The first and the last few pixels of each line are not displayed. Those lines and pixels outside of TV screen (109) are called the "invisible frame" (108). We can replace video signals in this "invisible frame" (108) with data signals as soon as (l)the spectrum of the data signal is within the bandwidth of the TV channel, (2)the amplitude of the data signal is within the ranges of conventional video signal, and (3)the timing signals (V sync, H sync, color burst) are preserved correctly. A data transfer method of the present invention using the invisible frame for data transfer is called "Invisible Frame Data Transfer" (IFDT). Since the video signal in the invisible frame is not used for TV display at all, there is highest degree of freedom in the data transfer format for IFDT. It is important to remember that there are prior art methods using part of the invisible frame to carry text. These prior art text signals always use blank level and black level. One way to maintain compatibility is to use BLDT and KLDT when the signal in the invisible frame is found to be at black level or blank level. For other levels, we have total freedom to use any combination of data formats of the present invention.
To support IFDT, the video signal analyzer (VSA) and the data signal analyzer (DSA) need to have a timing circuit. This timing circuit uses the video timing signals (V sync and H sync) and an internal timer to determine if the signal is within the invisible frame. When the video signal is found to stay within the invisible frame, data signal is inserted into the original video signal to create the video output signal. The data can be transferred using any one or any combination of the formats described in previous sections. FIG. 7(c) shows the float chart for one example of IFDT procedure.
Dedicated object data transfer (DODT)
While data transfer methods of the present invention is able to transfer data through TN signals without degrading picture quality, it is still not ethical to change the video signal without notifying TN viewers. Honesty is the best policy. We should always notify the TN viewers whenever we are using the video signal to transfer data. One way to do that is to display a special symbol (101) on one corner of the screen as shown in FIG. 1(a). In this example, we use characters "DT" to notify TN viewers that data transfer is executed. In case that the data transfer procedure indeed causes annoying effects, the TN viewers can feedback the problem to the data provider, and the data transfer methods should be improved. Naturally, the DT symbol (101) can be used for data transfer. One of the simplest methods to transfer data is to dedicate a small portion of the TV screen for data transfer. This method is called Dedicated Object Data Transfer (DODT) method. The data transfer procedures for DODT is the same as PODT as shown in the float chart in FIG. 7(b).
The present invention provides effective methods to utilize the TV network as a parallel path for Internet communication. Combining the data transfer methods (BLDT, WLDT, KLDT, VSDT, CTDT, PODT, SODT, IFDT, DODT) of the present invention, more than 90% of the TV bandwidth will be available for data transfer. The bandwidth for each pixel on a TV screen is about equal to 6 phone lines. If all the available TV channels are fully utilized, more than one billion bits per second (Gbps) is transmissible to every user in the world.
While specific data transfer methods of the invention have been illustrated and described herein, it is realized that other modifications and changes will occur to those skilled in the art. It should be understood that the above particular examples are for demonstration only and are not intended as limitation on the present invention.
A data transfer system of the present invention does not replace existing communication systems. Instead, it provides additional data path to existing systems. FIG. 8(a) shows a broad view of a communication system of the present invention. Information users (801) and information providers (803) are connected through Internet. Currently, the backbone of Internet is built on telephone systems and computer networks. In the mean time, TV stations (805) broadcast video signals to TV viewers. Currently, there are no connections between Internet and TV networks. Information users (801) send requests, commands, selections, and simple e-mails through the Internet. Usually those activities require little bandwidth; a simple telephone line is more than enough to handle them. Currently, the information providers (803) also use Internet for data transfer. The bottle neck is the bandwidth requirement for a popular Web site to send large data files to users. The present invention provides an alternative data path. Information provider can send data to a TV station with a data encoder of the present invention. This data encoder contains a video signal analyzer (VSA) and a video signal processor (VSP). The video signal analyzer (VSA) examines the outgoing video signal (OVS) looking for available bandwidth. The output of VSA is sent to VSP to select proper data transfer methods. This signal processor (VSP) inserts outgoing data signal (ODS) into the outgoing video signal (OVS) to create broadcast video signal (BVS). The BVS is broadcast through TV systems, and it is processed by both TV receivers and data users (801). Data transfer methods of the present invention preserve the quality of video signal, so that the TN viewer still can receive high quality video display using BVS. The data receiver has a data signal analyzer (DSA) and data decoder (807). DSA examines BVS and sends out a signal to the data decoder (807) whenever data signal is found. Control information such as the type of data transfer and the encoding keys are provide by the information providers (803) either through Internet or transferred as part of ODS. The data decoder (807) filters the right signal out of BVS, and extract the correct data for the data user. This alternative data path through TN network has much wider bandwidth then the telephone networks. It is therefore possible to improve overall system performance dramatically. FIG. 8(b) shows the layer structure of the system in FIG. 8(a). Most of the layer structures and communication protocols are the same as those of current art communication systems. There are no changes on application, presentation, session, and transport layers. Therefore, a communication system of the present invention can use most of existing software and hardware. The present invention provides one way parallel paths in the lower level layers.
The system in FIG. 8(a) can be used in a wide variety of communication applications. We will use a video game rental system as an example to demonstrate operation principles for communication systems of the present invention. FIG. 9(a) shows the block diagram for a video game controller of the present invention. This video game controller is equipped with a TV signal interface (901) for receiving television signals. Typical examples for this TN signal interface (901) are connections to TV antenna or cable TN box. TN signal received by the TN interface (901) is sent to a data decoder (903). The data decoder (903) is used to extract data from TN signal. This video game controller is also connected to an Internet interface (905). A typical example of an Internet interface is a computer equipped with modem. This Internet interface also can be placed inside of the video game control box. Both the Internet interface (905) and the data decoder (903) are connected to a storage unit (907) and a video game control unit (909). The storage unit (907) is a memory device used to store data. Typical examples of the storage unit are hard disk or tape. The video game controller (909) is the same as current art video game controllers except that it has programmable firmware to allow re-configuration for different games. Video game players can play different games by programming the controller firmware.
For a system which does not have TN interface (901), a player need to scan the web side of a video game provider, then load the whole set of a video game program into the game controller (909) in order to play a new game. If there are 1 million players wanting the same game, the same procedures will be repeated one million times. Most likely the web site will be jammed by requests for popular games. Even if the web side has enough bandwidth to handle the request, it is still a tremendous waste in resource.
When the system is equipped with TN interface (901), the procedures to obtain a new game will be extremely efficient. The video game player uses the Internet interface (905) to select a new game. The game provider sends a "decoder key" to the player. This "decoder key" tells the data decoder (903) when and how to down load data from the TV interface (901). High volume data such as the video images of web pages and the game programs are transferred through the TN interface using data transfer methods of the present invention. The Internet interface (905) only handle slow activities such as selection of game or transfer of the decoder key. The same decoder key can be given to multiple users, so that when many users are requesting for the same data simultaneously, the provider only need to send one copy through the TN interface. A data transfer that is initiated immediately after a request from the user is called a real time (RT) data transfer. One problem for RT data transfer is that players usually send out their requests at different time. If the provider always send out the data from the very beginning whenever a request is received, the same data will need to be sent many times. One way to solve the problem is to delay the data transfer, accumulate many requests, then send one copy out to satisfy all the requests. This method is called delayed data transfer. The other way to solve the problem is to break a large file into small packages. The game players do not need to receive a large data file from the beginning. Small packages can be received out of sequence. The final data file is established after all packages are received. This method is called package data transfer. Using package data transfer, the game provide simply keep on sending out packages of requested games as soon as there are requests for that game. All the players requesting the same game are given the same key. Whenever a player has collected all the necessary packages, a signal is sent back to the provider to notify end of request. The game providers stop the procedure when all the requests are satisfied. Another method to solve the problem is to schedule the TN data transfer ahead of time. This method is especially useful for introduction of a brand new game. All the players wanted the new game are given a key to access the data. Data for the new game is sent out at a pre-defined time to all players. In this way, the provider only needs to send one copy once. Another method is for the provider to send data to players who are likely to want the data before the player actually request for the game. These pre-sent data are stored in the data storage unit (907). When the player actually send out a request, game control software will first look into the storage unit (907). If the game already pre-sent into the storage unit, the provider only needs to give the player a key to activate the game; there is no more need for data transfer. Only when the requested game is not found in the storage unit (607) does the provider need to send new data to the player. FIG. 9(b) shows a float chart for the above communication procedures. The data transfer methods of the present invention are so efficient that they can support thousands of people playing the same game simultaneously. To support such a large scale game, each video game system should store the game map. Players send commands through conventional Internet connections, while a central system update the results through TN data transfer methods of the present invention. The TN signal updates all the game maps in all the involved individual systems with a single broadcast. In this way, thousands of people can play the same game without jamming the system.
Another practical application of the present invention is a real-time stock market data update system. FIG. 10 shows an example of stock market data update system of the present invention. This system is identical to current art stock market data update system except for a TV signal interface (991) that can obtain data from TN signals. The system is still connected to internet. Users still send their requests through Internet. The software programs used to display stock market information are the same as current art software programs. The only difference is that there is a parallel data path from the stock information provider to all users through TN data transfer methods of the present invention. In a prior art system, the most updated stock prices are sent to millions of users through Internet. That means millions of duplicated copies are sent to individual users. Using the TN data transfer methods of the present invention, the stock information providers only need to send out one copy of the latest stock data. Updating latest stock price only requires a few thousand bits per second. Any one data transfer method of the present invention will easily handle the bandwidth requirement, while all the users can obtain real-time stock prices simultaneously with minimum delay. Note that the system does not have to use a computer. A video game controller described in FIG. 9(a) can be programmed to have stock update capability.
Data transfer system of the present invention uses existing TV broadcast systems to send data. It will satisfy the bandwidth requirement for many applications without any changes to existing system. The system requires little resource to implement. It is the most cost efficient method to solve the bandwidth problem, and the system can be established in a short time.
The most important limitation for these data transfer system is that they are one-way broadcast system. The transmission path tends to be noisy. It is therefore necessary to implement data quality control methods such as parity check, check sum, error correction code, Hemming code, ... etc. Those methods to assure data quality for a noisy media are well-known to the art. There is no need to describe them in details. Another important issue is security. Security measured to protect broadcast data should be implemented for data with security concerns.
Although the present invention has been described in terms of the presently preferred embodiment, it is to be understood that such disclosure is not to be interpreted as limiting. Various alternations and modifications will no doubt become apparent to those skilled in the art after reading the above disclosure. Accordingly, it is intended that the appended claims be interpreted as covering all alternations and modifications as fall within the true spirit and scope of the invention.

Claims

CLAIMS I claim:
1. A wafer comprising a plurality of repeated unit disposed
(a)examining a TN video signal, comprising electromagnetic (EM) waves distributed over time, for finding a time slot with a suitable EM wave transient rate;
(b) generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal in said time slot having said suitable EM waves transient rate; and
(c) transmitting said data-carrying TN signal to a TN and a data receiver.
2. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a frequency-modulated (FM) data signal into said time slot having said suitable EM wave transient rate.
3. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting multiple frequency-modulated (MF) data signals into said time slot having said suitable EM wave transient rate.
4. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a phase-modulated (PM) data signal into said time slot having said suitable EM wave transient rate.
5. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a multiple-phase-modulated (MP) data signal into said time slot having said suitable EM wave transient rate.
6. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a modulated data signal with a compensated format (CF) into said time slot having said suitable EM wave transient rate.
7. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a compensated-amplitude (CA) modulated data signal into said time slot having said suitable EM wave transient rate.
8. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting a differential amplitude (DA) modulated data signal into said time slot having said suitable EM wave transient rate.
9. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting said data signal into said time slot employed for black level data transfer (BLDT).
10. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a frequency-modulated (FM) data signal into said time slot employed for BLDT.
11. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a multiple-frequency-modulated (MF) data signal into said time slot employed for BLDT.
12. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a phase-modulated (PF) data signal into said time slot employed for BLDT.
13. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a multiple-phase-modulated (MP) data signal into said time slot employed for BLDT.
14. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a modulated signal with compensated-format (CF) as data signal into said time slot employed for BLDT.
15. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a compensated amplitude (CA) modulated data signal into said time slot employed for BLDT.
16. The data transfer method of claim 9 wherein:
said step of inserting said data signal into said time slots employed for black level data transfer (BLDT) comprising a step of inserting a differential amplitude (DA)-modulated data signal into said time slot employed for BLDT.
17. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting said data signal into said time slot employed for white level data transfer (WLDT).
18. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a frequency-modulated (FM) data signal into said time slot employed for WLDT.
19. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a multiple-frequency-modulated (MF) data signal into said time slot employed for WLDT.
20. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a phase-modulated (PF) data signal into said time slot employed for WLDT.
21. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a multiple-phase-modulated (MP) data signal into said time slot employed for WLDT.
22. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a modulated signal with compensated-format (CF) as data signal into said time slot employed for WLDT.
23. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a compensated amplitude (CA) modulated data signal into said time slot employed for WLDT.
24. The data transfer method of claim 17 wherein:
said step of inserting said data signal into said time slots employed for white level data transfer (WLDT) comprising a step of inserting a differential amplitude (DA)-modulated data signal into said time slot employed for WLDT.
25. The data transfer method of claim 1 wherein:
said step (b) of generating a data-carrying TN signal by inserting into said TN signal a hidden-from-viewer data signal comprising a step of inserting said data signal into said time slot employed for blank level data transfer (KLDT).
26. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a frequency-modulated (FM) data signal into said time slot employed for KLDT.
26. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a multiple-frequency-modulated (MF) data signal into said time slot employed for KLDT.
27. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a phase-modulated (PF) data signal into said time slot employed for KLDT.
28. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a multiple-phase-modulated (MP) data signal into said time slot employed for KLDT.
29. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a modulated signal with compensated-format (CF) as data signal into said time slot employed for KLDT.
30. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a compensated amplitude (CA) modulated data signal into said time slot employed for KLDT.
31. The data transfer method of claim 25 wherein:
said step of inserting said data signal into said time slots employed for blank level data transfer (KLDT) comprising a step of inserting a differential amplitude (DA)-modulated data signal into said time slot employed for KLDT.
32. The data transfer method of claim 1 wherein:
said step (c) of transmitting said data-carrying TN signal to a TN and a data receiver further comprising a step of storing a data transmitted by said data-carrying TN signal in a user-accessible data-storage in said data receiver.
33. A data transfer method comprising steps of
(a) rearranging said TN signal into a non-viewer-interfering data- carrying TN signal; and
(b) transmitting said data-carrying TN signal to a TN and a data, receiver.
34. The data transfer method of claim 33 wherein:
said step (a) of rearranging said TN signal into said non-viewer- interfering data-carrying TN signal comprising a step of arranging said TN signal according to a color table data transfer (CTDT) method by best fitting a TN pixel signal to a color in one of at least two color tables for representing a binary level of a data according to a color table employed for encoding said binary level of said data into said TN pixel signal.
35. The data transfer method of claim 33 wherein:
said step (a) of rearranging said TN signal into said non-viewer- interfering data-carrying TN signal comprising a step of arranging said TN signal according to a predefined object data transfer (PODT) method by prearranging a TN pixel signal for showing a designated object and employing said TN pixel signal for transmitting a data signal.
36. The data transfer method of claim 33 wherein:
said step (a) of rearranging said TN signal into said non-viewer- interfering data-carrying TN signal comprising a step of arranging said TN signal according to a small object data transfer (SODT) method by detecting a TN pixel signal for showing a small object and employing said TN pixel signal for transmitting a data signal.
37. The data transfer method of claim 33 wherein:
said step (a) of rearranging said TN signal into said non-viewer- interfering data-carrying TN signal comprising a step of arranging said TN signal according to a dedicated object data transfer (DODT) method by designating a TN pixel signal for showing a dedicated object and employing said TV pixel signal for transmitting a data signal.
38. The data transfer method of claim 33 wherein:
said step (a) of rearranging said TN signal into said non-viewer- interfering data-carrying TN signal comprising a step of arranging said TN signal according to an invisible frame data transfer (IFDT) method by determining a TN pixel signal in an invisible frame and employing said TN pixel signal for transmitting a data signal.
39. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a frequency-modulated (FM) data signal.
40. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a multiple frequency-modulated (MF) data signal.
41. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a phase-modulated (PM) data signal.
42. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a multiple-phase-modulated (MP) data signal.
43. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a modulated data signal with a compensated format (CF).
44. The data transfer method of claim 38 wherein:
said step (a) of employing said TV pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a compensated-amplitude (CA) modulated data signal.
45. The data transfer method of claim 38 wherein:
said step (a) of employing said TN pixel signal in said invisible frame for transmitting a data signal comprising a step of transmitting a differential amplitude (DA) modulated data signal.
46. A video game system comprising:
a TN signal interface /decoding means for receiving a TN signal encoded with a video-game data-signal therein for decoding and employing said data signal.
47. The video game system of claim 46 wherein:
said TN signal interface /decoding means comprising a TN interface means for receiving said TV signal encoded with said data-signal from a TV.
48. The video game system of claim 46 further comprising:
a video game controller for allowing a video game player to control and play a video game on said video game system.
49. A stock price update system comprising:
a TV signal interface /decoding means for receiving a TV signal encoded with a data-signal comprising stock price data.
PCT/US2002/011181 2002-04-09 2002-04-09 Data transfer using television video signal WO2003088669A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2002/011181 WO2003088669A1 (en) 2002-04-09 2002-04-09 Data transfer using television video signal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2002/011181 WO2003088669A1 (en) 2002-04-09 2002-04-09 Data transfer using television video signal

Publications (1)

Publication Number Publication Date
WO2003088669A1 true WO2003088669A1 (en) 2003-10-23

Family

ID=29247969

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/011181 WO2003088669A1 (en) 2002-04-09 2002-04-09 Data transfer using television video signal

Country Status (1)

Country Link
WO (1) WO2003088669A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5589872A (en) * 1986-05-14 1996-12-31 Radio Telecom & Technology, Inc. Compatible interactive TV and multimedia delivery system two-way cable TV conversion system for data transmission over a dedicated frequency band
US5761602A (en) * 1995-09-13 1998-06-02 Wagner Dsp Technologies Hybrid multichannel data transmission system utilizing a broadcast medium
US5861881A (en) * 1991-11-25 1999-01-19 Actv, Inc. Interactive computer system for providing an interactive presentation with personalized video, audio and graphics responses for multiple viewers

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5589872A (en) * 1986-05-14 1996-12-31 Radio Telecom & Technology, Inc. Compatible interactive TV and multimedia delivery system two-way cable TV conversion system for data transmission over a dedicated frequency band
US5596361A (en) * 1986-05-14 1997-01-21 Radio Telecom & Technology, Inc. Two-way cable TV conversion system for data transmission over a dedicated frequency band
US5812184A (en) * 1986-05-14 1998-09-22 Radio Telecom & Technology, Inc. Transmission of frame-to-frame reversed polarity signals in a two-way cable TV conversion system
US5861881A (en) * 1991-11-25 1999-01-19 Actv, Inc. Interactive computer system for providing an interactive presentation with personalized video, audio and graphics responses for multiple viewers
US5761602A (en) * 1995-09-13 1998-06-02 Wagner Dsp Technologies Hybrid multichannel data transmission system utilizing a broadcast medium

Similar Documents

Publication Publication Date Title
KR100632838B1 (en) Expanded information capacity for existing communication transmission systems
US5587743A (en) Signal processors for transparent and simultaneous transmission and reception of a data signal in a video signal
US8161518B2 (en) Contents distribution system and distribution server and display terminal of contents distribution system
Sillman Television Captioning for the Deaf
JPH08237622A (en) Communication system for digital information in video signal
US20030169367A1 (en) System and method of adaptive timing estimation for horizontal overscan data
JP3083652B2 (en) Teletext transmission and reception device
US4335402A (en) Information transmission during first-equalizing pulse interval in television
US20060020983A1 (en) Data transfer using existing communication systems
US20060007355A1 (en) Data transfer using television video signal
WO2003088669A1 (en) Data transfer using television video signal
JPS58501103A (en) Color television system
US6005606A (en) Still picture television system
KR100639052B1 (en) Compatible transmission and reception of operational signals to be processed by a receiver
TW561784B (en) Data transfer using television video signal
WO1996038000A1 (en) Time-division still picture television system
CN1195376C (en) Caption signal broadcasting method for audience selecting caption broad cast
EP1931144A1 (en) Device for processingvideo images, video projection system and signal intended for use by the projection system
US6226042B1 (en) Color encoder
US7215711B1 (en) Broadcasting computer files using television
CN1462149A (en) Data transmission using television video frequency signal
Sadiku et al. High definition television in detail
KR0182434B1 (en) Data bit transmitter receiver and aspect ratio auto control system of tv receiver using back porch of composite signal
O’Reilly et al. Systems Case Studies
MXPA00010114A (en) Expanded information capacity for existing communication transmission systems

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CN JP KR

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP