US20060287140A1

US20060287140A1 - Automated line calling system

Info

Publication number: US20060287140A1
Application number: US11/154,004
Authority: US
Inventors: Richard Brandt; Alexander Jaoshvili; Ilya Rosenberg
Original assignee: Individual
Current assignee: Individual
Priority date: 2005-06-16
Filing date: 2005-06-16
Publication date: 2006-12-21
Also published as: WO2006138618A3; WO2006138607A3; WO2006138607A2; US20080314165A1; US7984544B2; WO2006138618A2; US20090143174A1

Abstract

A system and appertaining method are described to detect whether a projectile impact occurs on one side of a boundary line or the other. The system utilize force sensing resistor-based sensors that may be configured in sections or groups and connected to a control system with a display. An impact classification system is provided for distinguishing between various events, including a footstep, ball impact and tennis racquet contact. A sensor monitoring system is provided for determining the health of sensors and provding an error indication if sensor problems exist. A service detection system is provided when the system is used for tennis that permits activation of selected groups of sensors and deactivation of others.

Description

BACKGROUND

The present invention is directed to a system for determining if a projectile has impacted a surface on one side of a boundary line or another utilizing a force sensing resistor (FSR) or force sensing rubber sensor or other sensor for measuring force. Such a system may be used in a sport such as tennis in which a call must be made as to whether a tennis ball has landed in bounds or out of bounds.
Various sensor-based systems for monitoring such impacts are known. Many of these revolve around the use of a membrane switch or some form of a switch for the detection. However, such membrane-switch-based systems lack accuracy at the boundary and are not sensitive enough to light ball bounces, and generally utilize debounce circuitry for processing signals. Patents utilizing some form of this technology include, e.g., U.S. Pat. No. 5,672,128 (Conn); U.S. Pat. No. 3,982,759 (Grant); U.S. Pat. No. 4,365,805 (Levine). The lack of sensitivity in the Conn system cannot be corrected by minimizing the amount of “dead” area, because if the openings are made large enough to detect light impacts through a layer of tennis court paint, then even a slight error in manufacturing, or a slight bump in the court surface can cause the sensor to be stuck in the conducting state. This is unacceptable for a commercial system that would be required to work reliably for years in widely varying environmental conditions, and would have players running, sliding, and jumping on it. Furthermore, different versions of the system, with different sensitivities would have to be made for different court surfaces, since they have different thicknesses, and flexibility. Because these systems can only sense the presence or absence of a force above a certain threshold, during the start or end of a footstep, they may produce signals that may be confused with ball bounces. Typically, these systems use debouncing circuitry or algorithms to filter such events. Thus, they ignore signals that look like ball bounces within a predetermined time window before and after the detection of a footstep. Thus, they are unable to detect ball bounces that occur within a short time before or after a foot steps. Furthermore, despite these measures, systems that use membrane switches often misclassify foot steps as ball bounces, and are typically incapable of distinguishing events such as a tennis racquets hitting or scraping the sensors from a ball bounce.
A number of known systems are based on the use of piezoelectric mechanisms, such as U.S. Pat. No. 4,840,377 (Bowser); U.S. Pat. No. 4,855,711 (Harrop); and U.S. Pat. No. 6,367,332 (Fisher—utilizes two triboelectric cables to detect vibration). Although these systems utilize piezoelectric devices in different shapes that can work well under certain situations, they are quite expensive and can fail to detect a static force applied to them. U.S. Pat. No. 5,954,599 (Lin) teaches a use of two cables that conduct mechanically when pressure is applied; the use of two cables does not provide the necessary area coverage to detect balls farther away from the lines.
Other systems utilize exposed conductors on the court which can be difficult to install, can interfere with the game, and can be difficult to keep clean. For example, U.S. Pat. No. 4,092,634 (Kohorn) requires the use of a conductive ball. U.S. Pat. No. 5,900,292 (Brace) utilizes conductive pins on the court. U.S. Pat. No. 4,109,911 (Auken) uses wires on the court.
At present, the only practical and commercially viable line calling systems in use are camera-based systems (and Cyclops™ which uses infrared or similar beams across the service line so that when the ball crosses the beam, the detector is able to tell if it was “out”). Other camera- or laser-based systems are known. The primary disadvantage of camera based systems is that they are expensive and do not actually sense the ball impact. Furthermore, these systems generally require very bright lighting to work effectively. U.S. Pat. No. 6,816,184 (Harmath) discloses a system that utilizes a camera based system. This system requires many cameras and high-speed computers, and does not actually sense the impact point of the ball. Instead, it extrapolates it from a video feed. Finally, U.S. Pat. No. 5,059,944 (Carmona) discloses a system that utilizes lasers to detect the ball bounce. Such a system is not very accurate because the lasers have to be a certain distance above the court. Thus, they can call a ball that is flying at a shallow angle in, when it is really out. This is a similar problem that plagues the Cyclops system, which is why so many players complain about it. Also, such systems can be disturbed by bugs or debris landing on the court, and this system is prone to misalignment. Furthermore, the devices are visible and obtrusive and must be avoided by players, officials, and spectators that may be on the court.
U.S. Pat. No. 4,990,897 (Beyma) discloses the use of sensors whose resistance varies with force, but utilizes a constant current source that could consume substantial power when the system is not active if a current were fed through many sensor areas. Furthermore, very high voltages would be needed to drive a sensor such as a typical FSR sensor which has a resistance greater than 1 MΩ in an inactive state.
Beyma further utilizes a rate of change for signal calculations, which can make distinguishing a footstep, ball bounce, tennis racquet difficult because these can produce very similar rates of change and cannot be used to determine when a player is standing on the line.
Furthermore, Beyma uses a signal that is proportional to the resistance of the sensor. However, the resistance of such sensors is inversely proportional to the force applied. As a result, when a player stands on the sensor, the rate of change produced by a bouncing ball will be different from the rate of change produced when a player is not standing on a sensor.

SUMMARY

The present invention is a system that can be used to make line calls both in un-refereed games as well as assisting umpires in making good calls for refereed games.
In an embodiment of the invention pertaining to a tennis court, the line calling system comprises a plurality of sensors embedded under tennis court paint or other tennis court covering, wiring to connect the sensors, a control box that receives the incoming signals, processes the signals, allows configuration of the system, and notifies players or umpires when a ball is detected in or out. Notification may be done using an audio notification when the ball is out, a visual display, or by sending a message over a wired or wireless network.
According to an embodiment of the invention, a sensor based on force sensing resistors (FSR) is utilized in order to avoid the above mentioned problems of membrane switches. This is because, unlike a membrane switch, an FSR does not produce a simple ON/OFF signal. Instead, an FSR sensor has a resistance that decreases with the amount of force applied, giving an analog signal that can be analyzed for much more accurate results.
The resistance of an FSR may range from several mega-ohms when there is no force on the sensor to a few hundred ohms when a ball bounce or a footstep on the sensor occurs. The amount of force on the sensor can be approximated by one over the resistance of the sensor, also known as the conductance, which is approximately proportional to the force applied to the sensor. The conductance can be measured by various means such as applying a constant voltage to the sensor and measuring the current through the sensors, or by using a circuit that generates a voltage that is related to the conductance of the sensor in a known way, and measuring that voltage with an analog to digital converter. By analyzing this signal, one can accurately distinguish between a ball bounce, a foot step, the impact of a tennis racquet, and other types of contacting events.
In the case of a ball bounce, one can determine whether the ball was IN or OUT with an accuracy of a few millimeters, and one can even detect very light bounces, such as those from a drop shot. Furthermore, because the signal is analog, one can filter out the constant component of the signal which can be affected by sensor manufacturing, court surface material, imperfect installation, bubbles and cracks in the court surface, changing weather, the gradual wear of the sensor, etc.
Thus, unlike a membrane switch, it is almost impossible for the FSR to get stuck in the ON state. Even though the constant component of the signal may be filtered out, the transient changes which result from ball bounces, foot steps, tennis racquet impacts and the like can still be detected.
Because the signal is analog, it does not have to be de-bounced, and can be analyzed much more accurately than the ON/OFF signal from a membrane switch. As a further advantage, a sensor problem can be detected if the constant component of the signal is too high, indicating a short, or the signal is too low, indicating a break in a wire. As a further advantage, a base-level “off” signal, can be determined. Thus, even small forces can be detected as they will cause an increase above the base-level signal.
The line calling system can be made where a permanent resistor with a high resistance at each sensor area is used to provide a small leakage current. The absence of such a leakage current can be used to detect a break in the wire. Additionally, an integrating algorithm can be provided to make the correct decision by using the position and timing of events that happen within a small period of time to determine where the ball actually bounced (as described above).
Furthermore, the FSR sensors can be made with a total thickness of approximately 0.005″-0.050″—thin enough to be completely hidden under tennis court paint, without having to cut the concrete or asphalt base of the court. All of these aspects are advantageous for a commercially viable line calling system, and most of these can not be accomplished with the older membrane switch technology.
Advantageously, this approach is very accurate for all types of ball bounces, in a wide variety of weather and lighting conditions, and under all the situations that can occur during a tennis match. The system is exceptionally consistent in that it calls the same ball bounce the same way every time, on either side of the court—therefore, whatever margin of error exists on one side of the court will exist on the other, giving neither side an unfair advantage.
This system can work on cement or asphalt courts, which are painted with tennis court paint, cushioning system, or covered with a surface such as rubber tiles or synthetic grass.
And also advantageously, the present invention can be implemented simply and affordably so that it is commercially viable not only for big tournaments (e.g., grand slams), but also for clubs, universities, and private tennis court owners. It is unobtrusive so that it can work with existing court surfaces, tennis balls, shoes, and other athletic equipment. Players do not detect differences between a regular court and one with the inventive system installed, except for the small device that provides audio/visual indication about whether a ball was in or out.
The present invention is reliable and durable, so that it may last for many years, and can be configured to automatically detect if there is a problem requiring maintenance or replacement.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a top view of a tennis court layout comprising an embodiment of the invention;
FIG. 2A is a top view of a Section A corner area sensor overlap;
FIG. 2B is a top view of a Section B T-intersection area sensor overlap;
FIG. 2C is a top view of a Section C T-intersection of the service lines area sensor overlap;
FIG. 2D is a top view of a multi sectioned sensor layout in which the out portion is segmented;
FIG. 3A is a top view illustrating the detail of a bottom layer of the sensor with an interdigitated finger configuration;
FIG. 3B is a bottom view illustrating the detail of a top layer of the sensor with an interdigitated finger configuration;
FIG. 3C is a top view of the top layer of an exemplary sensor;
FIG. 3D is a top view of the bottom layer of the sensor shown in FIG. 3C;
FIG. 3E is a cross-sectional view of the sensor shown in FIGS. 3C and 3D;
FIG. 3F is a cross-sectional view of a sensor having an interdigitated finger layout;
FIG. 3G is a cross-sectional view of a sensor using a force sensing rubber configuration;
FIGS. 4A-D are top views of various sensor/glue layouts;
FIG. 5A is an isometric pictorial illustration of the sensor-cable interface using a flex cable connector;
FIG. 5B is an isometric pictorial illustration of the sensor-cable interface using a soldered ribbon cable;
FIG. 5C is an isometric pictorial illustration of the sensor-cable interface having an attached circuit board;
FIG. 5D is an isometric pictorial illustration of the sensor-cable interface using a flex circuit;
FIG. 5E is top view of the sensor-cable interface using embedded A/D converters and microprocessors along the length of the sensor;
FIG. 6 is a cross-section view of the sensors embedded in a tennis court;
FIG. 7A is a perspective pictorial diagram illustrating an exemplary control box mounted on one side of the net;
FIG. 7B is a pictorial front view control panel of the control box;
FIGS. 8A, B illustrate a voltage divider and op-amp embodiment for the control box;
FIG. 9 is a block diagram of the system hardware/software components;
FIG. 10A is a graph illustrating a typical shape waveform for a light ball impact;
FIG. 10B is a graph illustrating a typical shape waveform for a medium ball impact;
FIG. 10C is a graph illustrating a typical shape waveform for a hard ball impact;
FIG. 10D is a graph illustrating a typical shape waveform for a ball impact at a sharp angle;
FIG. 10E is a graph illustrating a typical shape waveform for a hard foot kick;
FIG. 10F is a graph illustrating a typical shape waveform for a foot slide;
FIG. 10G is a graph illustrating a typical shape waveform for a hard racquet hit;
FIG. 10H is a graph illustrating a typical shape for a normal racquet hit with dual thresholding;
FIG. 10I is a graph illustrating a typical shape for a racquet scrape;
FIG. 10J is a graph illustrating a typical shape for a simultaneous footstep and ball impact;
FIG. 11 is a pictorial top view illustrating sensor areas with chained resistors;
FIG. 12 is a plot of the combined chained section signals received;
FIG. 13 is a plot of the calculated isolated ball bounce signal; and

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Overview
FIG. 1 illustrates an exemplary embodiment of the invention for a tennis court 10. The court 10 is defined by a rectangular area having boundary lines 14 and a net 12 that bisects the rectangular area. A ball is considered IN if, upon impact, it touches the boundary line 14 of a court that is in play, and OUT if it bounces outside of the court without hitting the line 14. A control box 102, explained in more detail below, is shown in an exemplary configuration located proximate to one end of the net 12.
Although it is preferable to have the sensors along all of the boundary lines of a court as in FIG. 1, to reduce cost, sensors may cover a subset of the boundary lines on a court. For instance, sensors may be placed only along the service area to just call serves, or sensors may be placed only along the singles court and service area and not the doubles court.
Sensors
In the primary embodiments of the invention discussed below, a sensor 22 has two sensing areas, an IN sensing area 24 that it is arranged at and inside of the boundary lines 14, (the “IN side”), and an OUT sensing area 26 that is laid out to go outside of the boundary lines 14 (the “OUT side”). The advantage to having both areas is that a ball bounce that is on the line (and hence “IN”) will produce some signal on the IN sensing area 24 and can therefore be used in conjunction with the signal detected on the OUT area to better classify (with more accuracy) such a bounce as “IN”.
However, additional embodiments are possible that only utilize an OUT sensing area. Such embodiments may be utilized when cost or system simplicity can be traded for accuracy. Without having an IN area to provide a definitive indication of the ball bounce on the boundary line 14, the classification of a curve as being IN or OUT becomes difficult for close calls, and an arbitrary criteria must be established to make the determination that is less likely to be accurate. Nonetheless, the accuracy of such a system will be sufficient for many non-professional situations.
Court Layout
The primary interest is in detecting balls that are OUT—therefore, the OUT side 26 of the sensor 14 should be wide enough to detect all ball bounces that may come under dispute because they are too fast for the human eye to see. The necessary width may depend on the experience of the players, the presence of line judges, and the seriousness of the event. Furthermore, the width may vary depending on the location of the line. For instance, the service line 14.1, since it may receive balls traveling over 100 miles per hour, may need to be wider than the base line 14.2. Typically, the necessary width of the OUT sensors 26 will be somewhere around 1.5′, because when a ball bounces more than this distance outside the boundary, it is very easy to judge accurately.
The purpose of the IN side 24 of the sensor 22 is to suppress an OUT call when a ball hits both the IN 24 and the OUT 26 sides of a boundary 14. A ball is considered IN if any portion of it touches the IN side of the boundary, and its presence may be detected by a simultaneous signal on both the IN side 24 and the OUT side 26 of the sensor 22, but also, the ball can just land on the IN part without activating the OUT part.
Thus, the IN side 24 of the sensor 22 may be as narrow as 1″, but may be made wider to cover the entire width of the boundary line, which is typically 2″ wide all around the court. During different stages of play, such as service to the odd or even court, singles rally, or doubles rally, different sections of the boundary lines 14 will have to act as IN/OUT. This is accomplished by dividing the boundary lines 14 into multiple sections (Sections A, B and C), each section having an IN side 24 and an accompanying OUT side 26. See FIGS. 2A-C. To prevent dead spots in areas where two (or possibly more) sensor strips meet, the sensors can be designed to overlap in those areas such that when they are overlapped, the entire area at the intersection can be sensed by one or the other sensor. The sensors are designed to be thin enough to permit them to overlap without creating bumps on the court that could interfere with the game and because they are flexible, the sensor on the bottom will still be able to feel forces through the sensor above it.
The sensor section layouts are not limited to being segmented into solely a single IN portion 24 and a single OUT portion 26, but rather can be configured to have multiple segments, such as that illustrated by FIG. 2D, in which, by way of example, the OUT portion 26 is broken into three distinct segments.
The sections may be further subdivided into subsections (see FIG. 3A, where each wire is connected to a subsection), each subsection having an IN and an accompanying OUT side in order to allow the detection of a ball bounce even while a player is standing or running over a boundary line. This is necessary because a player standing on a line may create a signal that will “mask” the signal produced by a bouncing ball. Because the OUT side of each subsection may be approximately 1.5 feet wide, it may be divided further into sensor areas of smaller width. Note that it is not necessary to divide the service lines into subsections because a player should never stand on a service line during service. The smaller the sensor areas, the less the chance that a ball could hit a sensor area at the same time that a player is standing or running over it. The sensor areas do not need to be much smaller than 6″ in length or width since it is nearly impossible for a foot to be within a 6″×6″ area and for a ball to bounce in the same area. At the same time, It is desirable to have as few sensor areas as possible since more sensor areas means that the system will need more wires to conduct signals and more electronics to analyze them. Line calling systems made for professional use may have more of these subsections than those made for amateur or personal use.
There are several ways to reduce the number of IN sections 24 and OUT sections 26 needed by the system, and/or the number of signals generated by the sensors, although such reductions may also reduce accuracy—three such methods are described in more detail in following sections.
Sensor Structure
As described above, the sensors in a preferred embodiment of the invention may utilize force sensing resistor (FSR) technology, although the invention is not limited to this embodiment.
The enabling component of an FSR is, e.g., a semi-conductive force-sensitive resistive ink. Through a carefully controlled manufacturing process, the ink is created with a rough surface at the microscopic level such that when it is merely touching a conductor or another layer of resistive ink, the conductivity is poor. But when pressure is applied, the surface area of contact increases causing an increase in conductivity. Such inks are available from various manufacturers, and each has a different response curve to pressure. The ink chosen for this application should have a high standoff resistance (i.e., a poor conductivity when no pressure is applied), and sensitivity to forces in the range of forces produced by a players foot and the force of a tennis ball.
In a preferred embodiment, the FSR switch comprises an upper and lower flexible non-conductive plastic layer (such as Mylar™, a trade name for Polyester). The bottom layer (FIG. 3A) has a series of conductive traces printed on it using a process such as screen printing. The traces are printed in an interdigitated finger pattern 30 with one common trace 33 carrying a positive voltage along the middle, and a plurality of IN sections 37 formed along one side of the boundary line by groups of interdigitated traces, and a plurality of OUT sections 39 formed on the other side of the boundary line by more groups of interdigitated traces. All of the traces are run to one or more edges of the sensor line where they can be attached to sensing electronics via, e.g., IN/ OUT connectors 34, 36 through ribbon cables 40 respectively. FIG. 3H, described in more detail below, provides an exemplary layout of one possible embodiment.
The other (top) nonconductive plastic layer (FIG. 3B) is printed with semi-conductive resistive ink 44, 46 described above such that when the two plastic layers are attached, the resistive ink will face the interdigitated fingers 30 (the top layer, as shown in FIG. 3B, is flipped upside down before being glued to the bottom layer), with ink for the IN side 44 and the OUT side 46 respectively. It should be noted that the terms “glue” and “glued” are herein defined as shorthand nomenclature for any type of adhesive. Thus, when the resistive ink 44, 46 is merely touching the interdigitated fingers 30, the resistance between the interdigitated fingers 30 will be, e.g., several megaohms or more.
However, when a pressure such as a ball hitting the sensor or a foot stepping on the sensor 22 is applied, the resistance will drop approximately proportionally with pressure to a lower value around, e.g., 1 KΩ. The actual resistance values are not particularly relevant as long as there is a substantial and measurable difference between the force applied/no force applied conditions.
In another embodiment, the FSR can be made in the “through” configuration. In this configuration, the common trace is printed on one layer of nonconductive plastic, and the IN/OUT sensing areas are printed on the other layer. Then, either one or both of these layers is covered with semi-conductive resistive ink. This configuration works as well as the one above, but may be more expensive to manufacture since multiple layers of ink may be printed on top of each other.
FIGS. 3C through 3E illustrate a through layout of an exemplary multilayer sensor 22 in which the top layer comprises a top layer plastic sheet 220, both FSR ink/paint 222, and a conductor 224, and the bottom layer comprises a bottom layer plastic sheet 220′, both FSR ink/paint 222 and a conductor 224. A layer of glue 250 is provided between the layers and the wires 40″ are present along the outer portion.
FIG. 3H shows a wiring configuration in which the traces on the bottom layer cascade down to their respective sensor areas. The advantage of these cascading traces is that they are formed by a repeating pattern that is less costly to manufacture since the same pattern can be printed repeatedly for the majority of sensor areas of the sensor. Such a pattern could be printed repeatedly using a silk screening method, or a rotary printing method, and would avoiding the use of many different, or one long printing mask.
The top diagram of FIG. 3H illustrates a basic trace pattern print mask and a basic sensor plus trace pattern mask that can be repeated, in theory, indefinitely. The width permitted for the trace patterns would ultimately dictate the number of repetitions in a practical application. The middle diagram of FIG. 3H illustrates a sensor and trace pattern that is repeated four times. Accordingly, it is possible to design the mask according to such grouped sensor and trace patterns instead of individual ones. In the case of silk screening, such a pattern could be created on one large silk screen, thus four sensor areas would be created, each time the screen was printed. The bottom diagram of FIG. 3H illustrates an example of either the top pattern being repeated twelve times, or the middle pattern being repeated three times. This technique results in traces being printed that may be unused (i.e., those not connected to any of the sensor areas), but these unused traces are not problematic.
Other wiring configurations, which are generally known, may also be employed for either the interdigitated or thru embodiments of the device. These may include configurations in which sensor areas are located on an X/Y grid and can be scanned one row or column at a time, or more generally, a configuration in which sensors are grouped into clusters having separate common traces, and shared output traces, allowing the force values of the sensor areas in each cluster to be scanned independently of other clusters. Although such configurations may reduce the amount of wiring needed, extreme care must be taken when engineering such a configuration for use with a line calling system, because during footsteps, current may leak onto un-activated common traces, leading to false activations, missed ball bounces, or incorrect calls.
Several different conductive inks, such as silver, copper, aluminum or nickel conductive ink, may be used for the traces 30, interdigitated finger and/or conductive pads. The silver ink conducts the best, and is least likely to corrode, but is the most expensive. To reduce the cost of these inks and improve corrosion resistance, an ink that is carbon based, or is a mix of carbon and one of the above inks can be used. The ratio of carbon to silver/copper/aluminum/nickel ink will determine the final resistance. Using a cheaper and more corrosion resistant carbon ink, however, may greatly increase the resistance of the traces, since resistance is proportional to the length of the trace, and the traces are quite long. Thus, to achieve the best price vs. performance ratio, it may be possible to use a more conductive and less corrosion resistant ink such as silver ink for the traces, and less conductive, more corrosion resistive, and cheaper ink such as a silver and carbon mix on the sensor areas (which are comprised of interdigitated fingers 30 or pads).
Sensor Gluing
As illustrated in FIGS. 4A-D, the upper and lower layers should be glued securely all around the edges to prevent moisture from getting in. Furthermore, they should be glued in the pressure sensing area of the sensor 22 to prevent the top of the sensor from moving up and down and thereby fatiguing the sensor 22 so the sensor 22 will stand up to the forces created when players run and slide over it and when the ball bounces on it. The glue 50 should be applied in a pattern that will accomplish this goal while maximizing the sensing area and minimizing the non-sensing area. The glue or spacer should have a layout that will retain sufficient strength and should generally be evenly distributed around the IN/OUT boundary.
Furthermore, as little glue 50 as possible should be applied around the boundary between the IN 24 and OUT 26 areas, since glue 50 near the boundary may reduce the accuracy and consistency of the sensor 22. In a preferred embodiment, the pattern consists of fingers 51 (FIG. 4A) that go toward the center of the sensor 22, with an open “channel” around the sensor boundary to maximize the resolution at the IN/OUT boundary to accurately call close balls.
A further advantage of this layout is that it allows the air that is trapped between the upper and lower layer of the sensor to move around freely preventing areas between the two layers from having a high or a low air pressure. Several other possible layouts for the glue are also shown. FIG. 4B illustrates the use of squares 52 with air passages 53 to promote air movement. FIG. 4C illustrates the use of dots 55 for the glue pattern, and FIG. 4D illustrates the use of crosses for the glue pattern.
Note that these layouts can also be used for the middle nonconductive layers of sensors 22 employing membrane switches to increase their accuracy. The glue 50 can either be printed onto the sensor 22 (the preferred glue is a special UV activated glue), or a thin sheet of pressure sensitive adhesive which is cut to the appropriate shape can be used.
Force Sensing Rubber
In yet another embodiment of the invention, force sensing rubber may be used to create the sensor 22 by replacing the semi-conductive resistive ink with force sensing rubber in either of the above embodiments.
Force sensing rubber is made by mixing small conductive particles into rubber before it dries, and is similar to semi-conductive resistor ink in that its resistance decreases as force is applied. However, instead of using a surface effect, the effect is one that occurs within the volume of the rubber. Because the surface is not involved, this configuration may be more resilient to wear and environmental conditions, but may also create a thicker sensor, since the rubber must have a certain minimum thickness in order to work effectively. To attach the force sensing rubber between the upper and lower sensor layers, layouts of glue such as those described above and illustrated in FIGS. 4A-D may be used. An alternative to this is to use conductive rubber glue which may cover the entire surface of the rubber without reducing the sensing area of the sensor.
Sensor Interface
FIGS. 5A-E illustrate various embodiments of the sensor interface. The sensors 22 can e.g. (FIG. 5A), be wired to the control box 102 with flat flex cables 40′ utilizing a standard connector 34, 36.
The wiring, e.g., ribbon cable 40 (FIG. 5B), can be directly soldered to the sensors 22 or connected to the sensors 22 using standard flex cable connectors 34, 36. Another option (FIG. 5C) is to have a small circuit board 140 between the wiring 40, 40′ and sensor 22. This circuit board 140 may have electronics, such as an AND converter 150 and microprocessor(s) 160 to analyze the signals closer to their source, and to send a digital signal describing what has been picked up, thus lowering the number of wires that need to be run to the control box 102. The connections can be protected from moisture by covering them with a liberal layer of plastic, latex, or silicone sealant and/or glue 60 or some other form of waterproof enclosure 60′.
In one embodiment illustrated in FIG. 5D, the electronics 150 and microprocessors 160 may be embedded directly into the sensors 22 using flex-circuit technology. Each microprocessor 160 may process the signals from one or more sensing areas. The electronics 150 and microprocessors 160 may be embedded at one or more ends of a strip of sensors 22 (FIG. 5D), or they can be evenly spread along the length of the sensor (FIG. 5E). The microprocessors 160 can be connected with a common communication and power bus 40″ composed of just a few wires (for example: two for power, one for communications). The major advantage of this scheme is simplified wiring, although great care would have to be taken to ensure that the electronics are reliable, especially in areas where they are installed under court paint and might be run over by players.
Installation
The sensors 22 may be transported from the manufacturing/distribution site to the installation site easily. Because the sensors 22 as described above are thin and flexible, and comprise long strips, they can be easily folded or rolled up for shipping.
FIG. 6 illustrates an exemplary sensor 22 installation. Once the sensors 22 are delivered to the site, they are glued 72 to the top of concrete, asphalt, or existing court paint 70 with a glue that is resilient, bonds well to plastic and to the substrate, can tolerate a wide range of weather conditions, and can dry or stick without the presence of air (since the plastic will seal off air). Plastic epoxy, and some industrial strength rubber cements (like E6000) can work well for this purpose, as well as possibly pressure sensitive acrylic adhesive.
Thin wiring (40, 40′, 40″, FIGS. 5A-E) is run over to the control box 102, and glued 72 in place to the court 70. Typically, the court paint will be at least ⅛″ high—thick enough to completely cover all the wiring 40, 40′, 40″. If it is not, a groove can be cut in the surface of the court to make a channel for the wires 40, 40′, 40″. In order to make the entire area of the lines sensitive, the sensors 22 can overlap each other in the areas where the boundary lines meet, as illustrated in FIGS. 1, 2A-C, and discussed above. Because the sensors 22 are so thin, this can be done without leaving a visible bump on the surface of the court, and without affecting sensor performance. A sensor 22 can be constructed so that a portion of the wires is formed by an extended tail on the sensors 22 themselves (e.g., printed or formed on the same plastic strip).
Once the sensors 22 are glued 72 to the court, they must be treated in a way that will permit the tennis court paint 79, or cushioning system 76 to stick securely to the top and/or sides of the sensor 22. One way to do this is to first paint the lines with a primer such as GAC 2000 or CalproCorp Hi-Grab. A further way to improve adhesion of the tennis court paint 79 or cushioning system 76 is to first glue a thin mesh 74, or scrim to the top of the sensor 22 before painting with tennis court paint 79 or cushioning system 76. Another way to accomplish this is to manufacture the sensors 22 with a top layer that has a sufficiently rough upper surface to allow tennis court paint 79 to stick securely.
The wires 40, 40′, 40″ from the sensors 22 are run to the control box 102 that has the necessary electronics to process signals from the sensors 22. The control box 102 can be attached securely to one of the net 12 posts in order to fix it in place, and so that it will not interfere with the game. Another option is to locate the control box 102 at the side of the court 10, space permitting. An indicator light might be included as part of the control box 102, or one or more separate indicator lights can be attached to the net posts. The indicator light(s) can also be encased in a clear plastic tube or a clear flexible cable and placed on the court surface right below the net. Other types of indicators, including those based on sound/tones, could be used as well.
Control Box, and Communication
Referring to FIGS. 7A, B, the control box 102 collects signals from the sensors on the court, interprets them and notifies players whether a ball was IN or OUT when it hits a boundary line 14. The control box 102 is optimally designed solid enough withstand the impact of a tennis ball traveling at high speed. As noted previously, the control box 102 can be mounted, e.g., via straps 112, to a net post.
Although in the preferred embodiment, all of the analysis is performed by one control boxes, additional control boxes may be provided to serve either as backups, or to display information at different locations on the court. Furthermore, the functionality of the control box may be split among multiple control boxes on the court (for instance, one for each side of the court, or one for each line on the court). These control boxes may have mechanisms to communicate with each other and exchange information regarding line calls.
The control box 102 can be designed to minimize power consumption for its operation, and can be designed to go into a low power state automatically if it is accidentally left on. In one embodiment, it can be powered by a high capacity battery. Another embodiment might include utilizing a rechargeable battery and a small solar cell 126 that would charge the battery using ambient light. An additional embodiment includes supplying power from an external source, although a power cord on the court 10 could present a safety hazard and a nuisance unless it were embedded under the paint.
The control box 102 may comprise a number of switches 124 that can serve various functions. These switches 124 may be configured on a control panel 120, which could also include the solar cell 126 and a status or indicator display 122 that may be implemented, e.g., with LCDs or any other display technology. Possible switches 124 present may include an on/off switch for power, a switch to select between singles and doubles play, and switches for enabling or disabling communication via sound, lights, or network communications.
The control box 102 may also have an indicator light 122 to signal when it is working properly, when power is low, and when a problem is detected. The network communication may be wired or wireless, and it would allow umpires to log in from a laptop or other portable computer to receive IN/OUT decisions as well as more detailed information, such as where the ball bounce occurred, and details about the status of the line calling system. The information may also be distributed in real-time to TV stations broadcasting the event, sport web sites, and to electronic billboards at the stadium.
For unrefereed games, audio and visual indications may be used to notify the players when a ball has bounced IN or OUT, although these can be kept enabled during refereed games as well. When audio is enabled, the system may beep, or say “out” immediately after a ball bounces out via speakers 106. When a ball bounces IN, it is optimal for the system to not make any sound to keep from distracting the players, although some other signal can easily be implemented. When enabled, a light or lights 104 can also be used to indicate whether a ball was IN or OUT. Preferably, the lights 104 will be composed of bright LEDs, since LEDs are both durable and energy efficient. For example, a blinking red light may indicate OUT while a steady green light can indicate IN (when a ball hits a court line).
It should be noted that the areas of the courts that are in play vary depending on whether the game is singles or doubles and on whether the players are serving and to which court they are serving. Because the control box 102 is configured via, e.g., a switch 124 for singles or doubles play, it is simple for the control box 102 to know whether to monitor the outer or inner sidelines of the court 10.
However, the control box 102 also needs a way of knowing when players are serving, and which court the players are serving to. One solution to this problem is to have an umpire or a scorekeeper, using their networked laptop, or a remote control to explicitly tell the system when players are serving and into which court. The drawback to this approach is that it may be tedious, and it can not be used for unrefereed games. In a networked configuration, any of the user interface functions (i.e., display 122, switches 124) can be performed remotely.
Another approach which does not require user intervention, is to simply use a screen to display the place on the court where the ball bounced IN or OUT and let the players decide if it hit a service court that was in play. An audio beep can be produced when the ball lands outside the sidelines or baseline of the court, since in that case, it is also definitely outside of the service boxes. However, no beep will be made when the ball bounces outside the service boxes, because this can happen during the regular course of the game also. In this case, the players would have to look at the displays to see the result, although embodiments that do not use a screen are still feasible where cost or space restrictions are issues.
In a preferred embodiment, rather than having an entire screen, three lights may be used to indicate whether the ball bounced IN or OUT. A central light will turn red when a ball is outside the sidelines or baseline, the light on the player's left will turn red when a ball is out on either of the service courts to the players left, and the light on a players right will turn red when a ball is out on either of the service courts to the players right.
A further way to improve upon the indicating system above is to have intelligent software of a serve condition detection system, possibly associated with the control box 102, that attempts to determine whether the players are serving and which court they are serving to. This can work because when a player is preparing to serve, he or she typically stands directly behind the baseline 14.2 for an extended period of time. Since some sensors 22 are mounted in the region behind the baseline 14.2, the control box 102 can interpret a steady pressure on a region behind the baseline 14.2 as an indication that a player is about to serve. Furthermore, this software can tell whether the player is serving to the left or the right service court based on whether they are standing on the left or the right side of the service line 14.1. Based on this, the system will determine when a player is serving and into which box.
By automatically determining whether a player is serving and into which court they are serving, the system can activate the service lines in that court while keeping the baselines and sidelines active, thereby making the right call with very high probability, and only making a wrong “OUT” call in cases where it is very easy for the player to ignore the bad call (since the ball hits the service area during a volley).
By knowing this, the system can also be made to indicate, e.g., via a beep, when a player's serve lands outside the service box. Note that the system should continue to indicate OUT if a ball bounces outside of any sideline 14.3 or the baseline 14.2. This doesn't cause any conflict since a ball that is outside of a sideline 14.3 or baseline 14.2 is also outside the service box. However, it is possible that the player may stand near or on the area of the baseline 14.2 during a point, and that the system will detect this as a serve condition. Although this could occur, it is highly unlikely that it would cause any problems because it is very rare for a player to hit a shot from the baseline 14.2 that bounces on the service line 14.1 (unless they are serving).
Furthermore, if this did occur, the players could ignore the beep or other signal since it would be obvious that the ball is still in. Because this approach makes assumptions about where the player stands when they serve, it may not work for players that stand far away from the baseline before they serve. This feature may be implemented as an optional feature called “Auto Serve Detect” that can be disabled with a switch 124.
It is also possible that the players themselves could have devices (e.g., wireless controls) for remotely communicating with the control box 102 and for indicating the status of who is serving or providing any other status or control signal to the control box 102.
Control box electronics and software
The control box 102 comprises the hardware and software used to received and process information from the sensors 22, and is illustrated in FIGS. 8A, B. It comprises a plurality of analog to digital converters 172 and their associated circuitry which will convert the analog signals from the sensors 22 into digital values. The values will be fed into one or more microprocessors 170 which analyze the signals to make the line calls.
Because ball bounces occur within a 5 ms time-span, the signals from the sensors should be read at a rate of approximately 1000 to 4000 times per second to capture the characteristic waveform produced by the ball bounce.
The analog to digital converters 172 may be integrated into the microprocessor 170, or may be on separate chips. Each of these chips 172 may also have its own processor that would allow some amount of preprocessing to be done on the data, thereby reducing the computational load on the main microprocessor 170. As described in the above section on sensor construction, these chips 150 may also reside outside the control box 102, and may be attached to the ends of the sensors 22, or may be integrated into the sensors 22 using flex-circuit technology. Additionally, the control box 102 may contain the necessary components to supply power V_S, switches 124 to allow user configuration, a speaker 106 to produce sound, connections for lights or integrated lights 104, and components 108 to allow communication over a network.
The sensors 22 may be connected to inputs of the analog to digital converter 172 in a simple voltage-divider configuration, as illustrated in FIG. 8A. In an alternate embodiment (FIG. 8B), operational amplifiers 174 connected in known configurations may be used to amplify the signal from the sensors 22 before passing it into the analog to digital converters 172.
After conversion to a digital value by the analog to digital converter, the signal and information about the configuration of the circuitry used to feed the analog to digital converter may be used to calculate the resistance of the sensor. The conductance of the sensor may then be calculated by taking the multiplicative inverse of the resistance. The conductance can then substitute for the raw signals as the input to the rest of the algorithm. The advantage of using the conductance is that it is approximately proportional to the force applied to the sensor.
Furthermore, when two forces are applied simultaneously to a sensor, the conductance is the same as if the two forces were applied separately and then the conductances from the two were added. This guarantees that even if there is a non-zero base-level signal, the conductance will increase by the same amount and at the same rate when a ball bounces as if there were no base-level signal. Furthermore, when it is necessary to determine the total force exerted on two or more sensor areas, the conductances of the two areas can simply be added together. This is typically not true for the raw values obtained from the analog to digital converters.
FIG. 9 is a block diagram illustrating the system architecture. After the analog to digital conversion 172, a software algorithm loaded onto the microprocessor(s) and/or hardware circuitry will monitor the tennis lines. This algorithm will do the following task in the background: it establishes a base-level signal 184 that the sensor produces when it is not activated. When the system is first turned on, it sets the signals produced by each sensor or segment as the base-level. The base-levels may be determined for groups of sensors or sensor sections or may be determined on a per sensor basis. Because a problem could possibly occur during game-play, the system may continue to adjust the base-level during the game. It will do this by storing the values of the signal from the last several minutes of operation at predetermined intervals, throwing out the largest of those values (which occur during ball bounces, footsteps, and racquet hits), and then averaging the remaining values.
If the base-level signal is too high (above a pre-set threshold), this may indicate a short in the wiring or significant physical damage to the court, such as a crack or large bubble in the paint. If the signal is too low (below a pre-set threshold), it may indicate a break in the wiring (a resistor 38 with a large resistance value of several MΩ can be placed at each sensor pad to ensure a low signal as long as the wiring is okay, as described in more detail below). If the signal oscillates for a long time at a high frequency, it may indicate strong electrical interference, a poor connection, or other problem. If an error is detected in one section of the sensor 22, the system can notify the player/umpire of the error in that section and continue working in the rest. The health monitoring system 188 is described in more detail below.
Although the system is designed to have an extremely low failure rate, this may still be extremely useful in the unlikely case that a sensor 22 does malfunction. For instance, if, for any reason, a part of the doubles sideline stops working, the players may continue to play a singles game on the court. If a certain section of the court that is in play stops working, an extra line judge may be called in to monitor that part of the line.
As noted above, in order to detect possible sensor 22 failure, a sensor health monitoring system 188 may be employed. This system 188 compares, via an internal comparator, signal values or ranges with known values or ranges for a properly operating sensor and/or known values or ranges for an improperly operating sensor (these values possibly being stored in a table), and, based on the result of the comparison calculation, can send a signal to an error notification system 190 that can then further relay such error information to, e.g., an indicator such as an error light 122 or to another deice via the network 108.
One mechanism that may be employed for the detection of sensor problems is illustrated in FIGS. 8A, B in which a leakage resistor 38 is employed. The resistor 38 should be of a sufficiently high resistance value, e.g., 3MΩ, that any current drain through it during normal operation is minimal and thus does not represent a significant power loss for the sensor. However, the minimal current through such a resistor 38 should be adequate so that the absence of this current indicates a connection problem with the sensor. The health monitoring circuitry 188 would detect the presence or absence of the current through the leakage resistor 38 in its comparison. The circuitry 188 could further detect the presence of a short circuit or other types of abnormal signal levels, such as signal levels that persistently remain above certain threshold levels or that indicate other abnormal signals. This circuitry 188 could also be set into a programmed test mode in which the sensor sections are activated for testing and signal an error if a signal indicating an event such as a ball bounce, footstep, or racquet hit is not received within some predetermined period of time.
In the foreground, the system will do the following: first, a game state monitor 180 will determine which set of lines it should monitor based on whether a singles or doubles game is being played based on switch 124 or other parameter settings, and based on which players are currently serving and into which box they are standing. The game state monitor communicates the current game state 196 with the event integrator 186. As described above, this information could be indicated to the control box over the network by an umpire or scorekeeper, or it could be determined automatically based on footstep events that are generated when a player is about to serve from the baseline. The event integrator 186 provides information 198 related to IN/OUT calls and the detection of serve conditions to the game state monitor 180, and can be used to ignore certain IN/OUT determinations from sensor sections that are inconsistent or improbable with a known state of play. The event integrator can be used to discriminate between two recorded ball bounces from different areas of the court.
For each subsection of the court that is in play, the game state monitor 180 runs a separate instance of an algorithm hereafter referred to as a classifier 182 which detects an event and classifies it into contact by a ball, contact by a foot, contact by a tennis racquet or some other type of event. In the case of contact by a ball, the classifier then determines whether the ball was IN or OUT. Each classifier then feeds all of its events into an event integrator algorithm 186 which produces a final decision and an indication that uses sound via the speaker 106, light 104 and/or a message over the network 108. The game state monitor 180 may also send signals 194 to the classifiers 182 that indicate whether they should be enabled or disabled, based on the game state. The event integrator 186 could further incorporate the serve condition detection system.
Classification
The first part of the algorithm is the detection of an event and its classification into a ball bounce, footstep or tennis racquet hit event. As discussed above, according to an embodiment of the invention, one instance of this algorithm 182 is run for each subsection of the court lines. If sensors 22 having IN and OUT portions are used, the combined signals of the sensor portions should be utilized by the instance to make the determination. According to this implementation, if a player is standing or running over one subsection of the sensor 22, a separate classifier instance 182 will still be able to detect a ball bounce in another subsection.
The classification of ball bounce, foot step and tennis racquet hit is performed on the signals produced by the sensor 22. If a signal is present from an IN portion of a sensor (when the sensor is divided into IN and OUT portions), then a signal classified as a ball bounce in the OUT portion will be deemed to be IN bounds, since any part of the ball touching the line should be considered IN, according to the rules. As noted previously, however, it is possible to utilize sensors 22 only on the OUT side of the boundary line, in which case a somewhat less accurate decision will be made solely on the basis of the OUT sensor signal.
Any number of schemes may be utilized in performing the classification either individually or in any combination. These include:

- signal duration of activation at a given signal threshold level (or multiple threshold levels);
- signal amplitude;
- determination of number of peaks;
- signal frequency analysis (e.g., using Fast Fourier Transforms or similar analytical techniques based on spectral characteristics); and
- signal correlation or matching.

When viewed on an oscilloscope, the sensor 22 output signals all of these events typically have different characteristic shapes. As illustrated by the graphs in FIGS. 10A-D a ball bounce looks like a single, smooth, rounded peak, and typically lasts between 4 and 5 milliseconds and always seems to be within a range of 3.5 and 5.5 milliseconds no mater what speed or spin the ball has, provided the measurement threshold has been set at an appropriate level. The appropriate measurement threshold(s) can be manually set based on any form of calibration scheme, or may be defined in some relationship to other signals in the system, which could include the base-line signal.
FIGS. 10A, B and C illustrate respectively a light, medium and hard ball impact. FIG. 10D illustrates the asymmetry present when the ball impact occurs at a sharp angle.
The height of the peak increases with the vertical speed of the ball. Because a ball will always fall from at least net height, which is approximately 42 inches, there is a certain minimum vertical speed with which it will hit the sensor, and thus, it will produce a peak with a minimum height, which can be determined experimentally. Also, the maximum practical velocity of a tennis ball when it hits a court can range between 100 and 150 mph, and therefore a peak of maximum height can be determined experimentally as well. The actual amplitudes obtained for signals will be based on a calibration, and these may be adjusted based on the type of sensor used, variations in drive circuitry, court conditions, etc. It is also possible to schedule calibrations on a periodic basis to ensure the integrity of the system. Such a calibration could utilize firing tennis balls at a known speed on the sensors and having persons of varying weights stand or jump on the sensors.
As illustrated in FIGS. 10E and 10F a footstep produces a signal that is much longer than that of a ball bounce. These are typically greater than 50 ms in duration, and can be much longer. When a player kicks the line, the signal starts with a sharp peak and then settles to a lower value which lasts until the foot is released. When a player jumps or runs over a line, the signal may also have a peak at the beginning followed by a large hump. We have not observed any way in which a player can produce a peak of the same duration and height as a tennis ball with their foot, given the very different nature of the forces and inertia involved. In FIGS. 10E and 10F, the duration of the footstep is considerably longer than the duration of the ball bounce when the threshold is set at the level illustrated in FIG. 10A-D.
FIGS. 10G-I illustrates curves associated with a racquet impact. Due to their rigid construction, the hit of a tennis racquet usually produces one or more peaks which are often jagged, or shorter in duration that that of a tennis ball bounce. On extremely rare occasions, a tennis racquet may produce a single peak which may look like the peak produced by a tennis ball.
The scrape of a tennis racquet across the court, as illustrated by the graph shown in FIG. 10I, typically produces peaks that are much smaller in height than the smallest peaks produced by a ball, and would therefore not generally be confused with a ball bounce. The issue of false activation of the sensors by the tennis racquet is addressed below.
FIG. 10J illustrates a resultant curve when a simultaneous ball bounce and footstep occur. As discussed in more detail below, dual thresholds (see FIGS. 10H and 10J) may be utilized for further discrimination.
Because a ball that is traveling at an angle may skid across the boundary between two sensors as it bounces, it is possible that each sensor in isolation will register a peak that is shorter or smaller than that of a typical ball bounce. For this reason, the signals from adjacent sensors may be combined in software to create larger “virtual” sensor areas, which will register the full ball bounce. Doing this is especially important at the boundary between IN sensor portions and OUT sensor portions, when these are used, since a ball that bounces on the boundary should be detected reliably and scored as being IN. As explained before, if conductance is used as the signal to the algorithms, then the conductances of the two sensor areas can simply be added to determine the combined signal on the “virtual” sensor area. However, if raw input signals are added ignorantly, then the characteristic shape of the summed signal may not look like that of a ball, and may instead have multiple peaks.
Each of the signals, including the ball bounce has a “tail” region. This is due to hysterisis—in other words, FSR sensors turn on very quickly when activated, but turn off more slowly. This can also be due to vibration of the court surface near the point of impact. Also, the signals may have a small level of electrical noise due to interference from outside devices. To get an accurate timing of the peak, the width is measured at a threshold level which is a pre-determined margin above the base-level signal. This margin should be high enough to avoid detecting signals caused by hysterisis, vibration and electrical noise. However, the threshold should not be too high, since a high threshold may produce inaccurate timings of ball bounces and racquet hits.
To distinguish a tennis ball bounce from other events, the classification algorithm can measure the width of a peak in milliseconds and its height. If the peak is approximately 3.5 to 5.5 ms in width, and above a threshold height, determined by the lightest bounce a ball can take from the height of the net, it should be classified as a ball. If it is longer than approximately 5.5 ms in duration, it should be classified as a foot, and if it is shorter than approximately 3.5 ms in duration or smaller than the threshold height, it should be classified as a racquet hit. As a further criteria, the classifier may check that the signal has only a single peak. In other words, it should increase approximately monotonically from the start until its maximum value and then decreases approximately monotonically until its end. Because there might be some noise in the signal, only peaks bigger in size than the noise should be detected. More sophisticated algorithms which match the profiles of the peaks to pre-recorded profiles may also be employed to better distinguish between ball bounces and racquet hits, which in rare circumstances may have heights and durations similar to those produced by ball bounces, but are distinguishable due to their jagged shape.
As discussed above, it is also possible in a further embodiment of the invention to utilize multiple threshold levels as illustrated in FIGS. 10H and 10J. The use of multiple threshold levels permits additional discrimination for the various sensor signals based on the characteristic shapes of the curves. In performing such an analysis, a low threshold (a small percentage larger than the base-level signal) may be used to find the start time at which the sensor was activated and the end time at which it was deactivated. This entire period is treated as a single event. Then, the width of peaks above a higher threshold within that period are analyzed. This second threshold may be set through experimentation just high enough to reliably detect a ball dropping from net height. If more than one peak is found, or if there is a wide peak greater than about 5.5 ms, then this may be indicative of a footstep event. If the peak is shorter than, e.g., about 3.5 ms, the event may be classified as a racquet hit. Any number of thresholds may be utilized in order to better discriminate between various impact events that may occur with respect to the sensors.
As noted above, the frequency characteristics can be utilized to discriminate between and classify events. These may be calculated by means such as a Fast Fourier Transform (FFT) or a wavelet transform. The racquet hits generally comprise a number of high-frequency components not found in either the footstep or ball bounce. The ball bounce will have a large spectral spike at around 200 Hz (⅕ ms) and a general absence of the frequency components above and below this. The footstep will, as a rule, have large lower frequency components.
Furthermore, some form of a correlator may be used to compare the stored waveforms with those measured. In this arrangement, a digitized sampled sensor signal is compared with various stored characteristic curves according to some sampled interval, and a correlation coefficient is established for the sampled sensor signal for each stored characteristic. The stored characteristic having the highest correlation coefficient is deemed to indicate the type of event that occurred.
In the preferred embodiment of the classifier, a voltage divider configuration as shown in FIG. 8A is used, and the signal, is fed to an A/D converter which samples the signal at a rate of 4000 hertz. The signal is then converted to a conductance value using the formula 1/(Rd*((VadNs)−1)). The conductance value from the last 20 ms of signal are stored in a circular buffer in memory. Although a plot of the conductance might look similar to the plot of the raw output of the sensor, it is scaled differently, and produces more accurate results. Each time a new conductance value becomes available, the oldest one in the circular buffer is removed and the new value is inserted. Then, the classification algorithm analyzes the 20 ms of signal stored in the buffer to determine if a ball bounce occurred there. The classification algorithm first checks that the signal was below the base-level adjusted threshold for the first two milliseconds, above the threshold for the next 3.5 to 5.5 milliseconds, and below the threshold for the remaining time. It then checks that the maximum value of the signal was above the minimum peak value for a ball bounce. Finally, it checks that during those 3.5 to 5.5 ms, the signal had only one peak by verifying that, ignoring small noise, the signal increases for approximately the first half of that period and decreases afterwards. If all of these conditions are satisfied, then the signal is classified as a ball bounce. For detecting footsteps, the amount of time that the signal spends above the threshold is measured. If the signal is above the threshold for over a sufficient length of time, such as 50 ms, the signal is classified as a footstep. If the signal is classified neither as a ball bounce or a footstep, it is classified as a racquet hit.
Because, the timings, amplitudes of the signals, characteristic shapes, peak characteristics and spectral characteristics may change slightly depending on the type of balls used, the speed at which they impact, the type of court surface, environmental factors, and changes in the way tennis is played, a facility should be included in the system to allow calibration and fine-tuning of these parameters and stored characteristics during installation and maintenance. Furthermore, it should be possible to modify any and all parts of the algorithms presented and stored characteristics via a software update.
Once an event is classified as a ball bounce, the system must then determine if the ball was IN or OUT. In tennis, a ball is considered IN if it touches the white of the boundary line 14, even slightly. Thus, if the ball bounce was detected on an IN section, the ball counts as definitely IN. However, if the ball bounce occurs on an OUT section which is adjacent to an IN section, to make the determination, the signal produced by adjacent IN section of the segment is analyzed within a short (around 2 ms) window of the event. If at any time within that window, the signal increased past the lower threshold, the ball is considered IN.
The algorithms presented above, if properly implemented and adjusted, work with extremely high accuracy for discriminating between ball bounces, footsteps, and tennis racquet hits. A foot step is never confused with the bounce of a ball. However, we found that in extremely rare circumstances, a tennis racquet bouncing off the court can produce a signal that is indistinguishable from a ball bounce. In particular we saw this happen when the racquet was dropped, head-first onto the sensor, and allowed to bounce. Depending on the particular implementation of the classifier, there may be other situations in which it could, in rare circumstances, detect a tennis racquet hitting or scraping the court as a ball bounce. Such situations and a few others are resolved by the integrator 186, which is presented below.
Previously it was noted that there are three ways to reduce the number of IN sections 24 and OUT sections 26 needed by the system, and/or the number of signals produced by the sensor. The first is to analyze a waveform produced be each sensor 22 to find the approximately 5 ms long peaks created by ball bounces even while they are being “masked” by a footstep. However, as noted above, peaks at the beginning of a footstep (approximately the first 25 ms) may look like a ball bounce and would have to be ignored. Peaks created by a ball bounce can be detected by looking for a large increase in the signal (above a pre-determined threshold) within 2.5 ms, followed by a similarly sized decrease in the signal in the next 2.5 ms. This works best when using the conductance of the sensor as the input signal to the classifier, since when looking at the conductance, the signal will be exactly the same as if we had measured the signal from the ball bounce and from the foot step separately and added them together. This method allows one to decrease the number of subsections on the sensor, while still being able to detect a ball bounce and footstep that occur on the same subsection. Additional processing can be provided to further refine the criteria for distinction.
The second way to accomplish the goals described above is to chain two or more sensor subsections with small resistors R_ch(FIG. 11), and to compare the output from the subsections at the two ends. By computing I1+I2 (FIG. 12) we can thus get an idea of the total force applied over the chained sensors. By computing I1*Length/(I1+I2), the average position of the force applied to the sensors 22 can be calculated. With such a setup, a ball bounce would cause a peak in the total force signal, plus a sudden shift in the average position of the force that would last as long as the ball bounce (typically around 5 ms). FIG. 13 Illustrates the plot of I1/(I1+I2) that could potentially be used to draw out the ball bounce while eliminating the footstep.
A third way to reduce the number of signals, described previously, is to connect the sensor areas so that they may be scanned in groups. Such configurations involve having the sensor areas on an X/Y grid or having the sensor areas share separate common traces. Care should be taken in the engineering of such approaches to avoid problems associated with current leaking to un-activated common trace.
Integrator
Returning to FIG. 9, all of the events 192 from each of the independent classifiers 182 are sent to the integrator 186, along with the time of the event, and the location on the court where it occurred. The integrator 186 listens to all of the events and sorts out what happened based on those events. The integrator 186 may be configured to only receive events from subsections of the court that are currently being monitored, since those are the sections that were enabled by the game state monitor.
In the case that a single BALL IN or BALL OUT event 192 is passed from one of the classifiers 186, the job of the integrator 186 is very simple: it simply outputs that event to all of the enabled notification devices (sound 106, light 104, or network 108). However, when multiple BALL IN/BALL OUT events 192 occur within a very short time, the integrator 186 has to decide what notification to produce. If two BALL IN/BALL OUT events 192 happen simultaneously, or within a small (e.g., 2 ms) window of each other on adjacent subsections, this typically indicates that a ball landed on the boundary of two subsections. In this case, if either of the events 192 were BALL IN, BALL IN should be reported. Otherwise, BALL OUT should be reported.
If the events occur simultaneously on non-adjacent subsections on the same side of the court 10, the multiple events can only have happened because a ball bounce occurred at the same time as a tennis racquet hit (which was mistakenly classified by a classifier as a ball bounce). In such a situation, the event that occurred on the subsection closer to the front of the net will be used to produce a notification. This is because balls typically bounce in front of players. In the case that both of these events happen on the baseline, the event that happens closer to the outside of the baseline will be used to produce a notification.
If two or more BALL IN/BALL OUT events occur on one side of the court within a short time of each other (shorter than the time it takes for a ball to be returned to the same side of the court) the events could be produced by a ball that takes multiple bounces (such as during a drop shot), or one of the events could be produced by a tennis racquet hit (which was mistakenly classified by a classifier as a ball bounce). In such a case, the event that was the first to happen will produce the notification, and the second event will be ignored. This works because by carefully analyzing tennis play, it can observed that when a tennis racquet hits the ground, it usually occurs after the ball has already bounced. Furthermore, in the event that the racquet hit and ball bounce happen almost simultaneously in the same or adjacent subsections of the sensor, the result should still be correct. The reason is that if the ball hits the IN section and the racquet hits the OUT section, the ball will be counted as IN, which is correct. In case the ball hits the OUT section, the player will usually also hit the OUT section as they are reaching for it. The only situation in which a wrong call will be made is one in which the player hits an IN section while the ball hits an OUT section. This is very unlikely since players typically swing their racquets toward the ball from the OUT side of the court to the IN side.
This scheme gives the most reasonable guess given the nature of tennis and the inputs that are received, and will produce the correct notification in almost any possible situation that may occur. However, these algorithms could be modified, based on empirical data collected. Furthermore, the system may allow players and or umpires to review calls, and to see which events actually happened around the time of the call. Additionally, players and or umpires may have the option to disable the automatic integration of events, and receive notification about all the events as they detected.
Accuracy
The inventive system is designed for high accuracy and consistency. Accuracy is achieved by a combination of different factors. First, FSR based sensors 22 can be made much more sensitive than membrane switches without having false activations and can detect even a light force on the surface of the court. Unlike other sensors, such as piezoelectric vibration sensors, they will not be activated by vibrations that travel through the concrete. Furthermore the analog signals produced by the FSR based sensors can be analyzed using the methods described to distinguish between ball bounces, foot steps and tennis racquet hits much more accurately than could be done with previous approaches, virtually eliminating bad calls due to misclassification of events.
Second, accuracy comes from the way a ball 80 deforms when it bounces on a court. Experiments and computer simulations have shown that the bottom of the ball will actually inflect upwards during a bounce, such that the majority of the force will be exerted by the edge of the ball. Thus, if the edge of the ball touches a boundary line, the force will be sufficient to be detected by the IN sensor 24 under that line. These sensors 22 are consistent because they are equally sensitive on both sides of the boundary line 14 along their entire length on the court 10. High-speed camera experiments indicated that these sensors 22 had an accuracy better than 4 mm when balls were bounced from above.
The present system presents a viable embedded system that succeeds where others have failed. Although this system has been described in terms of a tennis court embodiment/implementation, there is nothing that inherently limits the principles of the invention to this application. This system, or parts thereof, may be applied for various other games including volleyball, badminton, etc. Any part of the settings utilized in the system are configurable and do not have to be hard-coded. This system can also be used in conjunction with a camera based system or other ball tracking technology (which may employ sound, radar, laser, or other means) to improve the accuracy of these system which do not “feel” the impact point, or to provide two independent opinions.
The term “ball” as used herein can be understood to include any form of game projectile capable of impacting a ground-based sensor, with the appropriate predetermined signal signatures being determined beforehand.
For the purposes of promoting an understanding of the principles of the invention, reference has been made to the preferred embodiments illustrated in the drawings, and specific language has been used to describe these embodiments. However, no limitation of the scope of the invention is intended by this specific language, and the invention should be construed to encompass all embodiments that would normally occur to one of ordinary skill in the art.
The present invention may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of hardware and/or software components configured to perform the specified functions. For example, the present invention may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, where the elements of the present invention are implemented using software programming or software elements the invention may be implemented with any programming or scripting language such as C, C++, Java, assembler, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Furthermore, the present invention could employ any number of conventional techniques for electronics configuration, signal processing and/or control, data processing and the like.
The particular implementations shown and described herein are illustrative examples of the invention and are not intended to otherwise limit the scope of the invention in any way. For the sake of brevity, conventional electronics, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail. Furthermore, the connecting lines, or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. It should be noted that many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the invention unless the element is specifically described as “essential” or “critical”. Numerous modifications and adaptations will be readily apparent to those skilled in this art without departing from the spirit and scope of the present invention.

REFERENCE CHARACTERS

10 tennis court
12 net
14 boundary lines
14.1 service line
14.2 baseline
14.3 sideline
22 sensor
24 IN sensing area
26 OUT sensing area
26.1 left-side OUT sensing area
26.2 common OUT sensing area
26.3 right-side out sensing area
28 glue area (non sensing)
30 interdigitated fingers
32 protective border
33 common trace
34 in connector
36 out connector
37 IN section interdigitated traces
38 leakage resistor
39 OUT section interdigitated traces
40 ribbon cable
40′ flex cables
40″ wire (power, communication bus)
44 FSR ink for IN side
46 FSR ink for OUT side
50 glue
51 glue finger pattern
52 squares in glue pattern
53 air passages in glue pattern
55 dot glue pattern
57 cross glue pattern
60 layer of sealant
60′ waterproof enclosure
70 concrete or asphalt
72 glue
74 mesh or other material
76 cushioning system
78 tennis court paint
79 line paint
80 tennis ball
82 outer contact edge of ball
84 inflected bottom of ball
102 control box
104 notification light(s)
106 speaker(s)
108 network jack
110 conduit for sensor/other wires
112 control box mounting straps
120 control panel
122 status LCD or indicator lights
124 switches
126 solar cell
140 attached circuit board
150 electronics, e.g., A/D converter
160 microprocessor
170 main control box microprocessor
172 analog-to-digital circuit
180 game state monitor
182 classifier algorithm
184 base-level signal calculation
186 event integrator
188 health monitoring
190 error notification
192 events
194 signals to enable/disable classifiers
196 current game state signal
198 IN/OUT calls, detection of serve condition
220 top layer plastic
220′ bottom layer plastic
222 FSR ink/paint
224 conductor
226 force sensing rubber
250 glue layer

Claims

1. A system for determining whether a ball bounce occurs on an inside of a boundary line or an outside of the boundary line, comprising:

a) a sensor or sensor sections comprising an output that provides either an analog signal or a multilevel digital signal related to a force present on the sensor or sensor section;

b) a signal analysis unit comprising:

b1) an input connected to the sensor signal output;

b2) circuitry or a memory containing predetermined information related to a footstep characteristic curve and a ball bounce characteristic curve, the characteristic curve information comprising at least one of duration of activation information, amplitude information, peak information, frequency information, or curve shape information;

b3) a classifier configured to classify a signal received from the sensor as at least one of: 1) a ball bounce by utilizing the ball bounce characteristic curve information, and 2) a footstep by utilizing the footstep characteristic curve information;

b4) the classifier comprising an event output at which a signal is output, the event output signal comprising at least an OUT event indicating a ball bounce classified outside of the boundary line.

2. The system as claimed in claim 1, wherein the sensor or sensor sections comprise:

an IN portion located on the inside of the boundary line, the IN portion having a signal output that is a part of the sensor output; and

an OUT portion located on the outside of the boundary line, the OUT portion having a signal output that is a part of the sensor output;

the classifier utilizing the IN portion signal output in order to classify a signal representing an IN event.

3. The system as claimed in claim 1, wherein multiple sensor sections are utilized, and the classifier event outputs associated with each sensor section are provided to inputs of an event integrator, the event integrator serving to further filter multiple classifier event outputs.

4. The system as claimed in claim 1, wherein the signal is transformed into a conductance signal prior to being classified.

5. The system as claimed in claim 1, wherein signals from multiple adjacent sensor sections are combined prior to being classified.

6. The system as claimed in claim 1, wherein:

the circuitry or memory containing predetermined information further contains information related to a racquet characteristic curve; and

the classifier is further configured to classify: 3) a racquet hit by utilizing the information related to a racquet hit characteristic curve.

7. The system as claimed in claim 1, wherein:

the characteristic curve information comprises duration of activation information having predetermined time duration values or ranges at a predetermined or calculated measurement threshold signal level;

the classifier further comprising circuitry or algorithms configured for determining a duration of an input signal at the predetermined threshold signal level and comparing the input signal duration against the predetermined time duration values or ranges for utilization in the classifying of the signals.

8. The system as claimed in claim 7, wherein:

the characteristic curve information additionally comprises amplitude information, the classifier further comprising circuitry or algorithms configured for utilizing the amplitude information in classifying the signals.

9. The system as claimed in claim 7, wherein:

the information comprising duration of activation information further comprises predetermined time duration values or ranges at an additional predetermined threshold signal level;

the classifier circuitry or algorithms further configured for determining an additional duration of the input signal at the additional predetermined threshold signal level and further comparing the additional input signal duration against the additional predetermined time duration values or ranges for utilization in the classifying of the signals.

10. The system as claimed in claim 1, wherein:

the information comprising duration of activation information further comprises predetermined peak detection values

the classifier circuitry or algorithms being further configured for determining the number of peaks in the input signal within a predetermined time duration and further comparing the number and size of peaks against the peak detection values.

11. The system as claimed in claim 1, wherein the calculated measurement threshold signal level is calculated based on a relationship with a base-level signal.

12. The system as claimed in claim 1, wherein the classifier further comprises a transform into the frequency domain such as a Fast Fourier Transforms (FFTs) or a wavelet transform, the classifier utilizing the frequency information in operating on the sensor signal and utilizing characteristic curve information containing frequency information.

13. The system as claimed in claim 1, wherein the classifier further comprises a correlator to compare sampled sensor signal data with characteristic curve shape information.

14. The system as claimed in claim 1, wherein the force sensing resistor comprises at least one layer of force sensing ink or paint and at least one layer of conductive material applied between sealed strips of plastic.

15. The system as claimed in claim 1, wherein the force sensing resistor comprises at least one layer of force sensing rubber and at least one layer of conductive material applied between sealed strips of plastic.

16. A system for determining whether a ball bounce occurs on an inside of a boundary line or an outside of the boundary line, comprising:

b) a health monitoring system comprising:

b1) an input for monitoring a signal related to each sensor or sensor section;

b2) at least one of circuitry and an algorithm for distinguishing a properly operating sensor or sensor section from one that is improperly operating indicating an error; and

b3) an output for providing an indication of a sensor, sensor section, or system error; and

c) an error notification system comprising an input connected to the output of the health monitoring system and an output connected to a user notification device.

17. The system as claimed in claim 16, wherein:

the sensors or sensor sections comprise a leakage resistor; and

the health monitoring system comprises circuitry for detecting a signal related to a current flow through the leakage resistor thereby indicating a connected and possibly correctly operating sensor and for detecting the absence of this signal thereby indicating a disconnected or malfunctioning sensor or sensor section, the health monitoring system configured to provide an error output to the error notification system given the absence of this signal.

18. The system as claimed in claim 16, wherein:

the health monitoring system comprises a table containing values or ranges indicative of proper operating values or ranges for the sensors, and comparator circuitry or algorithms for comparing a signal received at the respective monitoring inputs with the values or ranges contained in the table, the health monitoring system configured to provide an error output to the error notification system if the signal is outside of the value or ranges indicative of a proper operation of the sensors or sensor sections.

19. The system as claimed in claim 16, further comprising a base-level signal calculator, the base-level signal calculator comprising:

an input at which is received a signal from one or more of the sensors;

a calculator configured to establish a base-level sensor signal when the sensor is not under pressure; and

a comparator configured to determine if the base-level sensor signal is above or below a predetermined threshold; and

an output connected to the input of the health monitoring system at which information related to the base-level sensor signal is provided.

20. The system as claimed in claim 19, wherein the base-level signal calculator further comprises:

a timer configured to periodically trigger an adjustment of the base-level signal during play;

a memory for holding signal values between periodic triggering of the timer; and

a mechanism for eliminating the signal values stored in the memory that exceed values related to ball bounces, footsteps and racquet hits and determining a new base-level signal based on an average of remaining signal values.

21. A system for determining whether a ball bounce occurs on an inside of a boundary line or an outside of the boundary line, comprising:

b) a serve condition detection system, comprising:

b1) an input configured to receive an indication of a service and an indication of a service court for which the service is to occur; and

b2) an activation mechanism for activating a sensor group associated with service lines related to the service court and deactivating sensor groups associated with service lines unrelated to the service court.

22. The system as claimed in claim 21, wherein:

a sensor located behind one of the baselines provides its output to the input of the serve condition detection system; and

the detection system being configured to determine which service court sensor group to activate based on a predetermined prolonged signal from the sensors behind one of the baselines indicative of a person standing behind the baseline.

23. The system as claimed in claim 21, further comprising one or more wireless or wired devices configured to provide a signal indicative of the service area to the input of the serve condition detection system.

24. The system as claimed in claim 21, further comprising a switch for activating and deactivating the serve condition detection system.