WO2002025988A1

WO2002025988A1 - Real-time traffic transfer in multi-service communication networks system and method

Info

Publication number: WO2002025988A1
Application number: PCT/GR2001/000035
Authority: WO
Inventors: Themistoklis Rapsomanikis
Original assignee: Themistoklis Rapsomanikis
Priority date: 2000-09-25
Filing date: 2001-09-21
Publication date: 2002-03-28

Abstract

System and method for transfer of real-time traffic in multi-service packet switching communication networks. The system consists of a peak-rate policer (62) at the edge nodes (50), and an output queuing system (70) at the edge nodes (50) and the core nodes (52). The output queuing system (70) consists of two queues in series, a smoothing queue (78) and a final queue (84). Smoothing is performed on the first through a smoothing controller (82), while a packet scheduler (92) serves the second, as well as the queues of other supported services, at least the best-effort queue (96), driven by an adaptive tracker (88). The adaptive tracker (88) in order to produce periodically the driving signal (90) samples the occupancy of the final queue (86) and is synchronized with the smoothing controller (82), while the smoothing period is a multiple of the tracking period. The service method is based on tracking of an occupancy target of the final queue and provides explicit guarantees on the delay and the delay jitter, as well as constant 100% bandwidth allocation utilization. Alternatively, if an adaptive predictor (184) replaces the adaptive tracker (88), the service method can provide, apart from quality of service guarantees, the following of a selectable less-than-maximum allocation utilization target. The guarantees extend to multiple quality of service levels and independently of the packet length being constant or variable, while the delay jitter can be further decreased without altering the tracking-prediction period.

Description

REAL-TIME TRAFFIC TRANSFER IN MULTI-SERVICE COMMUNICATION NETWORKS SYSTEM AND METHOD

BACKGROUND-FIELD OF INVENTION

This invention relates generally to packet switching communication networks and more particularly to supporting real-time traffic in multi-service packet communication networks.

BACKGROUND-DISCUSSION OF PRIOR ART

Next generation packet switching networks are anticipated to support new and advanced applications that are expected to give users the closest feeling to physical colocation possible. Examples of such applications are multimedia teleconferencing, remote instrument and production line control, live digital television, interactive distance learning, teleimmersion in virtual shared environments for scientific visualization, and distributed interactive simulation. Applications of this type demand from the network a hard, inelastic real-time service; the performance guarantees on bandwidth, delay and delay jitter are strict and deterministic. This is in contrast to soft real-time services, where it is not uncommon for users to be willing to accept delays in the order of seconds and even tens of seconds, or no assurances are given about packet losses/delays, except at best in some form of probabilistic performance bounds developed with off-line statistical asymptotic techniques. Traditionally, per-flow connection admission control through user- declared source traffic characterization at connection setup is considered the main enabling mechanism for explicit quality of service (QoS) guarantees. Node service disciplines are tied more or less to admission control. This holds irrespective of the network being B-ISDN/ATM with Q.2931 signaling, or pure IntServ/IP with RSVP signaling.

The approach although offering an easy way to avoid network congestion has created in its own right some important problems. First, for live, real-time traffic it is very difficult, if not impossible, for the user to come up with an accurate description since no a priori traffic knowledge is available. This non-causality property results in practice in one of two extremes: the guarantees turn out very loose, that is, the allocated resources are underutilized to a large degree for most of the time, or, a great part of user traffic is not conforming to the description and as a result the user experiences severe degradation of application quality-although the part that is conforming does not cause network congestion. Traditional, stochastic model-based admission con- trol using a closed formula for the so-called effective bandwidth promises statistical QoS guarantees in an ideal, limiting, large buffer -r large number of sources/multiplexing capacity regime. Typically this involves traffic modeling assumptions that are hard to justify in real load for general sources or for the node's arrivals. Indeed, there is no general consensus on the correct approach to modeling because variable bit-rate (VBR) packet traffic often exhibits trends, spikes and long-range dependence that, in many cases (e.g. entertainment movies), have yet to be cap- tured with a static model. The potential of successful static statistical modeling for future advanced applications is even smaller, if we consider for example the fact that a composite multimedia VBR flow may be even more bursty than a component video flow.

On the other hand, production of variable bit-rate traffic may make its handling by the network more difficult but is associated with a constant application quality and the possibility of statistical multiplexing in the network for better exploitation-utilization of bandwidth. On the contrary, production of constant bit-rate (CBR) traffic makes traffic much more predictable and facilitates its management, but leads to variable application quality and inability to extract statistical multiplexing gain, rendering packet switched networks essentially equivalent to circuit switched networks in which, amelioration of resource exploitation efficiency is not possible. There are studies promising statistical QoS guarantees but the issue of resource allocation utilization by the real-time service is entirely neglected.

However, the close relation between service charging, service allocation utilization, and QoS is very important for wide deployment of guaranteed packet services. For example, users accustomed to a certain QoS level in digital, circuit switched telephony SONET/SDH PSTN may not find acceptable to trade-off lower price or rebates with lower quality in packet telephony, as is currently proposed taking into account the technical challenges packet networks face when trying to deliver traffic sensitive to delay and delay jitter. Instead, they might expect from a packet network to offer the same QoS at a lower price (because of the potential for higher bandwidth utilization). It is safe to assume that a prospective subscriber, if having to choose between a hard real-time service and a soft real-time service, the latter with or without statistical assurances, both offered at the same price/charge, would prefer the former.

Second, scalability problems about the end-to-end per-flow admission control, due to the need for state information maintenance for every flow in the network, have recently fuelled a shift in interest from integrated services-IntServ to differentiated services-DiffServ. According to the latter paradigm, after packets are classified into one of the predetermined so-called per-hop behaviors (PHB) at an administrative domain edge, are grouped with other packets ofthe same behavior and receive forwarding treatment at the domain core according to the PHB's codepoint. While service differentiation becomes scalable and robust in this way, the price with current solutions is paid in QoS guarantees, bandwidth utilization and granularity of control. More specifically, the premium service is based on the expedited forwarding PHB (EF). The latter promises the hardest QoS guarantees under the differentiated services umbrella. If we take the typical implementation of the premium service with priority service discipline, scheduling, it has been shown in prior art reference Providing Guaranteed Services Without Per Flow Man- agement, Ion Stoica and Hui Zhang, Technical Report CMU-CS-99-133, Carnegie Mellon University, May 1998, that in order for the end-to-end delay bound to stay at a reasonable value, the service-induced utilization has to be very small (around 10%). This result comes despite shaping of each user flow to CBR. Indeed, it has been found in reference Delay Bounds in a Network with Aggregate Scheduling, Anna Charny and Jean- Yves Le Boudec, International Workshop on Quality of future Internet Services 2000, September 2000, Berlin, that the utilization level depends on the maximum hop count that a flow, which is part of aggregate traffic, may travel. The effort is how not to drop the flow-stateless core concept, while achieving levels of assurances similar to those of flow-stateful networks, that is, of integrated services networks. The underlying notion is that the end-to-end performance with differentiated services can only be an ap- proximation and certainly never better of what is achieved with integrated services.

In prior art reference SCED+: Efficient Management of Quality of Service Guarantees, Rene La Cruz, IEEE INFOCOM '98, March/April 1998, San Francisco, the SCED+ discipline is proposed for ATM networks. A design goal is the efficient link sharing between guaranteed and best-effort (no quality of service guarantees) classes. The idea is to delay early-arrived guaran- teed packets, if there are best-effort packets awaiting transmission. The algorithm's complexity depends linearly on the number of 'virtual paths' carrying guaranteed traffic on each output port of the packet switched node. Additional complication arises from the two per-hop service curves needed to bound delay jitter. While better statistical multiplexing between classes is demonstrated, no quantitative allocation utilization assurances can be given.

Non-work-conserving disciplines in packet switching networks

The notion of 'non-work-conserving' and 'work-conserving' originates in queuing systems theory and refers to the ability of a queuing system to stay idle although there is work in it. In packet networks this translates to a node's ability to refrain from transmitting packets on the line although there are packets at the queues. Examples of work-conserving disciplines are priority queuing, weighted fair queuing, virtual clock queuing and self-clocked fair queuing. Non-work- conserving discipline examples are delay-jitter earliest deadline first, rate-jitter earliest deadline first, rate-controlled static priority queuing and rate-controlled earliest deadline first. Non-work- conserving disciplines are known to reshape traffic in the network so that it does not become burstier with the number of hops and for that, they are in principle suitable for real-time traffic (with use of per-flow state information), while commonly they cannot achieve good utilization of bandwidth. The reverse are observed in work-conserving disciplines, see reference Service Disciplines For Guaranteed Performance Service in Packet-Switching Networks, Hui Zhang, Proceedings of the IEEE, October 1995. The fact that work-conserving disciplines, as priority queu- ing and guaranteed-rate disciplines as for example weighted fair queuing (WFQ), allow bursti- ness and delay jitter to increase has the undesirable consequence that as more hops are encountered, exceedingly larger resources have to be reserved. As for rate-proportional WFQ, because ofthe delay-bandwidth coupling, it is not possible to attain high allocation (and hence link) utilization for VBR traffic with a long-term average rate smaller than the weighted minimum alloca- tion.

SUMMARY OF THE INVENTION

System and method for transfer of real-time traffic in multi-service packet switching networks with quantitative end-to-end guarantees on quality of service and bandwidth allocation utilization. The guarantees entail constant maximum per hop packet delay and delay jitter, as well as maximum or selectable less-than-maximum bandwidth allocation utilization.

OBJECTS AND ADVANTAGES

Accordingly, several objects and advantages of my invention are:

1. A real-time service support system in multiple services networks with unified provisioning of explicit guarantees on quality of service and bandwidth allocation utilization.

2. A real-time service without need of traffic parameter declaration at each call setup for the flow's admission into the network. 3. A real-time service that is not based on static statistical traffic modeling.

4. A real-time service without need of per flow state information maintenance, neither at the network's core nor at the network's edge.

5. A real-time service where sources are free to transmit traffic with their inherent variable rate and without any shaping-modification of the traffic pattern from the network as long as they do not exceed a peak rate that is the object of negotiation between the user and the service provider on a large time scale.

6. A real-time service with appropriate service discipline so that the delay bound is independent ofthe bandwidth utilization.

7. A real-time service with appropriate service discipline so that the bandwidth utilization and the quality metrics are independent ofthe maximum hop count encountered by an individual flow which is part of aggregate traffic, of the network topology, or the load of other supported services. 8. A real-time service that may offer the same quality guarantees on aggregate traffic and the individual flows. 5 9. A real-time service with constant 100% bandwidth allocation utilization and constant maximum end-to-end delay and delay jitter, independent of traffic burstiness and non-stationarity, buffer size, multiplexing capacity and degree of aggregation in the network. 10. A real-time service with selectable, less-than-maximum bandwidth allocation utilization target. 10 11. A real-time service with use of only local adaptive state for the output port's aggregate traffic and without need of a coordination protocol between packet switches.

12. A real-time service based on self-tuning and without need of human intervention for its normal micro-operation.

13. A real-time service support system where offered guarantees apply equally whether the 15 packet length supported by the network is constant or variable.

14. A real-time service with ability of providing guarantees on multiple quality levels under the same terms of bandwidth allocation utilization that apply in a service offering a single quality level.

15. An adaptive non- work-conserving scheduling discipline with the most efficient link sharing 0 possible between the real-time service and other services.

16. An adaptive non- work-conserving scheduling discipline that does not require per packet calculation of eligible times or deadlines.

17. An adaptive non- work-conserving scheduling discipline where the maximum service rate required for the aggregate traffic drops with the number of hops. 5 Further objects and advantages of my invention will become apparent from a consideration of the drawings and ensuing description.

BRIEF DESCRIPTION OF THE DRAWINGS

0 In the drawings, figures that are closely related have the same numeral but different letter suffixes, while the same reference numbers represents identical parts in different figures.

FIG. 1 is a real-time traffic transfer scenario through a sequence of administrative domains with a node-in-series topology, of a multi-service network.

FIG. 2A is a simplified block diagram ofthe data plane architecture of an edge node that sup- 5 ports the real-time service. FIG. 2B is a simplified block diagram of the data plane architecture of a core node that supports the real-time service.

FIG. 3 is a block diagram ofthe output queuing system for constant 100% bandwidth allocation utilization and one level of quality of service guarantees. FIG. 4 is a timing diagram of real-time packet service from the smoothing queue and the final queue for the basic embodiment, with fixed packet length.

FIG. 5 is a timing diagram of real-time packet service from the smoothing queue and the final queue for the embodiment of decoupling of delay and delay jitter from the control period, with fixed packet length. FIG. 6 is a timing diagram of real-time packet service from the smoothing queue and the final queue for the embodiment of variable packet length.

FIG. 7 is a simplified block diagram of the output queuing system for constant 100% bandwidth allocation utilization, and multiple levels of quality of service guarantees.

FIG. 8 is a timing diagram of real-time packet service from the final queue for the embodi- ment of multiple quality levels.

FIG. 9A shows the packet trace of aggregate video traffic.

FIG. 9B shows the autocorrelation function of aggregate video traffic.

FIG. 10A shows the real-time buffer occupancy ofthe first node in the topology with scheduling driven by adaptive tracking. FIG. 10B shows the real-time buffer occupancy of the second node in the topology with scheduling driven by adaptive tracking.

FIG. 11A is a detailed block diagram ofthe real-time queuing system, with one level of quality of service guarantees.

FIG. 1 IB is a detailed block diagram ofthe adaptive tracker. FIG. 11C is a flow diagram ofthe operation ofthe supervision unit.

FIG. 12 shows the virtual queue occupancy during initialization.

FIG. 13 A shows the equivalent queue occupancy when no arrivals occur for a specific interval.

FIG. 13B shows the service rate when no arrivals occur for a specific interval. FIG. 14 is a block diagram of the queuing system in the embodiment of less-than-maximum bandwidth allocation utilization target, and one level of quality of service guarantees.

FIG. 15 is a timing diagram of real-time packet service from the final queue for the embodiment of less-than-maximum bandwidth allocation utilization target, with fixed packet length.

FIG. 16 shows the allocation utilization in the embodiment of less-than-maximum bandwidth allocation utilization target.

DETAILED DESCRIPTION OF THE INVENTION

A. Basic Embodiment-Constant 100% Allocation Utilization with One Quality of Service Level and Constant Packet Length

In FIG. 1 a multi-service network topology of nodes in series that is formed by a concatenation of two administrative domains is shown. Each domain is self-sufficient in what regards the internal resource management, while end-to-end performance is realized through service level agreements between administrative entities of neighboring domains. The domain itself consists of the user terminal equipments, the edge nodes where: users gain access to the network, data plane and control plane state information are stored, classification in different per hop behaviors PHB-services takes place and service level agreements between neighboring domains are en- forced; and the core nodes, that are responsible for fast packet forwarding, depending on the service to which they belong. More specifically, in FIG. 1, a user terminal system-source (SI) to (S10) transmits traffic to an administrative domain (54) at an edge node EN1 (50) through an access link (AL1) to (AL10) respectively. Edge node EN1 (50) is connected to a core node CN1 (52) with a backbone link (BL1). Core node CN1 (52) is connected to a core node CN2 (52) through a backbone link (BL2). Core node CN2 (52) is connected to an edge node EN2 (50) with a backbone link (BL3). The latter is the only edge node (50) of administrative domain (54) that communicates with an administrative domain (56), with direct connection to a peer edge node EN3 (50) of administrative domain (56) through a backbone link (BL4).

In administrative domain (56), edge node EN3 (50) uses a backbone link (BL5) to connect to a core node CN3 (52). Core node CN3 (52) is connected physically to an edge node EN4 (50) through a backbone link (BL6). Finally, edge node EN4 (50) connects to a user terminal system- destination (Dl) to (D10) through an access link (ALU) to (AL20) respectively. Destination (Dl) to (D10) is the receiver of traffic transmitted to the network by source (SI) to (S10) respectively and which is routed/switched to destination via the edge and core node sequence, travers- ing a plurality of administrative domains. To administrative domain (54) belong the edge nodes EN1 and EN2 (50), and the core nodes CN1 and CN2 (52). Accordingly, to administrative domain (56) belong edge nodes EN3 and EN4 (50), and core node CN3 (52).

Regarding FIG. 2A, a simplified block diagram of the data plane of the edge node (50) in a network that supports the real-time traffic transfer method of this invention is shown. For this service, sources are free to transmit traffic in their inherent variable rate and without any shap- ing-modification of traffic pattern by the network, as long as they do not violate a peak rate that is the object of negotiation between user and service provider on a large time scale. The modification-shaping of traffic to conform to a static description because of the inaccuracy involved leads either to network underload in the case of loose description, or, to large shaping delays when the description underestimates source's instantaneous production of packets.

The basic units that make up the edge node in the data plane are an input port (60), a peak-rate policer (62) per input port, a service/PHB classifier (64) per input port, an output port selector unit (66) per input port, a central switching/routing fabric (68), an output queuing system (70) per output port, and an output port (72). Input port (60) carries out the physical layer operations, reconstructing a packet flow from a serial input (58) (packetization) whether the packets are IP datagrams in pure IP networks, or, ATM cells in networks based on the asynchronous transfer mode technology (ATM). Peak-rate policer (62) enforces the long time scale contract on the peak rate agreed upon by the user and the service provider. The interval used to measure the rate (as the information quantum, e.g. in bits, divided by the measuring interval) must be not greater and preferably equal to the period of the service discipline so that the operations of bandwidth dimensioning (large time scale) and bandwidth allocation (short time scale) have a common reference base.

Service/PHB classifier (64) inserts the DSCP codepoint in the real-time packet's header to be read by the core nodes for service identification. No signaling (control plane) between the sources and the edge node is needed because users have already negotiated a peak rate with the network. The terminal equipment may insert directly a real-time service identifier (not necessarily the same with the DSCP codepoint) in packets so that they are recognized by the service/PHB classifier (64) of the edge node. Output port selector (66) routes the real-time packet to the appropriate output port (72). This is performed either with the classical routing table lookup in IP networks, or, with label techniques, as in ATM networks/IP networks enhanced with Multiprotocol Label Switching (MPLS). In the case of a label technique, packet forwarding is simplified since it is not required to make a per packet routing decision but only to read a label in the incoming flow and to map a new label for the outgoing flow which corresponds to a fixed virtual path in the network for all real-time packets. The fixed path is also important for the reduction of delay jitter, which is inevitable when different packets ofthe same flow may take different routes in the network. Central switching/routing fabric (68) is the interconnection network that actually transfers packets from the input to the selected output. Output queuing system (70) leads packets to the real-time queue where they receive service treatment as aggregate traffic according to the scheduling method of this invention. Finally, output port (72) transmits packets on the output link transforming packets again to a serial stream (74) (serialization).

In FIG. 2B a simplified block diagram of the data plane of the core node (52) in a network that supports the real-time traffic transfer method of this invention is shown. The main difference in the data plane with the edge node is that peak-rate policer (62) and service/PHB classifier (64)

5 are no longer needed. The other units are exactly the same in structure and operation as the ones ofthe edge node.

FIG. 3 shows the output queuing system (70) ofthe real-time traffic transfer method for guaranteed constant 100% bandwidth allocation utilization and one level of quality of service guaran- 'Λ tees. An aggregate packet traffic (76) is the real-time input signal that has been switched/routed 0 to the specific output port of the node. A smoothing queue (78) is the queue that receives realtime aggregate packet traffic (76). A smoothing queue server (80) transmits real-time packets stored at smoothing queue (78). Its operation is guided by a smoothing controller (82). The output of smoothing queue server (80) is the input for a real-time output queue, or, final queue (84). The occupancy of output queue (86) is the input to an adaptive tracker (88). Adaptive tracker 5 (88) guides the operation of a packet scheduler, or, service discipline, or, service rate controller (92) through a scheduler control output (90). Packet scheduler (92) selects the packet of all queues that is eligible to be inserted to a transmitted packet flow (94). Among those queues are, apart the real-time output queue (84), queues of other supported services, at least a best-effort queue (96) that receives an aggregate traffic without-quality-guarantees (98), that is to be trans- 0 mitted from the specific output port.

Goal of adaptive tracker (88) is to grant such an allocation to the real-time output queue (84) by scheduler (92), that the occupancy of this queue follows, tracks a reference trajectory that more specifically is a constant and non-trivial, non-zero, value of B packets.

The smoothing window is constant for all the nodes, edge and core, and time-invariant. The 5 goal of smoothing is not to reduce significantly the burstiness of VBR traffic. This would create smoothing delays and subsequent playback delays at the destination that are prohibitive for a hard real-time service. The goal is to facilitate the operation ofthe adaptive tracker. An adaptive tracker or, controller in general is made of two loops: the regular feedback loop and the estimation loop. A way to robustify the estimation loop is to ensure that the fastest time scale of the 0 controlled system is smaller than the tracking, control period. It is not difficult to produce 'bursts' in parameter estimates and ultimately instability, if the high frequency dynamics are not taken into account. Smoothing is equivalent to filtering of the signal so that it has sufficient energy only at low frequencies. In the system of my invention this is achieved with a synchronization ofthe two mechanisms, smoothing and adaptive tracking, and by setting the smoothing win- dow equal to an integer multiple of the control period. How much larger than the control period must the smoothing window be, depends on two factors. First, it must not be very far from the tracker's sampling interval because that would undermine the per hop delay and the total delay budget; the smoothing delay introduced is part of the 'constant' component of overall delay that includes also serialization, propagation, packetization and switching delays. Second, it must not be too close to the tracking period because that would deteriorate the adaptive tracker's performance.

The operation of the output queuing system is generally as follows. The smoothed aggregate real-time traffic is fed to final queue (84), the occupancy of which (86) is sampled to compute, by adaptive tracker (88), the bandwidth allocation or the service rate for the real-time service that scheduler (92) must comply with. The service quantum, that is, the number of real-time packets that can be serviced on the tracking-scheduling period, is equal to the service rate computed times the tracking period. In this way packets are classified in two groups: the 'normal' packets that are within the service quantum and will depart from final queue (84) during the current pe- riod, and the 'excessive' packets that exceed the service quantum and will stay at final queue (84) to depart in the next period. So final queue (84) is never empty when scheduler (92) turns to it for packet service. This results in an adaptive tracking-based, aggregate non-work-conserving discipline, in the sense that scheduler (92) may not serve real-time packets although final queue (84) has non-zero occupancy (86). Viewing the queuing system ofthe final queue in terms of adaptive tracking, the smoothed arrivals of aggregated real-time traffic are the input signal of the system. The controlled system is the final queue, with occupancy (86) as the output. The latter is an input to adaptive tracker (88) together with the reference point, the constant occupancy of B packets. The output ofthe tracker is the control effort, e.g., the rate of service ofthe final queue by the scheduler.

Theory of operation of adaptive tracking and scheduling The symbols used in the following are collected in the table below.

Symbol Meaning y_n Final queue occupancy at the n^th sampling instant

Final queue service rate of period n

A, Final queue of period n

^SmQn Smoothing queue service quantum of tracking period n

SQn Final queue service quantum of period n

Part of A_n that are included in SQ_n

Tracking, control period

D P,^roc Adaptive tracker processing delay

B Adaptive tracking reference point

U„ Allocation utilization of period n

Per hop final queue delay of 'normal' packet i arriving at the final queue on pe¬

^J^nor riod /?

D ι,n Per hop final queue delay of 'excessive' packet i arriving at the final queue on ppeerriioodd nn

DJ Per hop final queue delay jitter of any two packets i,j arriving at the final queue d dduuurrriiinnnggg pppeeerrriiioooddd nnn

Per hop final queue delay jitter of any two packets i,j arriving at the final queue

^±JJm,n on periods m and n respectively, m≠n

The time is slotted in fixed computation intervals of h seconds and the allocation u_n, in pack- ets/s is available from the tracker D_proc seconds after the sampling instant that marks the start of the n^th control period. D is much smaller than h, for example equal to 0.1 h. The final queue occupancy measured in packets at the sampling instant is y_n. The actual arrivals at the final queue in the interval [nh, (n+l)h] are A_n. The queue dynamics are:

y_n = y_n-\ + Λ-l - ^Un- + ^un-\^Dproc ~ U_n-2^Dproc- ^{' (1)}

Since it is difficult to find an analytical expression for A_nΛ, y_n is modeled as a time-delayed ARMAX:

. y_n + ^a\y_n-\ +^•-+ a n-k = K-Δ + ^{• • •} + ⁽²⁾

+ ^bm^un-A-m ^{+ e}n + ^c\^en-\ + ' ' -<%-fc > where e_n are independent and identically distributed zero-mean Gaussian variables, Δ the delay and a_t, b_t, c_t unknown and time- varying parameters. In terms ofthe forward shift operator q this is written: ^A _gy„ = 1^k~ B_q u_n + C_q e_n , where (3)

, k-\ m _.

^Aq = 1 + ∑ «*-#' > ^Bq = ∑ ' «** /=0 /=0

C_q=q^k+∑c_k__lq'.

For a minimum variance adaptive controller, the time delay is equal to the pole excess ofthe system, Δ=k-m. Then (3) is equivalent to:

'^■qys„n=B q u **„n+C e_n,with (4) k-\ =9 +∑a_k__iqⁱ ,B, = b_iq^m~l and

.=0 ;=0 k-\ c_q=q^k + ∑c_k__iqⁱ. i=

For closed-loop identification of system parameters, extended recursive least squares (ELS) estimation is used that shows good convergence and adaptation. The parameters are first estimated with ELS and then used in the controller design as if they were the true ones. Let,

The true state at step n is then:

The estimation vector is defined as:

®n ⁼ [^al,n ^{■ ■ ■} <*k,n >

The estimated state at step n is:

y_n ^~Φn θ_n__x, where (8)

^{■ ε}n =y_n-yn> the prediction error. The computation of the state estimation vector with ELS and variable for- getting factor is given by:

ΛΛ = P_n-ι where

λ_n =λλ„__l +(l-^), the variable forgetting factor. The controller design is based on predicting the output A steps ahead and computing input u_n so that the predicted output is equal to the setpoint 5≠0. For an in- direct minimum variance self-tuner, this is accomplished by solving the following Diophantine equation:

q^K~XC_n,q = λ,q Ka + a • ^whe™ ^ ,q = ^^{Δ_1 +} An <l ^~2 + ' ^{* ■} + Λ-l,.. ^and G_n,_q = g₀,n <l^k~l + S\,n ^~2 + - + gk- ^■

The control input is then given by:

qKq «„ = <f^lC ,q ^{B ~} ₎₉Λ ■ ^(11}

Two saturation amplitude constraints apply to the control input computed from eq. (11). The one, a fixed externally configurable constraint, u_high , accounts for the requirement that the final queue service rate must not exceed the provisioned bandwidth for the specific hop for the real-time service. As the arrival rate at the smoothing queue of edge nodes is always enforced to be upper bounded by the service provision via peak-rate policing, the above constraint on the control input guarantees that the arrival rate at the smoothing queue is upper bounded network-wide by the per hop service provision. The other, a time- arying internal constraint, u_{low n} , represents the guarantee that the eventual positive tracking error, y_n > B , of one control period is always serviced on the next period and that otherwise the service rate is nonnegative. Thus, with u_n' from (11), the control input that takes into account the input saturation nonlinearities is finally:

u_n

where u_{hi h} = u_rt = — — , A_rt the upper bound on the arrivals at the smoothing queue during a h control period according to the per hop peak bandwidth provision and

So the procedure of adaptive tracking consists of, first, the recursive computation ofthe state vector from eq. (9), second, the recursive solution of the Diophantine equation (10) and finally the recursive computation of the control input u„ with eq. (11), subject to the constraints of eq.

(12). Adaptive tracker (88) essentially calculates the service quantum SQ_n, the number of real-time packets that are to be transmitted with the goal of leading the final queue occupancy (86) to the reference point. The same holds true for the smoothing controller (82) as well; the goal there is to smooth the variations in the number of packets arriving at the final queue over a plurality of control periods. How these packets are transmitted from the smoothing queue (78) or the final queue (84) during the control period is not of interest to the smoothing controller (82)/adaptive tracker (88). The service rate applied in both cases may follow any reasonable policy, as long as the constraints of the service quantum and the maximum bandwidth provisioned for the packet switching node are met. Such a policy is to service both queues with u_max at the start ofthe control period, where u_max is the link bandwidth, until the entire service quantum has been satisfied for the smoothing queue (78) and the final queue (84), and then to await the new cycle (for the scheduler, as far as real-time packet service is concerned). This replaces the straightforward way of servicing the smoothing queue (78) with the number of packets buffered during the smoothing interval divided by the smoothing interval, and the final queue (84) with u_n. The peak-rate policer (62) at the edge nodes enforcing the user-service provider contract and the combination of smoothing controller (82)-packet scheduler (92) guarantee that the arrival rate of the aggregate real-time traffic (76) at smoothing queue (78) of any hop and during any control interval never exceeds u_max. In fact, it will not exceed u_rt< u_max where u_n is the provisioned bandwidth allocation for the real-time service, since it is expected that a portion of the overall provisioned bandwidth will be reserved as a minimum allocation for other services, at least for the best-effort service. This mode of operation is the preferred one since it facilitates multi-service implementation. In general the computed SQ_n of any period is expended in possible 'excessive' packets of the previous period and 'normal' packets ofthe current period,

SQ_n = A_nΛ -S_{nΛ +}S_n. (13)

Let the allocation utilization of period n, U_n , be defined as the fraction of time from the expected service start of the first real-time packet of the period to the expected service completion ofthe last real-time packet ofthe period, the scheduler was busy servicing real-time packets. The following result on the bandwidth allocation utilization holds:

I. There are no idle service intervals between any two packets ofthe SQ_n for every «

II. If v„ > 0 for every n, then U„ =1 (100%) for every n.

The condition y_n > 0 is easily satisfied under stable adaptive tracking with a maximum negative prediction error less than the occupancy target B. The arrival of a complete 'normal' packet i on period n at final queue (84) occurs at the instant: 1 (14) t' ior = nh + i , i = l,...,S_n. ^umax

The departure of a 'normal' packet during period n from final queue (84) starts at the instant:

t„or = nh + [{A_n__l - S_n_ι + l)+ (/ - l)] — ⁽¹⁵⁾

"^■max

= nh + ( .- - S„__j + i) , i = 1,...,S„.

The arrival of a complete 'excessive' packet / on period n at final queue (84) is at the instant: \ 1 ⁽¹⁶⁾ c = ^{nh +} i^Sn + 0 > » = L- -5„. "max

The departure of an 'excessive' packet /' that arrived on period n at final queue (84) begins at the instant:

ή^» _xc = (n + l)h ₊ [l ₊ (i-l)]- - ^(1?)

"max

= (n + l)h + i , i = l,...,A_n -S„. ^umax The service of the first real-time packet of the period from final queue (84) starts not exactly at the n^th sampling instant, but after, so that the case A_nΛ-S_nΛ≠0 (that is, the case where ^umax there are 'excessive' packets left over from the previous period) is treated identically with the case A_nΛ-S_n__γ =0 for which transmission must start time units in period n. ^umax

For the 'normal' or 'excessive' packet i arriving at final queue (84) on period n, D^l _or or D^'_c denotes the queuing delay ofthe corresponding packet at the final queue (84) of any hop. The following result holds for D' and D£' _c : If A_n ≥l then:

H. ^De"c = ^Dexc = h ~ S_n ^umax III. 0 ≤ D_n"_or , D _xc ≤ h.

Let DJ^'^J denote the per hop delay jitter at the final queue of any two packets i,j that arrived at the final queue during period n. Likewise, DJ_m ^J„ denotes the per hop delay jitter at the final queue of any two packets i,j arriving at the final queue on periods m and n respectively, m≠n. The following result then holds for DJ' and DJ_m'^J _n :

I. 0 ≤ DJ^V ≤ h for every i,j

II. 0 ≤ DJ^' „ ≤ h for every i,j.

So one sees that the per hop delay of any packet and the per hop delay jitter between any two packets at final queue (84) have as worst case value the control period h.

Another advantage of the real-time traffic transfer system relates to the average delay of best- effort packets. Since packet scheduler (92) guarantees tight following of the load shape of the real-time VBR aggregate, the service rate and therefore the time allocated to the final queue will be lower than u_H, the real-time service provisioned partition, and (u_rth) respectively. Even ^umax more, the higher the burstiness of the real-time VBR aggregate, the lower the time allocated to the real-time service and the higher the time allocated to the best-effort service on the average. These (higher service rate and time dedicated to best-effort service) lead to better average delays for best-effort packets, compared to fixed partitioning, an important objective for QoS-enabled networks. The procedures for smoothing and packet scheduling for the basic embodiment are given in pseudocode form:

on packet_arrival (smoothing_queue,p) /* head of packet p arrives at the smoothing queue */ enqueue (smoothing_queue,p); on packet jsend (smoothing _queue,p) if (nh ≤ time < nh + S_mQ_n ) ^umax update ( S_mQ_n ); forp=l to S_mQ_n dequeue (smoothing_queue,p, u_maχ); else /* nh + S_mQ_n < time < (n + l)h V u_m n:=n+l; idle;

(i) smoothing on packet_arrival (real_time_queue,p) /* head of packet p arrives at the final queue */ enqueue (real_time_queue,p); on packet_send (real_time_queue,p) if( nh + < time < nh + (SQ„ + 1) ) update (SQ_n); forp=l to SQ_n dequeue (real_time_queue,p, u_maχ);

1 1 else /* nh + (SQ_n + 1) < time < (n + \)h + V u "max u "max n:=n+l; idle;

(ii) scheduling

FIG. 4 shows the procedures in the form of timing diagrams. Regarding the timing diagram of smoothing, a smoothing period (102) starts at a time instant (100) and is completed at a time in- stant (104). As already pointed out, this period is set equal to an integer multiple of a tracking period (106) and the two mechanisms are synchronized, that is, the start of tracking computation of a first tracking period (108) coincides with the start of smoothing computation (100) and the completion of tracking computation of the last tracking period (108) coincides with the completion of smoothing computation (104). Smoothing is done at the start of every tracking period (106). From smoothing queue (78) a number of packets is transmitted, the smoothing queue service quantum in a tracking period, equal to the smoothing queue service quantum in a smoothing period divided by the number of tracking periods (106) that make smoothing period (102). The smoothing queue service quantum in a smoothing period in turn is computed as the smoothing queue (78) occupancy at the sampling instant (100), (104) etc. Particularly, inside one tracking period (106), smoothing starts at tracker's sampling instant (108) and continues for, as much is needed to satisfy the smoothing queue service quantum in a tracking period with u_max. For example, for the n^th tracking period (106), the smoothing queue service quantum in a tracking period is S_mQ_n, transmission from smoothing queue (78) starts at the time instant nh (100), lasts for the

time duration (110) equal to S_mQ_n to end at the time instant (112) nh + S_mQ_n .

In relation to the timing diagram of scheduling, transmission from final queue (84) starts not at tracker's sampling instant (108), but after, as already noted. For example, for the n^th ^umax tracking period (106), the final queue service quantum is SQ_n, service from final queue starts at

1 1 an instant nh + (114), lasts for a time duration (116) equal to SQ to end at a time in- stant (118) nh + (SQ_n + 1)— — .

B. Constant 100% Allocation Utilization with One Quality of Service Level, Constant Packet Length, and Decoupling of Delay and Delay Jitter from the Control Period

If additional delay jitter decrease is desirable there are two alternatives, namely faster control, and overprovisioning with minimum delay enforcement.

The more direct way to reduce jitter is to shorten the control period h. It is a viable technique, as long as the scheme can be implemented in high speed networks. The reader should recall that the computation from adaptive tracker (88), which influences the transmission schedules of all real-time packets is done only once per period for the whole aggregate and not once per packet

(and even per flow) as with other mechanisms. For example, if h is equal to 1 ms, a reduction to

— h is well within current technology limits and for a path of 20 hops gives a worst case delay jit-

ter of 10 ms, that would be reached in 10 hops with the original cycle duration. So, the Jitter Reduction Factor, JRF =— in this case, means that by multiplying h with JRF we multiply the worst case delay jitter with JRF as well.

The other technique that forms the current embodiment is to retain the original cycle duration and enforce the following: (α) service from smoothing queue (78) for the n^th period starts at the nh + (l - JRF)h instant while (b) service of 'normal' packets from final queue (84) starts at

1 1 nh + (l - JRF)h + i— — and service of 'excessive' packets at the instant (n + \)h + i— — , where ^umax ^umax ^u _m' _ax ^{= u} _{m x >} <JRF<1. This has the benefit of decoupling the delay and the delay jitter from

JRF the control period. The goal now is to schedule the same service quantum in less time, so as to derive a smaller worst case per hop delay jitter. The equations (14) to (17) now become:

TRF (18) _v =nh + {\ -JRF)h + i -, i = l,...,S„. ^umax rap (19) =nh + (l -JBF)h ₊ i≡^-, i = l...,S„. ^umax

, / TRF (20) _* = ^nh + & - ^JRF)h + (S_n + —. / = 1,.. A - S_H. ^umax ή_e ⁿ _xc = {n ₊ \)h ₊ i^-, i = \,...,A_n -S_n. ⁽²¹⁾ ^umax

The following result holds forD^ and D^'_xc : If A_n >\ then:

I ^■"^■.^• D ^t-Η^i,oⁿr =D '-'n"or = 0 "

III. 0<D" ≤JRFh.

While for Z> and DJ^_n :

I. 0 < DJ_n ≤ JRFh for every i,j

II. 0 < Z J^„ < JRFh for every z, _/.

Therefore, by shifting service start from smoothing queue (78) (l - JRF)h time units after the equivalent instant of the basic embodiment, starting transmission of 'excessive' packets at

TRF' / TRF

(n + l)h + /' , of 'normal' packets at nh + { - JRF)h + i and by provisioning a link capac- ^umax ^{■ u}max ity that is times the equivalent of the basic embodiment, we multiply the worst case per

JRF

hop delay jitter with JRF. Of course (l - JRE)/z + or,

must apply. In words, by the time 'normal' packet service begins

'excessive' packet service must have finished. This means that one has to take into account the maximum positive prediction error when choosing the value of JRF. For practical jitter reduction, since the positive prediction error is quite small with stable adaptive control,

TRF

(l - JRF)h + is much larger than the time needed to transmit all 'excessive' packets.

"max

Below the procedures of smoothing and packet scheduling in pseudocode form for the embodiment of decoupling of delay and delay jitter from the control period are given: on packet_arrival (smoothing_queue,p) /* head of packet p arrives at the smoothing queue */ enqueue (smoothing _queue,p); on packet _send (smoothing_queue,p) if(( time > nh + (l - JRF)h )&. (time < nh + (l - JRF)h + S_mQ_n ^-)) ^umax update (S_mQ„); forp=\ to S_mQ_n dequeue (smoothιng_queue,p, ); ^umax nh + {l - JRF)h + S_mQ_n — < time < else /* u_max */

(n + l)h + (l - JRF)h n:=n+l; idle;

(i) smoothing

on packet _arrival (real_time_queue,p) /* head of packet p arrives at the final queue */ enqueue (real_time_queue,p); on packet _send (real_time_queue,p) if( nh + < time < nh + (A_n__x - S_n__x + 1) ) ^umax ^umax update ( SQ„ ); /* SQ„ = A„__x - S„„ + S„ */ forp^l to A_n__x - S_n__x dequeue ( treal r_tι •me_queue,p, ^JRF > ).; ^umax

/* 'excessive' packet service */

TRF else if (nh + ( - JRF)h H < time < ^umax nh + (l ~ JRF)h + (S„ + 1)^- ) forp=l to S_n

JRF dequeue yrealjιme_queue,p, ); ^umax

/* 'normal' packet service */

TRF else /* nh + (l - JRF)h + (S + 1) < time < u

J JRRtF

(n + )h + */ ^umax n:=n+l; idle;

(ii) scheduling FIG. 5 shows the procedures above in the form of timing diagrams. Regarding the timing diagram of smoothing, smoothing starts not at tracker's sampling instant (108) as in the basic embodiment but (l - JRF)h after. For example for the n^th tracking period (106), where the smoothing queue service quantum in a tracking period is S_mQ_n, transmission from smoothing queue (78)

TRF starts at an instant nh + (l - JRF)h (120), continues for a time interval (122) equal to S_mQ_n ^umax TRF to end at a time instant (124) nh + (l - JRF)h + S_mQ„ . In relation to the timing diagram of

"max scheduling, there are two service intervals. The first, in which transmission of 'excessive' pack- ets of the previous period takes place, starts for the n^th tracking period (106), at a time instant

TRF TRF

(126) nh + , lasts for a time interval (128) equal to (A„__X - S„__X) to be completed at a ^umax ^umax

JRF time instant (130) nh + (A_n__x - S_n__x + 1) . Afterwards, the scheduler stays idle in what regards ^umax

TRF transmission of real-time packets until a time instant (132) nh + (l - JRF)h + when transmis-

"^■max TRF sion of 'normal' packets begins, for a duration (134) equal to S„ until a time instant (136) ^umax

TRF nh + (l - JRF)h + (S_n + 1)-^- .

C. Constant 100% Allocation Utilization with One Quality of Service Level, and Variable Packet Length

The previous embodiments of my system have to do with a constant packet length, as in ATM networks. However my system can extend to the case of variable length real-time packets, as in 'pure' IP networks. First, the service quantum from the smoothing queue and the service quantum from the final queue can remain packet-based and need not become byte-based. So, smoothing control and adaptive tracking are unaffected (of course, queue depths will have to be dimen- sioned according to the L_max, the maximum real-time packet size). Second, we now have not one u_max, but many u_{max Ll}, where u_maxL, is the link bandwidth in packets/s for packets of length L_v

The main difference with fixed size packets is that we adopt cut-through service from final queue (84), while retaining store-and-forward service from smoothing queue (78). In cut-through service the transmission of a packet from the queue may start although the packet is not yet fully stored in the queue, while in store-and-forward service full buffering of a packet in the queue has to be completed for its transmission to begin. In this case, t^l _nor and t^' _xc denote the time instant that a head of a 'normal' and an 'excessive' packet of period n is entering final queue (84). Service starts exactly at the n^th sampling instant, for smoothing queue (78) and final queue (84). From smoothing queue (78) only packets that are complete are forwarded, while service from final queue (84) starts without need of full buffering. The arrival and departure times, without the decoupling of embodiment B are:

One may prove based on (22) to (25) that the same results hold for the worst case per hop delay and the worst case per hop delay jitter between any two packets that apply in embodiment A. With fixed length packets, transmission is back-to-back from final queue (84) either with cut- through service, or with store-and-forward service from final queue (84). This is due to the fact that the end of transmission from final queue (84) always coincides with the instant a full packet is received from smoothing queue (78), when both queues are serviced with the same rate. Hence, one can select cut-through service from final queue (84) for fixed length packets with no effect.

Next the procedures of smoothing and packet scheduling in pseudocode form for the embodiment of variable packet length without decoupling of delay and delay jitter from the control period are given: /* head of packet p arrives at the smoothing queue */

update (S_mQ_n ); forp=\ to S_mQ„ dequeue (smoothing_queue,p, u_{max L} );

else /*nh + d.me < (n + l)h */

n:=n+l; idle;

(i) smoothing on packet_arrival (real_time_queue,p) head of packet p arrives at the final queue enqueue (real_time_queue,p); on packet _s end (real_time_queue,p)

update ( SQ_n ); forp=\ to SQ„ dequeue_cut_through (real_time_queue,p, u_{max L} );

else /*nh + V ≤ time < (n + \)h */ l_=χ ^max,L_t n:=n+l idle

(ii) scheduling FIG. 6 shows the procedures in the form of timing diagrams. Regarding the smoothing diagram, for the n^th tracking period (106), transmission starts at the time instant nh, lasts for an in-

terval of time units (138) and ends at an instant (140) nh + ^" . While, in the _j=x ^umax,L, ^__j U_maxι_t scheduling diagram service from final queue (84) for the n^th tracking period (106) starts at the

instant nh, has a duration of units (142) and is completed at an instant

D. Constant 100% Allocation Utilization with Multiple Quality of Service Levels

The previous embodiments offer only one level of quality of a real-time service. However, there are cases where there is a need for flow differentiation in multiple levels of quality. The modified output queuing system supporting multiple quality levels for real-time traffic is shown in FIG. 7. Let the levels on which guarantees are offered be v in number.

The differences ofthe output queuing system of FIG. 7 with the one of FIG. 3 in what regards the structure are the following. In the place of one aggregate real-time packet traffic (76), now there are v real-time packet aggregates, a real-time packet aggregate (76-1) to (76-v), that are the inputs of the queuing system of the specific output port of the node. In the place of one device combination of smoothing queue (78)-smoothing queue server (80)-smoothing controller (82)- final queue (84)-adaptive tracker (88), in this embodiment there are v such device combinations: v distinct smoothing queues, a smoothing queue (78-1) to (78-v), v distinct smoothing queue servers, a smoothing queue server (80-1) to (80-v) that are driven by v distinct smoothing controllers, a smoothing controller (82-1) to (82-v), v identical final queues (84), and v distinct adaptive trackers, an adaptive tracker (88-1) to (88-v) that produce v distinct driving signals for the adaptive tracker, a driving signal (90-1) to (90-v).

The basic principle of operation is that the differentiation in the per hop packet delay and the per hop delay jitter between any two packets comes from the different smoothing windows (hence the distinct smoothing queues), that remain integer multiples of one tracking period, same for all levels. Having as basis a minimum value of tracking period (e.g. 125 μs) and a minimum ratio of smoothing period to tracking period (e.g. 40), by selecting a larger ratio than the minimum we can create different smoothing windows and consequently different per hop smoothing delays (e.g. 40-125 μs=5 ms, 44-125 μs=5.5 ms, 48-125 μs=6 ms, and so on). In fact, because the tracking period is the same for all levels, the tracking performance is not affected at all; on the contrary it is better with a larger smoothing window. Particularly, the differences of the output queuing system of FIG. 7 with the one of FIG. 3 in what regards the operation are the following. The synchronization is not anymore only between a smoothing controller, an adaptive tracker and the scheduler, but between v pairs of smoothing controllers-adaptive trackers and the scheduler. Adaptive tracking is on the one hand the same in its operation and in its period at all levels, differs on the other hand in the time instant on which it begins at every level (hence the distinct adaptive trackers (88-1) to (88-v)). At every level it starts with a time lag with relation to the previous level. The interval between is sufficient for the scheduler to transmit all the service quantum of the final queue of the previous level. As. it stands with one quality of service level, the transmission from the final queue occurs at the start of the tracking period, based on the computed service quantum and the maximum link bandwidth provisioned for the packet switching node. As in the case of one quality level, the peak rate policer at the edge nodes enforcing the user-service provider contract and the combination of smoothing controller-packet scheduler guarantee that the number of arrivals of aggregate real-time traffic of any level at the smoothing queue and therefore at the final queue also of any hop and during any control interval never exceeds the provisioned (by the service contract) quantum. The operation of the multi-level queuing system is shown in detail in the timing diagram of FIG. 8. All the levels have as tracking period the interval (106), but different sampling instants of tracking, a time instant (108-1) for the first level, a time instant (108-2) for the second level, to a time instant (108-v) for the v^th level. Time instant (108-2) follows time instant (108-1) after an interval (146-1), which covers the maximum provisioned service quantum for the first level. Time instant (108-3) follows time instant (108-2) after an interval (146-2), which covers the maximum provisioned service quantum for the second level. Similarly time instant (108-1) follows time instant (108-v) after an interval (146-v), which covers the maximum provisioned service quantum for the V^th (last) level. So, a scheduler (92') starts transmission of real-time packets of the first level from the final queue (84) of the first level at time instant (108-1) (or shortly after, depending on adoption of cut-through service or store-and-forward service respectively) that has a duration (148-1) to end at a time instant (150-1). In the worst case this duration may be equal to the interval (146-1). When the service of real-time packets of the first level ends and as long as there is time available until the start of service of real-time packets of the second level (108-2), the scheduler transmits packets of other services, besides the multi-level real-time service. The service of real-time packets ofthe second level from the final queue (84) ofthe second level lasts for an interval (148-2) with maximum value the interval (146-2) to end at a time instant (150-2) or in the worst case at the time instant (108-3). Similarly, when the service of realtime packets of the second level ends and as long as there is time available until the start of service of real-time packets ofthe third level (108-3), scheduler (92') transmits packets of other services, besides the multi-level real-time service. Finally, the service of real-time packets ofthe v^th level from the final queue (84) ofthe last level lasts for a duration (148-v) with maximum value the interval (146-v) to end at a time instant (150-v) or in the worst case at the time instant (108-1) thus closing the transmission cycle. Likewise, if there is time available until the start of service of real-time packets ofthe first level (108-1), scheduler (92') transmits packets of other services. Experimental Results

The efficiency of my system in adaptive tracking of real video traffic is depicted in the following experimental results. FIG. 9A shows an example of video packet traffic that results if we superimpose ten independent sources, that have been created with the rule given in reference Analysis, modeling and generation of self-similar video traffic, Mark Garrett and Walter Willinger, in the Proceedings of ACM SIGCOMM '94, London, Aug./Sept. 1994, by the MPEG-1 trace of the movie Star Wars coded with N=12, M=3, 25 frames per second and 384x288 pels resolution. I frame alignment has been chosen to investigate a worst case scenario. The trace is publicly available and is presented in reference Statistical Properties of MPEG video traffic and their impact on traffic modeling in ATM Systems, Oliver Rose, IEEE LCN '95, Minneapolis, Oct. 1995. One can tell from a visual examination that traffic is non-stationary, and it is known to exhibit trends, spikes and bursts in multiple time scales making its management with static methods difficult. FIG. 9B shows the diagram ofthe autocorrelation function for the aggregate traffic trace. Despite aggregation the autocorrelation decreases very slowly, even more slowly than in the autocorrelation function of the original aggregate; this means that there are long range bursts that, if not serviced will overload network buffers. Additionally, the pseudo-periodic component caused by the periodic coding of I frames in MPEG, creates on top of that the difficult problem for the network ofthe alignment of I frames and related bursts.

Regarding FIG. 10 A, the final queue occupancy (86) of the node EN1 (50) when scheduling is driven by adaptive tracking is shown. This result comes from a simulation with the above trace often aggregated Star Wars sources as load for the topology of FIG. 1. The smoothing period in this example is 40 ms, the period of tracking-scheduling is 1 ms and the target B of the adaptive tracker is 2 packets (one level of quality is considered). The ARMAX model ofthe closed loop system has k=2 and m=l. The variable forgetting factor is λ_n = 0.6A_n__x + 0.4 and A₀ =0.65. It is obvious that tracking is stable and accurate, with a constant variance around the target of only one packet and that the occupancy never goes to zero. The goal of 2 packets is the minimum possible for which one can have always v„>0, when the negative prediction error is well conditioned. These imply constant 100% utilization of the allocation and maximum delay and delay jitter at final queue (84) smaller than the tracking period. The same tracking performance is also observed in FIG. 10B, a diagram of the final queue occupancy (86) of the node CN1 (52) when scheduling is driven by adaptive tracking. In fact the same results are demonstrated for the other nodes of the topology also. As for the bandwidth allocation itself, a decreasing trend in the maximum service rate is observed as more hops the aggregate traffic traverses (16.378 Mb/s for the node EN1, 12.870 Mb/s for the node CN1, 11.965 Mb/s for the node CN2, 11.132 Mb/s for the node EN2, and so on). This is a very important fact, showing that the combination of smooth- ing-adaptive tracking leads to a phenomenon of reduction or 'weakening' of traffic burstiness and subsequent control easiness.

Output Queuing System Supervision

For the implementation of the queuing system of the previous embodiments of my invention, which are based on adaptive tracking, the handling of the initialization phase and the loss of excitation phase are very important. The detailed block diagram for one quality level that can handle these two phases is shown in FIG. 11 A. An adaptive tracker (88') producing a scheduler- driving signal (90'), apart from the final queue occupancy (86) receives also a loss of excitation signal (152) from a smoothing controller (82'). The detailed structure of adaptive tracker (88') is shown in FIG. 1 IB. In addition to adaptive tracker (88), identical to the one of FIG. 3, a virtual queue (154) and a supervision unit (156) that produces scheduler-driving signal (90') are shown. Virtual queue (154) is a counter the value of which is set by supervision unit (156) with a virtual queue set signal (158) and the 'occupancy' of which is read with a virtual queue occupancy signal (160). By means of loss of excitation signal (152), smoothing controller (82') notifies that at the current sampling instant the arrivals at the final queue A_n (service quantum from smoothing queue (78)) have been computed as null. The supervision unit receives also final queue occupancy (86). A signal (164) is the so-called 'equivalent queue' occupancy that supervision unit calculates as the sum of final queue occupancy (86) and virtual queue occupancy (160). Returning from adaptive tracker (88) to supervision unit (156) is an allocation of service of equivalent queue (162) based on which the supervision unit computes scheduler-driving signal (90'). Supervision unit (156) has a period of operation equal to the one of adaptive tracker (88), is synchronized with adaptive tracker (88) and a flow diagram analyzing the procedures adhered to during the initialization, the loss of excitation and the non-zero excitation phase is depicted in FIG. 1 lC. From the start of system operation in a block (168), supervision unit (156) follows the initiali- zation procedure for a time interval T_jnit, say for example an interval of 5 s. During this interval the node does not accept real packet traffic and the goal of the supervision unit is to lead the adaptive tracker to the occupancy target B with virtual traffic. In this way the use of real traffic for the transitional convergence of the adaptive tracker is avoided, for which it is probable that the delay and delay jitter bounds were larger than the equivalent ones ofthe steady state. So, in a block (172) with final queue (84) disconnected and therefore final queue occupancy equal to 0 packets, the virtual queue occupancy (and therefore the equivalent queue occupancy also through returning signal (160)) is increased by C packets every sampling instant (virtual arrival of C packets per period), with virtual queue set signal (158). The equivalent queue occupancy is given as input to adaptive tracker (88) with signal (164). The adaptive tracker computes how many vir- tual packets must be transmitted from virtual queue (154) so that the equivalent queue occupancy received as input is equal to the target B and the supervision unit sets accordingly (increases or decreases or leaves unchanged) the counter- virtual queue (154) with signal (158). This continues for an interval T_intt at every sampling instant (174). The initialization phase completed, the scheduler is connected to the final queue in order to start transmission of real packets, block (176). At every sampling instant, supervision unit (156) checks whether the arrivals at the final queue have been nullified, block (178). If this is the case (loss of excitation phase), the adaptive tracker will come to 'freeze' at the operating point [final queue occupancy, service rate]=[i?±ε, 0]. There is a possibility then that real-time packets are 'trapped' at the final queue for a duration larger than one control period. On this account, as soon as supervision unit (156) is notified that arrivals at final queue (84) have been nullified, it enforces on the scheduler to transmit all packets possibly left at final queue (84) and sets equivalent queue occupancy equal to B, block (180). If the arrivals are non-zero which constitutes the normal mode of operation, then it instructs the scheduler to transmit from final queue (84) as much of service quantum as the adaptive tracker computed, provided this is less than or equal from final queue occupancy (86). If this is larger than final queue occupancy (86), it transmits all the content of final queue (84) and continues by further decreasing virtual queue counter (154), so that the equivalent queue occupancy takes as new value the one it had before minus the computed service quantum, block (182).

In FIG. 12 the initialization is shown when the occupancy target is B=2 packets, the virtual arrivals are C=l packet per period and the initialization interval is T_init- 5 s. The latter has a dura- tion of 1 ms. The fast and accurate convergence to the occupancy target, achieved with the procedure of virtual arrivals is evident. In FIG. 13 A and 13B the phenomenon of 'freezing' due to loss of excitation (nullification of arrivals at final queue (84)) for the interval 100<t<200 is shown, having as load for the rest of the time the Star Wars trace of FIG. 9A. In FIG. 13A it is obvious that the occupancy during this interval fixes on the value B minus maximum negative prediction error, with an occupancy target of 2 packets. In FIG. 13B it is obvious that the service rate is nullified when the excitation is lost and remains null for as long as the arrivals are null. E. Selectable Allocation Utilization Target with One Quality of Service Level

In FIG. 14 a block diagram of the queuing system in the embodiment of less-than-maximum 5 allocation utilization target is shown. The difference with FIG. 3 is that in the place of adaptive tracker (88) there is an adaptive predictor (184) that produces a scheduler control output (186). This unit predicts the smoothed arrivals of real-time aggregate traffic A_n and taking the prediction as if it were the real value, computes, taking into account also the final queue occupancy- backlog from the previous period, such an allocation that a specific utilization in every prediction 0 interval is ensured. Therefore, the target now is not the occupancy of a buffer but the utilization U itself, U<\. How close to the target the system gets depends on the accuracy ofthe prediction.

Theory of operation of adaptive prediction and scheduling

5 The model Here is a time delayed ARMA:

C_q A_n =D_q e_n , ₍26₎

, k-l . , k-1

C_g - q + ∑c^q¹ and D_g = q + ∑d_k__tq^l . »=o .=0 where e_n axe independent and uniformly distributed zero-mean Gaussian variables. Again ELS identification with a variable forgetting factor is used. Here, ^θ _n ^T A^cι_,n - _" ^c _{k,n >} dι_,„

and (27)

Φ = l- y_n-\ ^{■ ■ ■}- y_n-k - ^en-l ^{■ ■ ■ e}n-kl

The service quantum that takes into account the possible backlog at the final queue from the previous interval y_n is:

P„ = y„ + CA_n , where (28)

C = U^~ , U the utilisation target. 0

The following result holds for the bandwidth allocation utilization:

I. The bandwidth allocation utilization U_n is constant inside the allocation interval n and exhibits a variation between successive intervals around the target U, and the maximum per hop packet delay and delay jitter between any two packets is smaller than h. 5 FIG. 15 shows a timing diagram for scheduling based on adaptive prediction, with constant packet length. Selecting store-and-forward service from final queue (84), for the n^th prediction period, service starts at a time instant nh + (114), has a duration (188) P„ time units and is completed at a time instant (190) nh + (P„ + 1) —

In FIG. 16 the instantaneous bandwidth allocation utilization is depicted when the target is £7=0.9. Utilization is not constant but varies around the target with very good transient and steady state behavior.

Conclusions, Ramifications and Scope Thus the reader will see that the system and the method of real-time traffic transfer of my invention provide a unified rational framework for strict quantitative guarantees on quality of service and, for the first time, constant maximum and selectable less-than-maximum bandwidth al- location utilization in multi-service packet switching networks that will make economic deployment of real-time applications in such networks feasible, reducing considerably the cost of using the network services.

While my above invention contains many specificities, these should not be construed as limitations to the scope of the invention, but rather as an exemplification of embodiments thereof. Many other variations are possible.

For example, estimation methods other than ELS could be used in adaptive tracking as well as in adaptive prediction, or, other method of controller design in adaptive tracking. As an example of other method for adaptive tracking, we mention the use of not one model of the closed loop system, but more than one, and the related technique of multiple models switching and tuning. Moreover, the smoothing interval could be reduced with relation to the control period with the introduction of a variable forgetting factor that is not deterministic but is driven by a neural network with the goal of reducing the adjustment error. Also the guarantees offered may be combined in many other ways. Thus, the system may offer:

■ Constant 100% allocation utilization with one level of guarantees for variable packet length and with decoupling of delay and delay jitter from the tracking period;

■ Constant 100% allocation utilization with multiple levels of guarantees for constant packet length and with decoupling of delay and delay jitter from the tracking period;

■ Constant 100% allocation utilization with multiple levels of guarantees for variable packet length and with decoupling of delay and delay jitter from the tracking period; ■ Selectable less-than-maximum allocation utilization with one quality level and variable packet length;

■ Selectable less-than-maximum allocation utilization with multiple quality levels and constant packet length; ■ Selectable less-than-maximum allocation utilization with multiple quality levels and variable packet length;

■ Selectable less-than-maximum allocation utilization with one quality level, constant packet length and with decoupling of delay and delay jitter from the prediction period; ■ Selectable less-than-maximum allocation utilization with one quality level, variable packet length and with decoupling of delay and delay jitter from the prediction period;

■ Selectable less-than-maximum allocation utilization with multiple quality levels, constant packet length and with decoupling of delay and delay jitter from the prediction period;

■ Selectable less-than-maximum allocation utilization with multiple quality levels, variable packet length and with decoupling of delay and delay jitter from the prediction period.

Furthermore, for the attainment of selectable less-than-maximum allocation utilization, adaptive tracking may be used in the place of adaptive prediction, with control target the utilization U itself.

Accordingly, the scope of the invention should be determined not by the embodiments illus- trated, but by the appended claims and their legal equivalents.

1. A system for transfer of real-time traffic in multi-service packet switching networks, comprising: (a) a peak rate policer disposed at edge network nodes

(b) an output queuing system disposed at all network nodes receiving for transmission the real-time packet traffic originating from said peak rate policer whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as a constant 100% utilization ofthe bandwidth allocation are provided.

2. The system defined in Claim 1 wherein said output queuing system comprises:

(a) a smoothing queue where real-time packet traffic originating from said peak rate policer is stored

(b) a server of said smoothing queue

(c) a smoothing controller driving said server in transmitting real-time traffic from said smoothing queue

(d) a final queue storing traffic served from said smoothing queue

(e) an adaptive tracker computing the service quantum to be allocated at said final queue based on the occupancy of said final queue

(f) a packet scheduler allocating the total link bandwidth to said final queue and to queues of other supported services, at least the best-effort service, driven by said adaptive tracker.

3. The system defined in Claim 2 wherein the packet length of the real-time traffic originating from said peak rate policer is constant.

4. The system defined in Claim 2 wherein the packet length of the real-time traffic originating from said peak rate policer is variable.

5. The system defined in Claim 3, further including:

(a) a plurality of distinct smoothing queues where real-time packet traffic of multiple quality of service levels originating from said peak rate policer is stored

(b) a plurality of servers of said distinct smoothing queues (c) a plurality of smoothing controllers with predetermined distinct smoothing windows driving said servers in transmitting real-time traffic from said distinct smoothing queues (d) a plurality of final queues storing traffic served from said distinct smoothing queues

(e) a plurality of adaptive trackers computing the service quantum to be allocated at said final queues based on the occupancy of said final queues

(f) a packet scheduler allocating the total link bandwidth to said final queues and to queues of other supported services, at least the best-effort service, driven by said adaptive trackers whereby a plurality of levels of end-to-end deterministic bounds on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

6. The system defined in Claim 4, further including:

(b) a plurality of servers of said distinct smoothing queues

(c) a plurality of smoothing controllers with predetermined distinct smoothing windows driving said servers in transmitting real-time traffic from said distinct smoothing queues

(d) a plurality of final queues storing traffic served from said distinct smoothing queues

(f) a packet scheduler allocating the total link bandwidth to said final queues and to queues of other supported services, at least the best-effort service, driven by said adaptive trackers whereby a plurality of levels of end-to-end deterministic bounds on the delay and the delay jitter, as well as a constant 100% utilization ofthe bandwidth allocation are provided.

7. The system defined in Claim 2, further including:

(a) a virtual queue

(b) a supervision unit having as inputs the occupancy of said final queue and a loss of excitation signal from said smoothing controller, interfacing with said virtual queue and said adaptive tracker and producing the scheduler-driving signal whereby the initialization of the adaptive tracker without real traffic and the handling of real traffic in the loss-of-excitation phase are achieved. A method for transfer of real-time traffic in multi-service packet switching networks, comprising the steps of:

(a) peak rate policing at edge network nodes

(b) queuing and service at all network nodes of real-time packet traffic been subjected to said peak rate policing whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as a constant 100% utilization ofthe bandwidth allocation are provided.

9. The method defined in Claim 8 wherein said queuing and service at all network nodes of real-time packet traffic been subjected to said peak rate policing comprises the steps of: (a) smoothing of real-time traffic arriving at said smoothing queue

(b) adaptive tracking ofthe occupancy of said final queue

(c) service from said final queue with a bandwidth allocation computed by said adaptive tracking.

10. The method defined in Claim 9 wherein the steps of said smoothing, of said adaptive tracking and of said service from said final queue are performed with constant packet length traffic.

11. The method defined in Claim 9 wherein the steps of said smoothing, of said adaptive tracking and of said service from said final queue are performed with variable packet length traffic.

12. The method defined in Claim 10, further including the steps of:

(a) a plurality of said smoothings of real-time traffic belonging to multiple quality levels

(b) a plurality of said adaptive trackings ofthe occupancy of said final queues

(c) service from said final queues with bandwidth allocations computed by said adaptive trackings whereby a plurality of levels of end-to-end deterministic bounds on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

13. The method defined in Claim 11, further including the steps of: (a) a plurality of said smoothings of real-time traffic belonging to multiple quality levels (b) a plurality of said adaptive trackings ofthe occupancy of said final queues (c) service from said final queues with bandwidth allocations computed by said adaptive trackings whereby a plurality of levels of end-to-end deterministic bounds on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation

5 are provided.

14. The method defined in Claim 9 wherein a means for decoupling the delay and the delay jitter from the period of said adaptive tracking is provided.

15. The method defined in Claim 9 wherein means for the initialization without real traffic and for the handling real traffic during the loss-of-excitation phase of said adaptive

10 tracking are provided.

16. A system for transfer of real-time traffic in multi-service packet switching networks, comprising:

(a) said peak rate policer disposed at edge network nodes

(b) an output queuing system disposed at all network nodes receiving for transmission the 15 real-time packet traffic originating from said peak rate policer whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as following of a selectable less-than-maximum bandwidth allocation utilization target.

17. The system defined in Claim 16, wherein said output queuing system comprises:

(a) said smoothing queue where real-time packet traffic originating from said peak rate 20 policer is stored

(b) said server of said smoothing queue >

(c) said smoothing controller driving said server in transmitting real-time traffic from said smoothing queue

(d) said final queue storing traffic served from said smoothing queue

25 (e) an adaptive predictor computing the service quantum to be allocated at said final queue based on the occupancy of said final queue

(f) said packet scheduler allocating the total link bandwidth to said final queue and to queues of other supported services, at least the best-effort service, driven by said adaptive predictor.

30 18. A method for transfer of real-time traffic in multi-service packet switching networks, comprising the steps of:

(a) said peak rate policing at edge network nodes

(b) queuing and service at all network nodes of real-time packet traffic been subjected to said peak rate policing whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as following of a selectable less-than-maximum bandwidth allocation utilization target are provided.

19. The method defined in Claim 18 wherein said queuing and service at all network nodes of real-time packet traffic been subjected to said peak rate policing comprises the steps of:

(a) smoothing of real-time traffic arriving at said smoothing queue

(b) adaptive prediction of arrivals at said final queue

(c) service from said final queue with a bandwidth allocation computed by taking into ac- count said adaptive prediction and possible backlog in the occupancy of said final queue.

Claims

AMENDED CLAIMS[received by the International Bureau on 25 January 2002 (25.01.02); claims 1-19 cancelled; replaced by new claims 1-16 (4 pages)]

1. A system for transfer of real-time traffic in multi-service packet switching networks, having a peak rate policer disposed at edge network nodes and an output queuing system disposed at all network nodes receiving for transmission the real-time packet traffic originating from said peak rate policer, said output queuing system comprising:

(a) a transmission queue storing real-time packet traffic

(b) means for tracking adaptively and autonomously at every network node a predetermined non-zero reference target for the occupancy of said transmission queue whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

2. The system defined in Claim 1 wherein said means for tracking comprises:

(a) a smoothing queue where real-time packet traffic originating from said peak rate policer is stored (b) a server of said smoothing queue

(c) a smoothing controller with predetermined smoothing window driving said server in transmitting real-time traffic from said smoothing queue to said transmission queue

(d) an adaptive tracker computing the service quantum to be allocated at said transmission queue based on the occupancy of said transmission queue (e) a packet scheduler allocating the total link bandwidth to said transmission queue and to queues of other supported services, at least the best-effort service, driven by said adaptive tracker.

3. The system defined in Claim 2 wherein said smoothing controller and said packet scheduler contain means for processing real-time traffic of constant packet length.

4. The system defined in Claim 2 wherein said smoothing controller and said packet scheduler contain means for processing real-time traffic of variable packet length.

5. The system defined in Claim 1, further including:

(a) a plurality of transmission queues storing real-time packet traffic of multiple quality of service levels (b) a plurality of means for tracking adaptively and autonomously at every network node a predetermined non-zero reference target for the occupancy of said transmission queues whereby a plurality of levels of end-to-end deterministic bounds on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

6. The system defined in Claim 5 wherein said plurality of means for tracking comprises:

(a) a plurality of smoothing queues where real-time packet traffic of multiple quality of service levels originating from said peak rate policer is stored (b) a plurality of servers of said distinct smoothing queues

(c) a plurality of smoothing controllers with predetermined distinct smoothing windows driving said servers in transmitting real-time traffic from said smoothing queues

(d) a plurality of transmission queues storing traffic served from said smoothing queues (e) a plurality of adaptive trackers computing the service quantum to be allocated at said transmission queues based on the occupancy of said transmission queues (f) a packet scheduler allocating the total link bandwidth to said transmission queues and to queues of other supported services, at least the best-effort service, driven by said adaptive trackers.

7. The system defined in Claim 2, further including:

(a) a virtual queue

(b) a supervision unit having as inputs the occupancy of said transmission queue and a loss of excitation signal from said smoothing controller, interfacing with said virtual queue and said adaptive tracker and producing the packet scheduler-driving signal whereby the initialization of the adaptive tracker without real traffic and the handling of real traffic in the loss-of-excitation phase are achieved.

8. The system defined in Claim 2, further including means for decoupling the delay and the delay jitter from the period of said adaptive tracker.

9. A method for transfer of real-time traffic in multi-service packet switching networks, by peak rate policing at edge network nodes, and output queuing at all network nodes of real-time packet traffic been originally subjected to said peak rate policing, said output queuing comprising:

(a) providing a transmission queue which is able to store real-time packet traffic

(b) tracking adaptively and autonomously at every network node a predetermined non-zero reference target for the occupancy of said transmission queue whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

10. The method defined in Claim 9 wherein said tracking comprises: (a) providing a smoothing queue where real-time packet traffic subjected to said peak rate policing is stored

(b) smoothing real-time traffic arriving at said smoothing queue with a predetermined smoothing window (c) providing an adaptive tracker computing the service quantum to be allocated at said transmission queue receiving real-time traffic from said smoothing queue based on the occupancy of said transmission queue

(d) scheduling said transmission queue with a bandwidth allocation computed by said adaptive tracker.

11. The method defined in Claim 10 wherein said smoothing and said scheduling are performed on constant length packets.

12. The method defined in Claim 10 wherein said smoothing and said scheduling are performed on variable length packets.

13. The method defined in Claim 9, further including: (a) providing a plurality of transmission queues which are able to store real-time packet traffic of multiple quality of service levels

(b) a plurality of trackings adaptively and autonomously at every network node a predetermined non-zero reference target for the occupancy of said transmission queues whereby a plurality of levels of end-to-end deterministic bounds on the delay and the de- lay jitter, as well as a constant 100% utilization of the bandwidth allocation are provided.

14. The method defined in Claim 13 wherein said plurality of trackings comprises:

(a) providing a plurality of smoothing queues where real-time packet traffic of multiple quality of service levels subjected to said peak rate policing is stored

(b) a plurality of smoothings real-time traffic belonging to multiple quality levels ar- riving at said smoothing queues with predetermined distinct smoothing windows

(c) providing a plurality of adaptive trackers computing the service quantum to be allocated at said transmission queues based on the occupancy of said transmission queues

(d) scheduling said transmission queues receiving real-time traffic from said smooth- ing queues with a bandwidth allocation computed by said adaptive trackers.

15. A system for transfer of real-time traffic in multi-service packet switching networks, having a peak rate policer disposed at edge network nodes and an output queuing system disposed at all network nodes receiving for transmission the real-time packet traffic originating from said peak rate policer, said output queuing system comprising:

(c) a smoothing controller with predetermined smoothing window driving said server in transmitting real-time traffic from said smoothing queue

(d) a transmission queue storing traffic served from said smoothing queue

(e) an adaptive predictor computing the service quantum to be allocated at said transmission queue based on the arrivals at said transmission queue and possible backlog in the occupancy of said transmission queue

(f) a packet scheduler allocating the total link bandwidth to said transmission queue and to queues of other supported services, at least the best-effort service, driven by said adaptive predictor whereby an end-to-end deterministic bound on the delay and the delay jitter, as well as following of a selectable less-than-maximum bandwidth allocation utilization target.

16. A method for transfer of real-time traffic in multi-service packet switching networks, by peak rate policing at edge network nodes, and output queuing at all network nodes of real-time packet traffic been originally subjected to said peak rate policing, said output queuing comprising:

(a) providing a smoothing queue where real-time packet traffic subjected to said peak rate policing is stored

(b) smoothing real-time traffic arriving at said smoothing queue with a predetermined smoothing window (c) providing a transmission queue receiving real-time traffic from said smoothing queue

(d) providing an adaptive predictor computing the service quantum to be allocated at said transmission queue based on the arrivals at said transmission queue and possible backlog in the occupancy of said transmission queue

(e) scheduling said transmission queue with a bandwidth allocation computed by taking into account said adaptive predictor.