DOUBLE-ENDED QUEUE IN A CONTIGUOUS ARRAY WITH CONCURRENT NON-BLOCKING
INSERT AND REMOVE OPERATIONS
TECHNICAL FIELD
The present invention relates to coordination amongst processors in a multiprocessor computer, and more particularly, to structures and techniques for facilitating non-blocking access to concurrent shared objects
Background Art
Non-blocking algorithms can deliver significant performance benefits to parallel systems However, there is a growing realization that existing synchronization operations on single memory locations, such as compare-and-swap (CAS), are not expressive enough to support design of efficient non-blocking algorithms As a result, stronger synchronization operations are often desired One candidate among such operations is a double-word compare-and-swap (DCAS) If DCAS operations become more generally supported in computers systems and, in some implementations, in hardware, a collection of efficient current data structure implementations based on the DCAS operation will be needed
Massalin and Pu disclose a collection of DCAS-based concurrent algorithms See e g H Massalin and C Pu, A Lock-Free Multiprocessor OS Kernel, Technical Report TR CUCS-005-9, Columbia University, New York, NY, 1991, pages 1-19 In particular, Massalin and Pu disclose a lock- free operating system kernel based on the DCAS operation offered by the Motorola 68040 processor, implementing structures such as stacks, FIFO-queues, and linked lists Unfortunately, the disclosed algorithms are centralized in nature In particular, the DCAS is used to control a memory location common to all operations, and therefore limits overall concurrency
Greenwald discloses a collection of DCAS-based concurrent data structures that improve on those of Massalin and Pu See e g M Greenwald Non-Blocking Synchronization and System Design, Ph D thesis, Stanford University Technical Report STAN-CS-TR-99-1624, Palo Alto, CA, 8 1999, 241 pages In particular, Greenwald discloses implementations of the DCAS operation in software and hardware and discloses two DCAS-based concurrent double-ended queue (deque) algorithms implemented using an array Unfortunately, Greenwald's algorithms use DCAS in a restrictive way The first, described m Greenwald, Non-Blocking Synchronization and System Design, at pages 196- 197, used a two-word DCAS as if it were a three-word operation, storing two deque end pointers in the same memory word, and performing the DCAS operation on the two pointer word and a second word containing a value Apart from the fact that Greenwald's algorithm limits applicability by cutting the index range to half a memory word, it also prevents concurrent access to the two ends of the deque Greenwald's second algorithm, described in Greenwald, Non-Blocking Synchronization and System Design, at pages 217-220) assumes an array of unbounded size, and does not deal with classical array-based issues such as detection of when the deque is empty or full
Arora et al disclose a CAS-based deque with applications m job-steahng algorithms See e g N S
Arora, Blumofe, and C G Plaxton, Thread Scheduling For Multφrogrammed Multiprocessors, m
Proceedings of the 10th Annual ACM Sy nposmm on Parallel Algorithms and Architectures, 1998 Unfortunately, the disclosed non-blockin g implementation restricts one end of the deque to access by only a single processor and restricts the other erd to only pop operations
Accordingly, improved techniques are desired that do not suffer from the above-descπbed drawbacks of prior approaches
DISCLOSURE OF INVENTION
A set of structures and techniques are described herein whereby an exemplary concurrent shared object, namely a double-ended queue (deque), is provided Although a described non-blocking, hneaπzable deque implementation exemplifies several advantages of realizations in accordance with the present invention, the present invention is not limited thereto Indeed, based on the description herein and the claims that follow, persons of ordinary skill in the art will appreciate a variety of concurrent shared object implementations For example, although the described deque implementation exemplifies support for concurrent push and pop operations at both ends thereof, other concurrent shared objects implementations in which concurrency requirements are less severe, such as LIFO or stack structures and FIFO or queue structures, may also be implemented using the techniques described herein
Accordingly, a novel array-based concurrent shared object implementation has been developed that provides non-blocking and hneaπzable access to the concurrent shared object In an application of the underlying techniques to a deque, the array-based algorithm allows uninterrupted concurrent access to both ends of the deque, while returning appropriate exceptions in the boundary cases when the deque is empty or full An interesting characteristic of the concurrent deque implementation is that a processor can detect these boundary cases, e g , determine whether the array is empty or full, without checking the relative locations of the two end pointers in an atomic operation
BRIEF DESCRIPTION OF DRAWINGS
The present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings
FIGS. 1A and IB illustrate exemplary empty and full states of a double-ended queue (deque) implemented as an array in accordance with the present invention
FIG. 2 illustrates successful operation of a pop_rιght operation on a partially full state of a deque implemented as an array in accordance with the present invention
FIG. 3 illustrates successful operation of a push_πght operation on a empty state of a deque implemented as an array in accordance with the present invention
FIG. 4 illustrates contention between opposing pop_lef t and pop_rιght operations for a single
remammg element in an almost empty state of a deque implemented as an array in accordance with the present invention
FIGS. 5A, 5B and 5C illustrate the results of a sequence of push_lef t and push_rιght operations on a nearly full state of a deque implemented as an array in accordance with the present invention Following successful completion of the push_rιght operation, the deque is in a full state FIGS. 5A, 5B and 5C also illustrate an artifact of the linear depiction of a circular buffer, namely that, through a series of preceding operations, ends of the deque may wrap around such that left and right indices may appear (in the linear depiction) to the right and left of each other
The use of the same reference symbols in different drawings indicates similar or identical items
IVIODE(S) FOR CARRYING OUT THE INVENTION
The description that follows presents a set of techniques, objects, functional sequences and data structures associated with concurrent shared object implementations employing double compare-and-swap (DCAS) operations in accordance with an exemplary embodiment of the present invention An exemplary non-blocking, linearizable concurrent double-ended queue (deque) implementation is illustrative A deque is a good exemplary concurrent shared object implementation, in that it involves all the intricacies of LIFO-stacks and FIFO-queues, with the added complexity of handling operations originating at both of the deque's ends Accordingly, techniques, objects, functional sequences and data structures presented in the context of a concurrent deque implementation will be understood by persons of ordinary skill in the art to describe a superset of support and functionality suitable for less challenging concurrent shared object implementations, such as LIFO-stacks, FIFO-queues or concurrent shared objects (including deques) with simplified access semantics
In view of the above, and without limitation, the description that follows focuses on an exemplary linearizable, non-blocking concurrent deque implementation which behaves as if access operations on the deque are executed in a mutually exclusive manner, despite the absence of a mutual exclusion mechanism Advantageously, and unlike prior approaches, deque implementations in accordance with some embodiments of the present invention allow concurrent operations on the two ends of the deque to proceed independently
Computational Model
One realization of the present invention is as a deque implementation, employing the DCAS operation, on a shared memory multiprocessor computer This realization, as well as others, will be understood m the context of the following computation model, which specifies the concurrent semantics of the deque data structure
In general, a concurrent system consists of a collection of n processoi s Processors communicate through shared data structures called objects Each object has an associated set of primitive operations that provide the mechanism for manipulating that object Each processor P can be viewed in an abstract sense as a
sequential thread of control that applies a sequence of operations to objects by issuing an invocation and receiving the associated response A history is a sequence of invocations and responses of some system execution Each history induces a "real-time" order of operations where an operation A precedes another operation B, if A's response occurs before B's invocation Two operations are concurrent if they are unrelated by the real-time order A sequential history is a history in which each invocation is followed immediately by its corresponding response The sequential specification of an object is the set of legal sequential histories associated with it The basic correctness requirement for a concurrent implementation is lineanzability Every concurrent history is "equivalent" to some legal sequential history which is consistent with the real-time order induced by the concurrent history In a linearizable implementation, an operation appears to take effect atomically at some point between its invocation and response In the model described herein, a shared memory location L of a multiprocessor computer's memory is a linearizable implementation of an object that provides each processor P, with the following set of sequentially specified machine operations
Read, (L) reads location L and returns its value
Write, (L v) writes the value v to location L DCAS, (LI, L2, o\, o2, n \, «2) is a double compare-and-swap operation with the semantics described below
Implementations described herein are non-blocking (also called lock-free) Let us use the term higher-level operations in referring to operations of the data type being implemented, and lower-level opeiations in referring to the (machine) operations in terms of which it is implemented A non-blocking implementation is one in which even though individual higher-level operations may be delayed, the system as a whole continuously makes progress More formally, a non-blocking implementation is one in which any history containing a higher-level operation that has an invocation but no response must also contain infinitely many responses concurrent with that operation In other words, if some processor performing a higher-level operation continuously takes steps and does not complete, it must be because some operations invoked by other processors are continuously completing their responses This definition guarantees that the system as a whole makes progress and that individual processors cannot be blocked, only delayed by other processors continuously taking steps Using locks would violate the above condition, hence the alternate name lock- free
Double-word Compare-and-Swap Operation Double-word compare-and-swap (DCAS) operations are well known in the art and have been implemented in hardware, such as in the Motorola 68040 processor, as well as through software emulation Accordingly, a variety of suitable implementations exist and the descriptive code that follows is meant to facilitate later description of concurrent shared object implementations in accordance with the present invention and not to limit the set of suitable DCAS implementations For example, order of operations is merely illustrative and any implementation with substantially equivalent semantics is also suitable
Furthermore, although exemplary code that follows includes overloaded variants of the DCAS operation and facilitates efficient implementations of the later described push and pop operations, other implementations,
lncludtng single variant implementations may also be suitable
boolean DCAS (val *addrl, val *addr2, val oldl, val old2, val newl, val new2) { atomically { if ( (*addrl==oldl) && (*addr2==old2) ) { *addrl = newl; *addr2 = new2 ; return true ; } else { return false; }
}
boolean DCAS (val *addrl, val *addr2, val oldl, val old2, val *newl, val *new2) { atomically { tempi = *addrl; temp2 = *addr2; if ((tempi == oldl) && (temp2 == old2)) *addrl = *newl; *addr2 = *new2; *newl = tempi ; *new2 = temp2; return true; } else {
*newl = tempi ; *new2 = temp2 ; return false; }
Note that in the exemplary code, the DCAS operation is overloaded, I e , if the last two arguments of the DCAS operation (newl and new2) are pointers, then the second execution sequence (above) is operative and the original contents of the tested locations are stored into the locations identified by the pointers In this way, certain invocations of the DCAS operation may return more information than a success/failure flag
The above sequences of operations implementing the DCAS operation are executed atomically using support suitable to the particular realization For example, in various realizations, through hardware support (e g , as implemented by the Motorola 68040 microprocessor or as described in M Herlihy and J Moss,
Transactional memory Architectural Support For Lock-Free Data Structures, Technical Report CRL 92/07, Digital Equipment Corporation, Cambridge Research Lab, 1992, 12 pages), through non-blocking software emulation (such as described in G Barnes, A Method For Implementing Lock-Free Shared Data Structures, in Proceedings of the 5th ACM Symposium on Parallel Algorithms and Architectures, pages 261-270, June 1993 or in N Shavit and D Touitou, Software transactional memory, Distributed Computing, 10(2) 99-1 16, February 1997), or via a blocking software emulation
Although the above-referenced implementations are presently preferred, other DCAS
lmplementations that substantially presei ve the semantics of the descriptive code (above) are also suitable Furthermore, although much of the descnption herein is focused on double-word compare-and-swap (DCAS) operations, it will be understood that N- catio'i compare-and-swap operations (N > 2) may be more generally employed, though often at some increased overhead
A Double-ended Queue (Deque)
A deque object S is a concurrent shared object, that in an exemplary realization is created by an operation of a constructor operation, e g , make_deque ( length_s ) , and which allows each processor P„ 0 ≤ ι ≤ n - 1, of a concurrent system to perform the following types of operations on S push_rιght1 ( v) , push_lef tx ( v) , pop_rιght2 ( ) , and pop_lef tα ( ) Each push operation has an input, v, where v is selected from a range of values Each pop operation returns an output from the range of values Push operations on a full deque object and pop operations on an empty deque object return appropriate indications
A concurrent implementation of a deque object is one that is linearizable to a standard sequential deque This sequential deque can be specified using a state-machine representation that captures all of its allowable sequential histories These sequential histories include all sequences of push and pop operations induced by the state machine representation, but do not include the actual states of the machine In the following description, we abuse notation slightly for the sake of clarity
The state of a deque is a sequence of items S = (v0 , ,vk) from the range of values, having cardinality 0 < I S\ ≤ length_S The deque is initially in the empty state (following invocation of make_deque ( 1 ength_S ) ), that is, has cardinality 0, and is said to have reached a full state if its cardinality is length_S
The four possible push and pop operations, executed sequentially, induce the following state transitions of the sequence S = (v0, ,vk), with appropriate returned values
push_rιght ( new) if S is not full, sets S to be the sequence S = (v0, ,vk,vnm) push_left (vnew) if 5 is not full, sets S to be the sequence S = (vmw,vQ, ,vk) pop_rιght ( ) if 5" is not empty, sets S to be the sequence S = (v0. ,vk y) pop_left ( ) lfS is not empty, sets S to be the sequence 5 = {v ,vk)
For example, starting with an empty deque state, 5 = (), the following sequence of operations and corresponding transitions can occur A push_rιght ( 1 ) changes the deque state to S = (1) A push_lef t (2 ) subsequently changes the deque state to S = (2, 1) A subsequent push_πght (3 ) changes the deque state to S = (2,1,3) Finally, a subsequent pop_πght ( ) changes the deque state to 5 = (2, 1)
An Array-Based Implementation
The description that follows presents an exemplary non-blocking implementation of a deque based on an underlying contiguous array data structure wherein access operations (illustratively, push_lef t, pop_lef t, push_πght and pop_πght) employ DCAS operations to facilitate concurrent access Exemplary code and illustrative drawings will provide persons of ordinary skill in the art with detailed understanding of one particular realization of the present invention, however, as will be apparent from the description herein and the breadth of the claims that follow, the invention is not limited thereto Exemplary right-hand-side code is described in substantial detail with the understanding that left-hand-side operations are symmetric. Use herein of directional signals (e g , left and right) will be understood by persons of ordinary skill in the art to be somewhat arbitrary Accordingly, many other notational conventions, such as top and bottom, first-end and second-end, etc , and implementations denominated therein are also suitable
With the foregoing in mind, an exemplary non-blocking implementation of a deque based on an underlying contiguous array data structure is illustrated with reference to FIGS. 1A and IB In general, an array-based deque implementation includes a contiguous array S [0 . . length_S - 1] of storage locations indexed by two counters, R and L The array, as well as the counters (or alternatively, pointers or indices), are typically stored in memory. Typically, the array S and indices R and L are stored in a same memory, although more generally, all that is required is that a particular DCAS implementation span the particular storage locations of the array and an index
In operations on S, we assume that mod is the modulus operation over the integers (e g , - 1 mod 6 = 5, - 2 mod 6 = 4, and so on) Henceforth, in the description that follows, we assume that all values of R and L are modulo length_S, which implies that the array S is viewed as being circular The array S [ 0 . . length_S - 1] can be viewed as if it were laid out with indexes increasing from left to right We assume a distinguishing value, e g , "null" (denoted as 0 in the drawings), not occurring in the range of real data values for S Of course, other distinguishing values are also suitable
Operations on S proceed as follows Initially, for empty deque state, L points immediately to the left of R In the illustrative embodiment, indices L and R always point to the next location into which a value can be inserted If there is a null value stored in the element of S immediately to the right of that identified by L (or respectively, in the element of S immediately to the left of that identified by R), then the deque is in the empty state Similarly, if there is a non-null value in the element of S identified by L (respectively, in the element of S identified by R), then the deque is m the full state FIG. 1A depicts an empty state and FIG. IB depicts a full state During the execution of access operations in accordance with the present invention, the use of a DCAS guarantees that on any location in the array, at most one processor can succeed in modifying the entry at that location from a "null" to a "non-null" value or vice versa
An illustrative pop_πght access operation in accordance with the present invention follows
val pop_πght { while (true) { oldR = R; newR = (oldR - 1) mod length_S; oldS = S [newR] ; if (oldS == "null") { if (oldR == R) if (DCAS(&R, &S [newR] , oldR, oldS, oldR, oldS) ) return "empty";
} else { newS = "null"; if (DCAS(&R, &S [newR] , oldR, oldS, &newR, &newS) ) return newS; else f (newR == oldR) { if (newS == "null") return "empty
}
}
To perform a pop_rιght, a processor first reads R and the location in S corresponding to R- 1 (Lines 3-5, above) It then checks whether S [R- l] is null As noted above, S [R- l ] is shorthand for S [R- 1 mod length_S] If S [R- l] is null, then the processor reads R again to see if it has changed (Lines 6- 7) This additional read is a performance enhancement added under the assumption that the common case is that a null value is read because another processor "stole" the item, and not because the queue is really empty Other implementations need not employ such an enhancement The test can be stated as follows if R hasn't changed and S [R- l] is null, then the deque must be empty since the location to the left of R always contains a value unless there are no items in the deque However, the conclusion that the deque is empty can only be made based on an instantaneous view of R and S [R- l] Therefore, the pop_πght implementation employs a DCAS (Lines 8-10) to check if this is in fact the case If so, pop_rιght returns an indication that the deque is empty If not, then either the value in S [R- l] is no longer null or the index R has changed In either case, the processor loops around and starts again, since there might now be an item to pop
If S [R- l ] is not null, the processor attempts to pop that item (Lines 12-20) The pop_rιght implementation employs a DCAS to try to atomically decrement the counter R and place a null value in S [R- 1 ] , while returning (via &newR and knewS) the old value in S [R- l] and the old value of the counter R (Lines 13-15) Note that the overloaded variant of DCAS described above is utilized here
A successful DCAS (and hence a successful pop_rιght operation) is depicted in FIG. 2 Initially,
S = (vt, v2, v3, vf) and L and R are as shown Contents of R and of S [R- l ] are read, but the results of the reads may not be consistent if an intervening competing access has successfully completed In the context of the deque state illustrated in FIG. 2, the competing accesses of concern are a pop_πght or a
push_πght, although in the case of an almost empty state of the deque, a pop_lef t might also intervene Because of the nsk of a successfully completed competing access, the pop_πght implementation employs a DCAS (lines 14-15) to check the instantaneous values of R and of S [R- l ] and, if unchanged, perform the atomic update of R and of S [R- l] resulting in a deque state of S = (v v2, v3)
If the DCAS is successful (as indicated in FIG. 2), the pop_πght returns the value v4 from
S [R- l] If it fails, pop_rιght checks the reason for the failure If the reason for the DCAS failure was that R changed, then the processor retries (by repeating the loop) since there may be items still left in the deque If R has not changed (Line 17), then the DCAS must have failed because S [R- 1 ] changed If it changed to null (Line 18), then the deque is empty An empty deque may be the result of a competing pop_lef t that "steals" the last item from the pop_rιght, as illustrated in FIG. 4
If, on the other hand, S [R- l ] was not null, the DCAS failure indicates that the value of S [R- l ] has changed, and some other processor(s) must have completed a pop and a push between the read and the DCAS operation In this case, pop_rιght loops back and retries, since there may still be items in the deque Note that Lines 17-18 are an optimization, and one can instead loop back if the DCAS fails The optimization allows detection of a possible empty state without going through the loop, which in case the queue was indeed empty, would require another DCAS operation (Lines 6-10)
To perform a push_rιght, a sequence similar to pop_πght is performed An illustrative push_rιght access operation in accordance with the present invention follows
val push_rιght (val v) { whi le ( true ) { oldR = R ; newR = (oldR + 1 ) mod length_S ; oldS = S [oldR] , if ( oldS ι = "null " ) { if (oldR == R) if (DCAS R , &S [oldR] , oldR, oldS, oldR, oldS) ) return "full";
} else { newS = v; if DCAS(&R, &S [oldR] , oldR, oldS, &newR, &newS) return "okay"; else if (newR == oldR) return "full"; } } } Operation of pop_rιght is similar to that of push_rιght, but with all tests to see if a location is null replaced with tests to see if it is non-null, and with S locations corresponding to an index identified by, rather than adjacent to that identified by, the index To perform a push_rιght, a processor first reads R and the location in S corresponding to R (Lines 3-5, above) It then checks whether S [R] is non-null If S [R] is
non-null, then the processor reads R agai 1 to see if it has changed (Lines 6-7). This additional read is a performance enhancement added under t le assumption that the common case is that a non-null value is read because another processor "beat" the pro;essor, and not because the queue is really full. Other implementations need not employ such an enhancement. The test can be stated as follows: if R hasn't changed and S [R] is non-null, then the deque must be full since the location identified by R always contains a null value unless the deque is full. However, the conclusion that the deque is full can only be made based on an instantaneous view of R and S [R] . Therefore, the push_right implementation employs a DCAS (Lines 8- 10) to check if this is in fact the case. If so, push_right returns an indication that the deque is full. If not, then either the value in S [R] is no longer non-null or the index R has changed. In either case, the processor loops around and starts again.
If S [R] is null, the processor attempts to push value, v, onto S (Lines 12-19). The push_right implementation employs a DCAS to try to atomically increment the counter R and place the value, v, in S [R] , while returning (via &newR) the old value of index R (Lines 14-16). Note that the overloaded variant of DCAS described above is utilized here.
A successful DCAS and hence a successful push_right operation into an empty deque is depicted in FIG. 3. Initially, S = {) and L and R are as shown. Contents of R and of S [R] are read, but the results of the reads may not be consistent if an intervening competing access has successfully completed. In the context of the empty deque state illustrated in FIG. 3, the competing access of concern is another push_right, although in the case of non-empty state of the deque, a pop_right might also intervene. Because of the risk of a successfully completed competing access, the push_right implementation employs a DCAS (lines 14- 15) to check the instantaneous values of R and of S [R] and, if unchanged, perform the atomic update of R and of S [R] resulting in a deque state of 5 = (vι). A successful push_right operation into an almost-full deque is illustrated in the transition from deque states of FIGS. 5B and 5C.
In the final stage of the push_right code, in case the DCAS failed, there is a check using the value returned (via &newR) to see if the R index has changed. If it has not, then the failure must be due to a non-null value in the corresponding element of S, which means that the deque is full.
Pop_lef t and push_lef t sequences correspond to their above described right hand variants. An illustrative pop_lef t access operation in accordance with the present invention follows:
val pop_left { while (true) { oldL = L; newL = (oldL + 1) mod length_S; oldS = S [newL] ; if (oldS == "null") { if (oldL == L) if (DCAS L, &S [newL] , oldL, oldS, oldL, oldS)) return "empty"; }
else { newS = "null " ; if (DCAS ( &L, &S [newL] , oldL, oldS , &newL, &newS) ) return newS ; else if (newL == oldL) { if (newS == "null " ) return " empty" ;
} } } }
An illustrative push_lef t access operation in accordance with the present invention follows
val push_left (val v) { while (true) { oldL = L; newL = (oldL - 1) mod length_S; oldS = S [oldL] ; f (oldS ι= "null") { if (oldL == L) if (DCAS(&L, &S [oldL] , oldL, oldS, oldL, oldS)) return "full" ;
} else { newS = v; if (DCAS(&L, &S [oldL] , oldL, oldS, &newL, &.newS) ) return "okay"; else if (newL == oldL) return "full";
}
}
}
FIGS. 5A, 5B and 5C illustrate operations on a nearly full deque including a push_lef t operation (FIG. 5B) and a push_rιght operation that result in a full state of the deque (FIG. 5C) Notice that L has wrapped around and is "to-the-πght" of R, until the deque becomes full, in which case again L and R cross This switching of the relative location of the L and R pointers is somewhat confusing and represents a limitation of the linear presentation in the drawings However, in any case, it should be noted that each of the above described access operations (push_lef t, pop_lef t, push_πght and pop_πght) can determine the state of the deque, without regard to the relative locations of L and R, but rather by examining the relation of a given index (R or L) to the value in a corresponding element of S
While the invention has been described with reference to various embodiments, it will be understood that these embodiments are illustrative and that the scope of the invention is not limited to them Many variations, modifications, additions, and improvements are possible Plural instances may be provided for components described herein as a single instance Finally, boundaries between various components, services, servlets, and data stores are somewhat arbitrary, and particular operations are illustrated in the context of specific illustrative configurations Other allocations of functionality are envisioned and may fall within the scope of claims that follow Structures and functionality presented as discrete components in the exemplary
configurations may be implemented as a combined structure or component. These and other variations, modifications, additions, and improvements may fall within the scope of the invention as defined in the claims that follow.