WO2001022215A1 - Mechanism for implementing multiple thread pools in a computer system to optimize system performance - Google Patents

Mechanism for implementing multiple thread pools in a computer system to optimize system performance Download PDF

Info

Publication number
WO2001022215A1
WO2001022215A1 PCT/US2000/026151 US0026151W WO0122215A1 WO 2001022215 A1 WO2001022215 A1 WO 2001022215A1 US 0026151 W US0026151 W US 0026151W WO 0122215 A1 WO0122215 A1 WO 0122215A1
Authority
WO
WIPO (PCT)
Prior art keywords
request
thread
thread pool
service
pool
Prior art date
Application number
PCT/US2000/026151
Other languages
French (fr)
Other versions
WO2001022215A8 (en
Inventor
Ruslan Belkin
Viswanath Ramachandran
Original Assignee
Sun Microsystems, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Microsystems, Inc. filed Critical Sun Microsystems, Inc.
Priority to AU76090/00A priority Critical patent/AU7609000A/en
Priority to EP00965364A priority patent/EP1242871A4/en
Publication of WO2001022215A1 publication Critical patent/WO2001022215A1/en
Publication of WO2001022215A8 publication Critical patent/WO2001022215A8/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5055Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering software capabilities, i.e. software resources associated or available to the machine
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5011Pool
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5018Thread allocation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00Data processing: database and file management or data structures
    • Y10S707/99941Database schema or data structure
    • Y10S707/99943Generating database or data structure, e.g. via user interface

Definitions

  • This invention relates generally to computer systems, and more particularly to a mechanism for implementing multiple thread pools in a computer system to optimize system performance.
  • multi -threading In a multi-threaded system, there is allocated within a single process space a plurality of "threads" of execution. Each thread of execution has its own private stack, and each thread is used to execute a specific set of computer code at a time. During execution, each thread uses its stack to maintain state and other information specific to that thread of execution. This thread-specific information cannot be accessed or altered by other threads. As a result, each thread can execute code independent of other threads.
  • multi-threading is implemented by first giving rise to a process space (e.g. by running an instance of a particular program, such as a web server program). Then, a plurality of threads (referred to herein as a thread pool) are allocated within that process space. In allocating the threads, each thread is given a unique thread ID, and each thread is allocated a stack having a particular size, where all of the threads within the thread pool are given the same stack size.
  • the system is ready to service requests. When a request is received, the system determines whether the thread pool has an available thread. If so, then a thread is assigned to the request, and that thread is used to service the request.
  • servicing a request it is meant that a set of code is executed to carry out the functions needed to satisfy the request.
  • the execution of the set of code is carried out using the assigned thread and its associated stack. Multiple requests can be serviced concurrently; thus, if another request is received, then another thread is assigned to that other request, and that other thread is used to service the request.
  • the two threads will execute independent of each other. As a result, the two requests can be serviced concurrently.
  • the current methodology for implementing multi-threading described above is effective when all of the services demanded by the requests have substantially the same requirements. However, when the requirements of the services differ substantially, the current methodology can lead to inefficiencies. To illustrate this problem, suppose that a system is required to provide two different types of services in response to requests: (1) a lightweight service (such as an HTML static file retrieval service provided by a web server); and (2) a heavyweight service (such as a JAVA-type service). For the lightweight service, it would be optimal to have a large number of threads, with each thread having a small stack. The large number of threads allows a large number of requests to be serviced concurrently, while the small stacks conserve memory.
  • a lightweight service such as an HTML static file retrieval service provided by a web server
  • a heavyweight service such as a JAVA-type service
  • the current methodology has to reach a compromise.
  • the compromise is a combination of the extremes, namely, a thread pool with a small number of threads, with each thread having a large stack size.
  • this compromise ensures that even in the worst case scenario, where all of the threads are used for heavyweight services, the system will still function adequately.
  • On the negative side though, this compromise leads to inefficiencies.
  • the small number of threads unnecessarily limits the number of lightweight services that can be provided at any one time, and the large stack size causes memory waste (the lightweight services do not need large stacks).
  • the current methodology sacrifices efficiency in the general case to ensure proper operation in the worst case, which clearly is an undesirable result. To achieve greater system efficiency, an improved mechanism for implementing multi-threading is needed.
  • the present invention provides a more efficient mechanism for implementing multi-threading in a computer system.
  • the present invention is based, at least partially, upon the observation that the inefficiencies of the current methodology stem from the fact that only one thread pool is used. With just one thread pool, it is necessary to make the compromise discussed above. However, if a plurality of thread pools is implemented, with each thread pool customized for one or more particular types of service, then no compromise is needed. When one type of service is needed, a thread from the customized pool associated with that type of service is used. When another type of service is needed, a thread from the customized pool associated with that other type of service is used. There is no need to use one type of thread (e.g. a heavyweight thread) when another type of thread (e.g. a lightweight thread) is needed. By implementing multiple thread pools, the present invention eliminates many if not all of the inefficiencies of the current methodology.
  • a plurality of thread pools is allocated within a process space, with each thread pool comprising one or more threads.
  • Each thread pool has a set of characteristics associated therewith, and in one embodiment, the characteristics of each thread pool are customized for one or more particular types of service.
  • the characteristics of a thread pool may include but are not limited to: (1) the maximum number of threads that can be allocated in that thread pool; (2) the stack size of each thread within that thread pool; and (3) optionally, whether each thread in that thread pool has additional private storage. These characteristics may be set such that they are optimal for particular types of services.
  • the characteristics may be set such that the number of threads in the thread pool is large, and the stack size is small.
  • the characteristics may be set such that the number of threads is small, and the stack size is large.
  • Each thread pool may have its characteristics customized for one or more types of service.
  • the system is ready to service requests.
  • a request is received, it is processed to determine with which thread pool the request is to be associated. In one embodiment, this processing is carried out by determining the type of service being requested by the request, and then determining which thread pool is associated with that type of service. In another embodiment, this processing is carried out by extracting a set of indication information (e.g. a universal resource identifier) from the request, and then determining which thread pool is associated with that set of indication information. Once the proper thread pool is determined, a thread from that thread pool is used to carry out servicing of the request. By servicing the request, it is meant that a set of code is executed to carry out the functions needed to satisfy the request.
  • a set of indication information e.g. a universal resource identifier
  • the execution of the set of code is carried out using the assigned thread and its associated stack. In this manner, the request is serviced. Because the request is serviced using a thread from the thread pool customized for the type of service being requested, the servicing of the request is optimized. This in turn optimizes system performance.
  • Fig. 1 is a functional block diagram of a system in which one embodiment of the present invention may be implemented.
  • Fig. 2 shows an embodiment of a thread pool configuration table that may be used by a user to define and to specify characteristics of one or more thread pools.
  • Fig. 3 shows possible embodiments of association tables that may be used by a user to specify associations between thread pools and services types, and between thread pools and URI's.
  • Fig. 4 is a flow diagram illustrating the overall operation of the server in accordance with one embodiment of the present invention.
  • Fig. 5 is a flow diagram illustrating the manner in which one embodiment of the sticky attach mechanism operates.
  • Fig. 6 is a hardware block diagram of a computer system in which the present invention may be implemented.
  • FIG. 1 there is shown a functional block diagram of a system 100 in which one embodiment of the present invention may be implemented, the system comprising a plurality of clients 102, a network 104, and a web server 106.
  • the invention will be described with reference to a web server 106, but it should be noted that the invention is not so limited. Rather, the invention may be implemented in any type of server or computer system in which it is desirable to implement multi-threading.
  • the client 102 may be any mechanism capable of communicating with the server 106, including but not limited to a computer running a browser program.
  • the client 102 may communicate with the server 106 using any known protocol, including but not limited to HTTP and FTP, and the client 102 communicates with the server 106 via the network 104.
  • the network 104 may be any type of network, including but not limited to a local area network and a wide area network such as the Internet.
  • the network 104 may even be as simple as a direct connection. Any mechanism capable of facilitating communication between the clients 102 and the server 106 may serve as the network 104.
  • the server 106 is the component responsible for providing most of the functionality of the system 100. More specifically, the server 106 receives requests from the clients 102, and responds to the requests by providing response pages.
  • the response pages may be derived by simply accessing static files, or by executing one or more applications to dynamically generate the response pages.
  • the term application is used broadly herein to refer to any type of program or routine (e.g. a Java servlet) that is capable of performing one or more particular functions. What types of services need to be provided by the server 106 to derive the response pages is typically specified in the requests.
  • the server 106 implements multi-threading. That is, within the server 106. there are multiple threads of execution, with each thread capable of independent execution. Because each thread is capable of executing independently, multiple threads can service multiple requests concurrently.
  • the server 106 implements multiple thread pools. Each thread pool comprises one or more threads, and each thread pool has associated with it a set of characteristics. In one embodiment, the characteristics of each thread pool are set such that they are customized for one or more types of service. Each of the threads in the server 106 belongs within one of the thread pools.
  • requests are serviced slightly differently than with a single thread pool. More specifically, when a request is received, the request is first processed to determine with which thread pool the request is to be associated. This determination may be made in one of many ways, as will be discussed in a later section.
  • the thread pool that is associated with the request is the one that is customized for the type of service being requested by the request. Once the proper thread pool is determined, a thread from that thread pool is assigned to the request. Thereafter, the assigned thread is used to service the request. In servicing the request, the assigned thread and its associated stack are used to execute any computer code that is necessary for servicing the request. Because the thread that is actually used to service the request is from the thread pool customized for the type of service being requested, servicing of the request will be optimized. This in turn helps to optimize the overall operation of the server 106.
  • the server 106 and the manner in which multiple thread pools is implemented will now be described in greater detail.
  • the server 106 comprises a request processing mechanism 110, a set of services 1 12, a set of thread pool information 1 14, and an initialization mechanism 140.
  • the initialization mechanism 140 is invoked upon system start-up to create all of the thread pools that will be used in the server 106. In performing this task, the initialization mechanism 140 uses the information contained in the thread pool information 1 14, as will be described in a later section.
  • the request processing mechanism 1 10 is the component primarily responsible for receiving incoming requests, determining which service or services 112 need to be invoked to service the requests, and then assigning threads from the appropriate thread pools to be used by the services 1 12 to service the requests. In assigning the threads, the request processing mechanism 1 10 uses the information in the set of thread pool information 1 14 to determine the proper thread pools from which the threads should be taken. The manner in which the request processing mechanism 1 10 determines which service to invoke, and assigns the threads, will be described in detail in a later section.
  • various services 1 12 within the server 106 may be invoked in order to service the request. For example, if the request is a simple request for an HTML page, then the request processing mechanism 1 10 forwards the request on to HTML engine 122 for further servicing. On the other hand, if the request is for a common gateway interface (CGI) program, then the request processing mechanism 1 10 forwards the request on to the CGI engine 124 for servicing. In response, the CGI engine 124 invokes one or more CGI applications 130. If the request is for a JAVA type service, then the request processing mechanism 1 10 forwards the request on to the JAVA services engine 126. In turn, the JAVA services engine 126 invokes one or more JAVA type applications 132 (e.g.
  • CGI common gateway interface
  • JVM JAVA virtual machine
  • the request may also request other types of services, such as certain legacy code 120. If that is the case, then the request processing mechanism 1 10 invokes the legacy code 120 to further service the request.
  • the services 112 mentioned thus far are just some examples of the types of services that may be provided by the server 106. Other services may also be provided within the spirit of the invention.
  • each of the services 1 12 uses a particular thread from a particular thread pool to carry out the execution of code necessary for servicing the request. Which thread and which thread pool the thread comes from is determined by the request processing mechanism 1 10. In making this determination, the request processing mechanism 110 uses the information contained in the set of thread pool information 114. With reference to Figs. 2 and 3, the contents of the thread pool information 1 14 will now be described in detail.
  • the thread pool information 1 14 first comprises a thread pool configuration table 200 (Fig. 2). It is in this table 200 that all of the thread pools within the server 106 are specified and defined. Each row of the table represents a particular thread pool in the server 106, and each column specifies a characteristic or parameter of each thread pool.
  • the information in table 200 is specifiable by a user. Thus, depending upon the types of services 1 12 that are provided by the server 106, a user may choose to specify certain thread pools for certain services. The ability to freely specify thread pools enables a user to tune the server 106 to fit specific needs.
  • each thread pool has a plurality of characteristics (i.e. columns) associated therewith. These characteristics include but are not limited to: a pool ID 202, a thread type 204. an initial number of threads 206, a maximum number of threads 208, a throttle 210, a stack size 212, a maximum queue length 214, a maximum time out period 216, additional private storage 218, and a reference to an evaluation function 220.
  • the pool ID column 202 specifies a unique ID associated with a particular thread pool. This ID allows the thread pool to be uniquely identified in the server 106.
  • the thread type column 204 specifies the type of thread that belongs within a particular thread pool. The types of thread that can be in a system vary depending upon the system, but typical thread types include operating system scheduled threads, user or application scheduled threads, user-level operating system scheduled threads, and user-level threads.
  • the next three columns 206, 208, 210 control the number of threads in a particular thread pool.
  • the initial number of threads specifies the number of threads that are allocated to a thread pool when the thread pool is first created by the initialization mechanism 140 (Fig. 1).
  • the maximum number of threads specifies the maximum number of threads that can be within a particular thread pool, and the throttle specifies the increment by which the number of threads in the thread pool may be increased.
  • the stack size column 212 specifies the size of the stacks allocated to threads within a particular thread pool. All of the threads within a particular thread pool have the same stack size.
  • the maximum queue length 214 and the maximum time out 216 columns determine the behavior of the system when all of the threads in a thread pool are currently being used.
  • the maximum queue length specifies the maximum number of requests that can be waiting for a thread from the thread pool at any one time, and the maximum time out specifies the maximum period of time that a request can wait for a thread from the thread pool before being "timed out”. If a request is timed out, then the request is dropped and a "server busy" message is returned to the client 102 submitting the request.
  • the additional storage column 220 specifies whether the threads in a particular thread pool have additional private storage associated therewith in addition to a stack. Some services (such as JAVA type services) require threads with additional private storage. For thread pools designed to accommodate such service types, additional private storage may be specified.
  • the evaluation function column 220 specifies a reference to a user-provided function that may be invoked when a certain condition is met, for example, when all of the threads in a particular thread pool are currently being used. This column 220 provides a hook that a user can use to execute a special function. The use of column 220 will be elaborated upon in a later section.
  • Table 200 allows a user to freely define any and all thread pools used in the server 106. Given this ability, a user may define and customize a thread pool for any type of service provided by the server 106. If, for example, the server 106 provides a lightweight service, such as an HTML page retrieval service 122, the user can customize a thread pool for that service type. For such a service type, the user most likely will set the maximum number of threads in the thread pool to a relatively large number so that a large number of requests can be serviced at a time, and the stack size to a relatively small value since only a relatively small amount of memory is needed to provide the lightweight service.
  • a lightweight service such as an HTML page retrieval service 122
  • the server 106 also provides a heavyweight service, such as a JAVA type service
  • a heavyweight service such as a JAVA type service
  • the user can customize another thread pool for that service type.
  • the user most likely will set the maximum number of threads in the thread pool to a relatively small number so as not to overburden the server 106 with too many concurrent heavyweight requests, and the stack size to a relatively large value to provide sufficient memory for processing the requests.
  • JAVA type requests require additional private storage, the user will most likely also specify that the threads in the thread pool have additional private storage associate therewith.
  • table 200 the user may define and customize a thread pool for any type of service.
  • the thread pool configuration table 200 is just one of the sets of information contained within the thread pool information 1 14. Another set of information is the association table 300 shown in Fig. 3. It is in this table 300 that the user specifies the association between particular thread pools and other components in the server 106. Table 300 may take on many different forms. Two possible forms 300a, 300b are shown in Fig. 3. Table 300a is the form used when it is desirable to associate a particular thread pool with one or more types of service. Table 300b is the form used when it is desirable to associate a particular thread pool with a set of indication information, such as the universal resource identifier (URI) prefix of a web request.
  • URI universal resource identifier
  • the association table 300a comp ⁇ ses a column 302 for the service type (e.g. JAVA, CGI, HTML, etc.), a column 304 for the pool ID of a particular associated thread pool, and a column tor the pointer 306 to the thread pool.
  • the pointer to the thread pool column 306 provides a reference to the list of threads currently allocated to a particular thread pool At the time that a user specifies the association between a thread pool and a particular type of service, this column is left blank.
  • the initialization mechanism 140 Fig. 1 upon system start-up, the mechanism 140 will insert a reference into column 306. This reference may thereafter be used to access the threads in the thread pool.
  • Each row of table 302 specifies one association between a particular thread pool and a particular type of service.
  • the contents of table 300a are specifiable by a user.
  • the user can freely define any thread pool, and associate any type of service with any defined thread pool.
  • a separate thread pool may be defined and associated with the HTML service 122 of Fig. 1.
  • a separate thread pool may be defined and associated with the CGI service 130, and a separate thread pool may be defined and associated with the JAVA type service 126.
  • Each thread pool may be customized for its associated service type.
  • a thread pool may be defined and associated with the request processing mechanism 110.
  • this thread pool is a default thread pool that is created upon system start-up, and is given a relatively large number of threads and a relatively small stack size.
  • any thread pool may be defined and associated with any type of service, and a single thread pool may be associated with more than one type of service
  • all of the tables 200, 300 of the thread pool information 1 14 are modifiable by a user using a simple user interface (not shown), such as a text editor or a simple graphical user interface
  • table 300b which may be used by a user to specify associations between thread pools and specific sets of indication information.
  • Indication information may include but is not limited to any information particularly identifying a specific application or set of code to be invoked.
  • the indication information takes the form of a URI prefix
  • table 300 sets forth associations between thread pools and specific applications. This type of association may be useful, for example, when it is desirable to associate a thread pool with a very specific application, such as a set of legacy code 120 (Fig. 1). This will be discussed in greater detail in a later section.
  • a URI prefix column 308 accepts a URI prefix associated with a specific application or set of code
  • the pool ID column 310 accepts the ID of a particular thread pool
  • the pointer to thread pool column 312 provides a pointer to the actual threads of a thread pool.
  • column 312 is left blank at the time that a user specifies the association between a thread pool and a URI prefix.
  • the structure of the server 106 has been disclosed. With reference to Figs. 1-3, and the flow diagram of Fig. 4, the operation of the server 106 will now be described. Operation of the server 106 begins with system start-up. At start-up, computer code constituting the server 106 is executed, and this gives rise to a server process space. As part of the start-up process, the initialization mechanism 140 of the server 106 is invoked. When invoked, the initialization mechanism 140 performs a number of initialization functions, one of which is to allocate (402) all of the thread pools that will be used by the server 106. These thread pools are allocated within the server process space that has just been created.
  • the initialization mechanism 140 consults the thread pool definitions contained in the thread pool configuration table 200. More specifically, the initialization mechanism 140 inspects each row of the table 200, and for each row, determines the values in the initial number of threads column 206, the stack size column 212, and the additional private storage column 218. Based upon these values, the initialization mechanism 140 allocates a thread pool accordingly. That is, the mechanism 140 creates a number of threads equal to the initial number of threads specified in column 206. Each of these threads is assigned a unique thread ID, and each thread is allocated a stack having the stack size specified in column 212, and additional private storage if so specified in column 218. In addition, each thread is associated with the pool ID of the thread pool that is currently being allocated.
  • a structure such as a list structure or a table, which enables the threads to be easily referenced and accessed.
  • a reference to this structure is then derived and stored.
  • the reference or pointer to the structure is stored in the pointer to thread pool column 306, 312 of the association tables 300.
  • the reference is stored in the row or rows corresponding to the appropriate pool ID.
  • the thread pool is thus allocated. This process is repeated for each row of the thread pool configuration table 200 to allocate all of the thread pools defined by a user.
  • the server 106 is ready to service requests from clients 102.
  • a request is received (404) (at a communications port by a low level mechanism such as an operating system)
  • a thread from the request processing mechanism thread pool is selected. This thread is assigned to and associated with the request, and is thereafter used to execute the computer code constituting the request processing mechanism 1 10 to process the request.
  • Multiple requests can be processed concurrently; thus, if another request is received, then another thread is selected from the request processing mechanism thread pool, and that thread is used to execute the computer code constituting the request processing mechanism 1 10 to process the request.
  • the request processing mechanism 1 10 first determines (406) with which thread pool the request is to be associated. In one embodiment, this determination involves up to two inquiries. The first inquiry is whether the request is requesting a functionality that has a specific thread pool associated therewith. In one embodiment, this inquiry is made by consulting the association table 300b. More specifically, the request processing mechanism 1 10 extracts from the request a set of indication information, such as a URI prefix, and then compares that information with the information stored in the URI prefix column 308 of the association table 300b. If a matching entry is found, then the pool ID corresponding to the matching entry is the ID of the pool with which the request is to be associated. The associated thread pool is thus determined.
  • the first inquiry is whether the request is requesting a functionality that has a specific thread pool associated therewith. In one embodiment, this inquiry is made by consulting the association table 300b. More specifically, the request processing mechanism 1 10 extracts from the request a set of indication information, such as a URI prefix, and then compares that information with the information stored
  • a second inquiry is made.
  • the second inquiry involves making a determination as to the type of service that is being requested by the request. In one embodiment, this is done by executing one or more name translation functions. More specifically, in one embodiment, there is a name translation function associated with most if not all of the services 1 12 provided by the server 106. The name translation functions determine, based upon the URI of the request, which of the services 1 12 needs to be invoked to service the request. These name translation functions are executed in turn. For example, the name translation function associated with the HTML engine 122 is invoked first to determine whether the HTML engine 122 needs to be invoked to service the request.
  • the name translation function associated with the CGI engine 124 is invoked to determine whether the CGI service engine 124 needs to be invoked to respond to the request. This process of executing the name translation functions continues until it is determined which type of service is being requested by the request. Once the service type is determined, it is compared with the service types stored in the service type column 302 of the association table 300a. Unless there is an error, there should be a match between the service type of the request and one of the ent ⁇ es in table 300a. Once a matching entry is found, the pool ID corresponding to the matching entry is the ID of the pool with which the request is to be associated. The associated thread pool is thus determined.
  • the request processing mechanism 1 10 determines (using the first or the second inquiry), once the request processing mechanism 1 10 knows the associated thread pool, it references the threads in the thread pool using a pointer. Depending upon how the associated thread pool was determined, this pointer may come from column 306 of table 300a, or from column 312 of table 300b.
  • a free thread from the thread pool is assigned (408) and associated with the request. The assigned thread is thereafter used to service the request. More specifically, the request processing mechanism 1 10 invokes (410) the service 1 12 requested by the request, and passes the request to the service 1 12. In invoking the service 1 12, the request processing mechanism 1 10 provides to the service 1 12 the assigned thread.
  • the invoked service 112 thereafter uses the assigned thread to execute the computer code necessary for servicing the request.
  • the request processing mechanism 1 10 invokes the JAVA engine 126, passes the request to the JAVA engine 126, and provides to the engine 126 a thread from the thread pool associated with JAVA type services. That thread is thereafter used to execute the computer code constituting the JAVA engine 126. That same thread may also be used to execute whatever application(s) 132 are needed to provide the desired functionality.
  • one of the services 112 Upon receiving the request and the assigned thread from the request processing mechanism 110, one of the services 112 proceeds to service the request.
  • the servicing of the request may involve executing one or more applications 130, 132. This execution is carried out using the assigned thread and its associated stack and private storage (if any).
  • servicing of the request by the service 1 12 is completed.
  • one or more response pages are generated and sent (414) by the service 112 to the client 102 submitting the request.
  • the request is fully serviced.
  • the request is serviced, there is no longer any need for the assigned thread. Thus, it is returned (416) to the thread pool from which it was obtained, where it is free to be used to service other requests. In the manner described, processing of a request is achieved.
  • one of the thread pools that may be defined and allocated within the server 1 10 is the thread pool associated with JAVA type services.
  • threads from this thread pool are used whenever JAVA type services are provided, and these threads are used to execute both the code constituting the JAVA services engine 126 and the code constituting the one or more JAVA applications 132. Because threads from this thread pool are used to execute JAVA type applications 132, they are subject to more processing than most other threads.
  • the present invention further provides a "sticky attach" mechanism.
  • a thread is attached to the JVM 116 by invoking the JNI AttachThread procedure of the JVM 1 16.
  • this procedure performs two major functions. First, it registers the thread with the JVM 1 16. More specifically, it stores the thread ID of the thread being attached into the internal structures of the JVM 1 16. This causes the JVM 1 16 to recognize the thread as a JAVA attached thread so that when the JVM 116 performs any action involving attached threads, it will perform the action on this thread. For example, when the JVM 1 16 performs garbage collection, it will suspend this thread before it removes objects from the heap.
  • a second function performed by the JNI AttachThread procedure is allocation of a structure, referred to herein as a JAVA environment, to the thread being attached.
  • This structure is stored within the p ⁇ vate local storage of the thread (recall that threads used to execute JAVA applications have p ⁇ vate local storage), and contains all of the resources needed by the thread to invoke the services of the JVM 116.
  • the JAVA environment comp ⁇ ses a method table portion and a local data portion.
  • the method table contains a list of references to the methods of the JVM 1 16. These references may be used by the thread to invoke the services of the JVM 1 16 du ⁇ ng execution of JAVA applications
  • the local data portion provides storage tor sto ⁇ ng local references to objects.
  • a reverse process is taken. Namely, a JNI_DetachThread procedure of the JVM 116 is called, and this procedure removes the thread ID of the detaching thread from the JVM, and deletes the JAVA environment from the p ⁇ vate local storage of the detaching thread. Once that is done, the thread is fully detached from the JVM 1 16, and may be returned to its associated thread pool
  • a typical execution of a JAVA application is earned out as follows First, a thread from a thread pool is attached to the JVM 1 16. Then, the attached thread is used to execute the JAVA application. Once the execution of the JAVA application is completed, the thread is detached from the JVM 1 16 and returned to the thread pool. This is repeated for every JAVA application. As can be seen, this process is inefficient because it requires constant attachment and detachment from the JVM 116.
  • the sticky attach mechanism of the present invention takes a different approach. Rather than attaching and detaching a thread each time a JAVA application is executed, a thread is attached to the JVM 1 16 only once: the first time it is used to execute a JAVA application Once attached, it is used to execute a JAVA application When it completes execution of the JAVA application, the thread is not detached from the JVM 1 16. Instead, a small amount of clean up is performed (this will be desc ⁇ bed in more detail in a later section), and then the thread is returned to the thread pool still attached to the JVM 1 16. The next time that same thread is obtained from the thread pool, there will be no need to reattach to the JVM 1 16. Instead, the thread may be used as is to execute a JAVA application In this manner, the sticky attach mechanism of the present invention enables a thread to be attached only once, and be used to execute an unlimited number of JAVA applications
  • the sticky attach process will now be desc ⁇ bed in greater detail
  • the process of initializing the server 106 and processing requests using the request processing mechanism 1 10 is the same in this process as in the general case desc ⁇ bed above; thus, these will not be repeated.
  • the request processing mechanism 1 10 obtains a free thread from the thread pool associated with JAVA type services, and assigns (502) this thread to the request. Thereafter, the request processing mechanism 1 10 invokes (504) the JAVA services engine 126, and passes to it the request and the assigned thread.
  • the request processing mechanism 1 10 performs all of the necessary context switching. Thereafter, the assigned thread and its associated stack are used to execute the code constituting the JAVA services engine 126. At this point, processing by the request processing mechanism 1 10 is completed; hence, the thread used to execute the mechanism 1 10 is returned to the request processing mechanism thread pool.
  • the JAVA services engine 126 In servicing the request, the JAVA services engine 126 will most likely invoke one or more of the JAVA applications 132. When invoked, these applications 132 will execute using the assigned thread. Before this occurs, however, the JAVA services engine 126 first determines (506) whether the assigned thread is already attached to the JVM 1 16. In one embodiment, this is done by checking the p ⁇ vate local storage of the assigned thread for the JAVA environment discussed above. If this environment is present, then the thread is already attached to the JVM 116. In such a case, the JAVA services engine 126 may proceed to execute (510) one or more of the JAVA applications 132 using the assigned thread. On the other hand, if the JAVA environment is not present, then the engine 126 proceeds to (508) to attach the thread to the JVM 1 16.
  • the engine 126 attaches the thread to the JVM 1 16 by invoking the JNI AttachThread procedure of the JVM 1 16. As desc ⁇ bed previously, this procedure stores the ID of the assigned thread into the internal structures of the JVM 116. In addition, it allocates and stores withm the p ⁇ vate local storage of the assigned thread a JAVA environment structure, comprising a method table portion and a local data portion. Once attached to the JVM 1 16, the assigned thread may be used to execute (510) one or more JAVA applications 132.
  • JAVA applications 132 In executing the JAVA applications 132, services of the JVM 1 16 will be invoked. These services are invoked using the references contained in the method table of the JAVA environment. As the JVM 1 16 services are invoked, one or more local object references will be created These references are stored with the local data portion of the JAVA environment. As this discussion shows, the JAVA environment provides a self- contained structure for facilitating execution of JAVA code by the thread. Execution of the JAVA applications 132 continues until the request is fully serviced. At that point, one or more response pages are generated and sent to the client 102 that submitted the request. Servicing of the request is thus completed.
  • the JAVA environment is not removed from the private local storage of the thread.
  • the thread is still attached to the JVM 116. That being the case, the next time that same thread is assigned from the thread pool, no reattachment will be necessary. Rather, the thread may be used as is to immediately execute JAVA applications 132.
  • the sticky attach mechanism of the present invention significantly decreases the overhead incurred by the server 106. This in turn increases the efficiency of the overall system.
  • the present invention provides a solution to this problem.
  • a separate thread pool is defined and associated with the thread-unaware application. Unlike other thread pools, however, this thread pool has its maximum number of threads parameter set to " 1 ".
  • the second instance will start executing only after the first instance has completed execution.
  • serializing execution potential problems caused by concurrently executing multiple instances of the thread- unaware application are prevented. In this manner, execution of the thread-unaware application in a multi -threaded environment is made safe.
  • a user first defines for each thread-unaware application a separate thread pool. This is done by creating an entry in the thread pool configuration table 200 of Fig. 2. In addition to specifying other parameters for this thread pool, such as the pool ID and stack size, the user sets the values in the initial number of threads column 206 and the maximum number of threads column 208 to "1 ". This guarantees that there will be no more than one thread in the thread pool at any one time. Thereafter, the user associates the thread pool with the thread-unaware application (for the sake of example, it will be assumed that the thread-unaware application is the legacy code 120 of Fig. 1 ).
  • the initialization mechanism 140 of the server 106 is invoked.
  • the initialization mechanism 140 creates the thread pool associated with the thread-unaware application 120, and allocates one thread to it.
  • This thread is assigned a thread ID and the proper pool ID, and information pertaining to the thread is stored in a structure, such as a list structure or a table, which enables the thread to be easily referenced and accessed.
  • a reference to this structure is then derived and stored in the pointer to thread pool column 312 of the association table 300b in the row corresponding to the appropriate pool ID.
  • the thread pool is thus allocated.
  • the server 106 is now ready to service requests.
  • the request processing mechanism 1 10 When a request for the thread-unaware application 120 is received, the request is processed by the request processing mechanism 1 10. In processing the request, the mechanism 1 10 extracts a URI prefix from the request, and compares it with the URI prefixes stored in column 308 of table 300b. Unless there is an error, a matching entry will be found in table 300b for the URI prefix. Once the matching entry is found, the thread pool associated with the thread-unaware application 120 is accessed using the corresponding pointer in column 312. Once the thread pool is accessed, the request processing mechanism 1 10 determines whether the thread pool has a free thread available. If not, then it means that the one thread in the thread pool is currently being used to execute an instance of the application 120. In such a case, the request processing mechanism 1 10 waits for the thread to become available. Thus, only one instance of the thread-unaware application 120 can be executed at a time.
  • the request processing mechanism 1 10 invokes the thread-unaware application 120, passes the request on to the application 120, and provides the free thread to the application 120 for use in servicing the request. In passing the request on to the application 120, the request processing mechanism 1 10 performs all of the necessary context switching. Once that is done, the thread used to execute the request processing mechanism 1 10 is returned to the request processing mechanism thread pool. Thereafter, the code associated with the thread-unaware application 120 is executed using the assigned thread to service the request. When the application 120 completes execution, it generates and sends one or more response pages to the client 102 that submitted the request. The request is thus fully serviced. Once that is done, the assigned thread is returned to the thread pool associated with the thread-unaware application 120, where it can be used for another request. In the manner described, the thread-unaware application 120 is executed safely in a multithreaded environment.
  • one of the parameters that can be specified for each thread pool is a reference to an evaluation function (column 220).
  • This evaluation function is user-provided, and may be invoked to evaluate a request when one or more conditions are satisfied. When invoked, this function determines, based upon the request, how that request should be processed. Since the evaluation function is user-provided, it may be programmed to perform any desired operations, and to take any desired considerations and factors into account. This ability to define and associate an evaluation function with a thread pool gives a user great flexibility in controlling the functionality of the server 106. When used properly, the evaluation function can be an advantageous and powerful tool.
  • the server 106 of Fig. 1 is used to implement an Internet shopping website.
  • requests may range from requests to start a shopping transaction, to requests to add additional items to ongoing transactions, to requests to finalize transactions.
  • these requests are all equal, that is, they are all requests for services. From a business standpoint, however, these requests are not all equal. Rather, the request to finalize a transaction is much more valuable than a request to start a transaction because the request to finalize a transaction is much more likely to produce immediate revenue.
  • not all requests to finalize transactions are made equal.
  • a request to finalize a $5,000 transaction is much more important than a request to finalize a $5 transaction.
  • the evaluation function allows these considerations to be taken into account in processing requests.
  • the mechanism 1 10 may invoke the evaluation function referenced in the evaluation function column 220 of the thread pool.
  • the evaluation function analyzes and evaluates the request to determine how the request should be handled. If the evaluation function determines, for example, that the request is a request to finalize a $5,000 transaction, then the evaluation function may place the request at the head rather than the tail of the queue. Alternatively, the evaluation function may ensure that the request will not dropped from the queue but rather, will be serviced even if the maximum time out pe ⁇ od has elapsed.
  • the evaluation function determines that the request is a request to start a transaction, then the evaluation function may simply put the request at the tail of the queue.
  • the evaluation function is user- provided; thus, it may implement any logic desired by the user.
  • the condition prompting the execution of the evaluation function is the unavailability of a free thread
  • the considerations taken into account in evaluating the request are the type of request and the amount of the request
  • the actions taken involve placement of the request on the queue and insurance that the request will not be dropped from the queue. It should be noted that these are illustrative only.
  • any conditions may prompt the execution of the evaluation function, any critena may be taken into account in evaluating the request, and any actions may be taken in disposing of the request. All such possibilities are withm the scope of the present invention.
  • the request processing mechanism 1 10 that is p ⁇ ma ⁇ ly responsible for invoking the evaluation function associated with a thread pool.
  • the request processing mechanism 1 10 that checks for the conditions (whatever they might be) prompting the execution of the evaluation function.
  • the code constituting the request processing mechanism 1 10 comprises logic for checking for these conditions. It should be noted that this is just one possible embodiment. Alternatively, the tasks of checking for the proper conditions and invoking the evaluation function may be performed by one or more other components. This and other modifications are within the scope of the present invention.
  • the server 106 of the present invention and its various components are implemented as a set of instructions executable by one or more processors.
  • the invention may be implemented as part of an object oriented programming system, including but not limited to the JAVATM programming system manufactured by Sun Microsystems, Inc. of Mountain View, California.
  • Fig. 6 shows a hardware block diagram of a computer system 600 in which an embodiment of the invention may be implemented.
  • Computer system 600 includes a bus 602 or other communication mechanism for communicating information, and a processor 604 coupled with bus 602 for processing information.
  • Computer system 600 also includes a main memory 606, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 602 for storing information and instructions to be executed by processor 604.
  • main memory 606 such as a random access memory (RAM) or other dynamic storage device
  • Main memory 606 may also be further used to store temporary variables or other intermediate information during execution of instructions by processor 604.
  • Computer system 600 further includes a read only memory (ROM) 608 or other static storage device coupled to bus 602 for storing static information and instructions for processor 604.
  • ROM read only memory
  • a storage device 610 such as a magnetic disk or optical disk, is provided and coupled to bus 602 for storing information and instructions.
  • Computer system 600 may be coupled via bus 602 to a display 612, such as a cathode ray tube (CRT), for displaying information to a computer user.
  • a display 612 such as a cathode ray tube (CRT)
  • An input device 614 is coupled to bus 602 for communicating information and command selections to processor 604.
  • cursor control 616 is Another type of user input device
  • cursor control 616 such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 604 and for controlling cursor movement on display 612.
  • This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
  • the functionality of the present invention is provided by computer system 600 in response to processor 604 executing one or more sequences of one or more instructions contained in main memory 606.
  • Such instructions may be read into main memory 606 from another computer-readable medium, such as storage device 610.
  • Execution of the sequences of instructions contained in main memory 606 causes processor 604 to perform the process steps described herein.
  • hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention.
  • embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
  • Non-volatile media includes, for example, optical or magnetic disks, such as storage device 610.
  • Volatile media includes dynamic memory, such as main memory 606.
  • Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 602. Transmission media can also take the form of acoustic or electromagnetic waves, such as those generated during radio-wave, infra-red, and optical data communications.
  • Computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
  • Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 604 for execution.
  • the instructions may initially be carried on a magnetic disk of a remote computer.
  • the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
  • a modem local to computer system 600 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal.
  • An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 602.
  • Bus 602 carries the data to main memory 606, from which processor 604 retrieves and executes the instructions.
  • the instructions received by main memory 606 may optionally be stored on storage device 610 either before or after execution by processor 604.
  • Computer system 600 also includes a communication interface 618 coupled to bus 602.
  • Communication interface 618 provides a two-way data communication coupling to a network link 620 that is connected to a local network 622.
  • communication interface 618 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line.
  • ISDN integrated services digital network
  • communication interface 618 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • LAN local area network
  • Wireless links may also be implemented.
  • communication interface 618 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
  • Network link 620 typically provides data communication through one or more networks to other data devices.
  • network link 620 may provide a connection through local network 622 to a host computer 624 or to data equipment operated by an Internet Service Provider (ISP) 626.
  • ISP 626 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 628.
  • Internet 628 uses electrical, electromagnetic or optical signals that carry digital data streams.
  • the signals through the various networks and the signals on network link 620 and through communication interface 618, which carry the digital data to and from computer system 600, are exemplary forms of carrier waves transporting the information.
  • Computer system 600 can send messages and receive data, including program code, through the network(s), network link 620 and communication interface 618.
  • a server 630 might transmit a requested code for an application program through Internet 628.
  • ISP 626 local network 622 and communication interface 618.
  • the received code may be executed by processor 604 as it is received, and/or stored in storage device 610, or other non-volatile storage for later execution. In this manner, computer system 600 may obtain application code in the form of a carrier wave.

Abstract

A mechanism is disclosed for implementing multiplementing multiple thread pools (114) in a computer system to optimize system performance. In accordance with the invention, a plurality of thread pools is initially allocated within a process space, with each thread pool comprising one or more threads. Each thread pool has a set of characteristics associated therewith, and the characteristics of each thread pool are customized for one or more particular types of service (112). After the thread pools (114) have been allocated, the system receives one or more requests (102). When a request is received, it is processed to determine with which thread pool the request is to be associated. This processing is carried out by determining the type of service being requested by the request, and then determining which thread pool is associated with that type of service. Alternatively, this processing is carried out by extracting a set of indication information (e.g. a universal resource identifier) from the request.

Description

MECHANISM FOR IMPLEMENTING MULTIPLE THREAD POOLS IN A COMPUTER SYSTEM TO OPTIMIZE SYSTEM PERFORMANCE
This application claims the benefit of U. S. Provisional Application entitled "Web Server Architecture", No. 60/156,305, filed September 24, 1999, and U. S. Provisional Application entitled "Web Server Architecture", No. 60/155,71 1, filed September 24, 1999. The entire contents of these provisional applications are hereby incorporated by reference.
BACKGROUND
This invention relates generally to computer systems, and more particularly to a mechanism for implementing multiple thread pools in a computer system to optimize system performance.
Many of today's high capacity computers, such as web servers, are required to process a large number of requests concurrently. To enable them to do so efficiently, many of these computer systems implement a technique known as multi -threading. In a multi-threaded system, there is allocated within a single process space a plurality of "threads" of execution. Each thread of execution has its own private stack, and each thread is used to execute a specific set of computer code at a time. During execution, each thread uses its stack to maintain state and other information specific to that thread of execution. This thread-specific information cannot be accessed or altered by other threads. As a result, each thread can execute code independent of other threads. It is this ability of threads to execute code independently that makes it possible for multiple threads to service multiple requests concurrently. While each thread can maintain its own set of private information, each thread can also share information with other threads within the same process space. This information sharing is carried out much more easily than in multi-processing systems (where inter-process communication is needed). These two properties, among others, make multi-threading an advantageous mechanism for concurrently servicing multiple requests.
In a typical multi-threaded system, multi-threading is implemented by first giving rise to a process space (e.g. by running an instance of a particular program, such as a web server program). Then, a plurality of threads (referred to herein as a thread pool) are allocated within that process space. In allocating the threads, each thread is given a unique thread ID, and each thread is allocated a stack having a particular size, where all of the threads within the thread pool are given the same stack size. Once the process space is created and the thread pool is allocated, the system is ready to service requests. When a request is received, the system determines whether the thread pool has an available thread. If so, then a thread is assigned to the request, and that thread is used to service the request. By servicing a request, it is meant that a set of code is executed to carry out the functions needed to satisfy the request. The execution of the set of code is carried out using the assigned thread and its associated stack. Multiple requests can be serviced concurrently; thus, if another request is received, then another thread is assigned to that other request, and that other thread is used to service the request. The two threads will execute independent of each other. As a result, the two requests can be serviced concurrently.
At some point, all of the execution that needs to be done to satisfy a request is completed. Once that point is reached, the thread assigned to the request is returned to the thread pool, and is thereafter free to be used to service another request. In the manner described, threads are assigned from the thread pool when needed, and threads are returned to the thread pool when servicing is completed.
The current methodology for implementing multi-threading described above is effective when all of the services demanded by the requests have substantially the same requirements. However, when the requirements of the services differ substantially, the current methodology can lead to inefficiencies. To illustrate this problem, suppose that a system is required to provide two different types of services in response to requests: (1) a lightweight service (such as an HTML static file retrieval service provided by a web server); and (2) a heavyweight service (such as a JAVA-type service). For the lightweight service, it would be optimal to have a large number of threads, with each thread having a small stack. The large number of threads allows a large number of requests to be serviced concurrently, while the small stacks conserve memory. In contrast, for the heavyweight service, it would be optimal to have a small number of threads, with each thread having a large stack. The small number of threads prevents the system from being overburdened by heavyweight requests, while the large stack size is needed for proper execution of the heavyweight service. Clearly, the requirements of these service types conflict.
To accommodate both in a single system, the current methodology has to reach a compromise. Typically, the compromise is a combination of the extremes, namely, a thread pool with a small number of threads, with each thread having a large stack size. On the positive side, this compromise ensures that even in the worst case scenario, where all of the threads are used for heavyweight services, the system will still function adequately. On the negative side, though, this compromise leads to inefficiencies. The small number of threads unnecessarily limits the number of lightweight services that can be provided at any one time, and the large stack size causes memory waste (the lightweight services do not need large stacks). As this discussion illustrates, the current methodology sacrifices efficiency in the general case to ensure proper operation in the worst case, which clearly is an undesirable result. To achieve greater system efficiency, an improved mechanism for implementing multi-threading is needed.
SUMMARY OF THE INVENTION The present invention provides a more efficient mechanism for implementing multi-threading in a computer system. The present invention is based, at least partially, upon the observation that the inefficiencies of the current methodology stem from the fact that only one thread pool is used. With just one thread pool, it is necessary to make the compromise discussed above. However, if a plurality of thread pools is implemented, with each thread pool customized for one or more particular types of service, then no compromise is needed. When one type of service is needed, a thread from the customized pool associated with that type of service is used. When another type of service is needed, a thread from the customized pool associated with that other type of service is used. There is no need to use one type of thread (e.g. a heavyweight thread) when another type of thread (e.g. a lightweight thread) is needed. By implementing multiple thread pools, the present invention eliminates many if not all of the inefficiencies of the current methodology.
In light of this observation, there is provided an improved mechanism for servicing requests in a multi -threaded system. Initially, a plurality of thread pools is allocated within a process space, with each thread pool comprising one or more threads. Each thread pool has a set of characteristics associated therewith, and in one embodiment, the characteristics of each thread pool are customized for one or more particular types of service. The characteristics of a thread pool may include but are not limited to: (1) the maximum number of threads that can be allocated in that thread pool; (2) the stack size of each thread within that thread pool; and (3) optionally, whether each thread in that thread pool has additional private storage. These characteristics may be set such that they are optimal for particular types of services. For example, for a thread pool customized for lightweight services, the characteristics may be set such that the number of threads in the thread pool is large, and the stack size is small. In contrast, for a thread pool customized for heavyweight services, the characteristics may be set such that the number of threads is small, and the stack size is large. Each thread pool may have its characteristics customized for one or more types of service.
After the thread pools have been allocated, the system is ready to service requests. When a request is received, it is processed to determine with which thread pool the request is to be associated. In one embodiment, this processing is carried out by determining the type of service being requested by the request, and then determining which thread pool is associated with that type of service. In another embodiment, this processing is carried out by extracting a set of indication information (e.g. a universal resource identifier) from the request, and then determining which thread pool is associated with that set of indication information. Once the proper thread pool is determined, a thread from that thread pool is used to carry out servicing of the request. By servicing the request, it is meant that a set of code is executed to carry out the functions needed to satisfy the request. The execution of the set of code is carried out using the assigned thread and its associated stack. In this manner, the request is serviced. Because the request is serviced using a thread from the thread pool customized for the type of service being requested, the servicing of the request is optimized. This in turn optimizes system performance.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a functional block diagram of a system in which one embodiment of the present invention may be implemented.
Fig. 2 shows an embodiment of a thread pool configuration table that may be used by a user to define and to specify characteristics of one or more thread pools.
Fig. 3 shows possible embodiments of association tables that may be used by a user to specify associations between thread pools and services types, and between thread pools and URI's.
Fig. 4 is a flow diagram illustrating the overall operation of the server in accordance with one embodiment of the present invention. Fig. 5 is a flow diagram illustrating the manner in which one embodiment of the sticky attach mechanism operates.
Fig. 6 is a hardware block diagram of a computer system in which the present invention may be implemented.
DETAILED DESCRIPTION OF EMBODIMENT(S)
With reference to Fig. 1 , there is shown a functional block diagram of a system 100 in which one embodiment of the present invention may be implemented, the system comprising a plurality of clients 102, a network 104, and a web server 106. For the sake of illustration, the invention will be described with reference to a web server 106, but it should be noted that the invention is not so limited. Rather, the invention may be implemented in any type of server or computer system in which it is desirable to implement multi-threading.
For purposes of the present invention, the client 102 may be any mechanism capable of communicating with the server 106, including but not limited to a computer running a browser program. The client 102 may communicate with the server 106 using any known protocol, including but not limited to HTTP and FTP, and the client 102 communicates with the server 106 via the network 104. The network 104 may be any type of network, including but not limited to a local area network and a wide area network such as the Internet. The network 104 may even be as simple as a direct connection. Any mechanism capable of facilitating communication between the clients 102 and the server 106 may serve as the network 104.
The server 106 is the component responsible for providing most of the functionality of the system 100. More specifically, the server 106 receives requests from the clients 102, and responds to the requests by providing response pages. The response pages may be derived by simply accessing static files, or by executing one or more applications to dynamically generate the response pages. The term application is used broadly herein to refer to any type of program or routine (e.g. a Java servlet) that is capable of performing one or more particular functions. What types of services need to be provided by the server 106 to derive the response pages is typically specified in the requests.
In servicing the requests, the server 106 implements multi-threading. That is, within the server 106. there are multiple threads of execution, with each thread capable of independent execution. Because each thread is capable of executing independently, multiple threads can service multiple requests concurrently. In addition, the server 106 implements multiple thread pools. Each thread pool comprises one or more threads, and each thread pool has associated with it a set of characteristics. In one embodiment, the characteristics of each thread pool are set such that they are customized for one or more types of service. Each of the threads in the server 106 belongs within one of the thread pools.
With multiple thread pools, requests are serviced slightly differently than with a single thread pool. More specifically, when a request is received, the request is first processed to determine with which thread pool the request is to be associated. This determination may be made in one of many ways, as will be discussed in a later section. In one embodiment, the thread pool that is associated with the request is the one that is customized for the type of service being requested by the request. Once the proper thread pool is determined, a thread from that thread pool is assigned to the request. Thereafter, the assigned thread is used to service the request. In servicing the request, the assigned thread and its associated stack are used to execute any computer code that is necessary for servicing the request. Because the thread that is actually used to service the request is from the thread pool customized for the type of service being requested, servicing of the request will be optimized. This in turn helps to optimize the overall operation of the server 106. The server 106 and the manner in which multiple thread pools is implemented will now be described in greater detail.
In one embodiment, the server 106 comprises a request processing mechanism 110, a set of services 1 12, a set of thread pool information 1 14, and an initialization mechanism 140. The initialization mechanism 140 is invoked upon system start-up to create all of the thread pools that will be used in the server 106. In performing this task, the initialization mechanism 140 uses the information contained in the thread pool information 1 14, as will be described in a later section.
In server 106, the request processing mechanism 1 10 is the component primarily responsible for receiving incoming requests, determining which service or services 112 need to be invoked to service the requests, and then assigning threads from the appropriate thread pools to be used by the services 1 12 to service the requests. In assigning the threads, the request processing mechanism 1 10 uses the information in the set of thread pool information 1 14 to determine the proper thread pools from which the threads should be taken. The manner in which the request processing mechanism 1 10 determines which service to invoke, and assigns the threads, will be described in detail in a later section.
Depending upon the request, various services 1 12 within the server 106 may be invoked in order to service the request. For example, if the request is a simple request for an HTML page, then the request processing mechanism 1 10 forwards the request on to HTML engine 122 for further servicing. On the other hand, if the request is for a common gateway interface (CGI) program, then the request processing mechanism 1 10 forwards the request on to the CGI engine 124 for servicing. In response, the CGI engine 124 invokes one or more CGI applications 130. If the request is for a JAVA type service, then the request processing mechanism 1 10 forwards the request on to the JAVA services engine 126. In turn, the JAVA services engine 126 invokes one or more JAVA type applications 132 (e.g. servelets, server pages, JAVA programs, etc.). To run JAVA type applications, a JAVA virtual machine (JVM) 116 is needed. Hence, a JVM 1 16 may be incorporated into the server 106. The request may also request other types of services, such as certain legacy code 120. If that is the case, then the request processing mechanism 1 10 invokes the legacy code 120 to further service the request. The services 112 mentioned thus far are just some examples of the types of services that may be provided by the server 106. Other services may also be provided within the spirit of the invention.
In servicing a request, each of the services 1 12 uses a particular thread from a particular thread pool to carry out the execution of code necessary for servicing the request. Which thread and which thread pool the thread comes from is determined by the request processing mechanism 1 10. In making this determination, the request processing mechanism 110 uses the information contained in the set of thread pool information 114. With reference to Figs. 2 and 3, the contents of the thread pool information 1 14 will now be described in detail.
In one embodiment, the thread pool information 1 14 first comprises a thread pool configuration table 200 (Fig. 2). It is in this table 200 that all of the thread pools within the server 106 are specified and defined. Each row of the table represents a particular thread pool in the server 106, and each column specifies a characteristic or parameter of each thread pool. The information in table 200 is specifiable by a user. Thus, depending upon the types of services 1 12 that are provided by the server 106, a user may choose to specify certain thread pools for certain services. The ability to freely specify thread pools enables a user to tune the server 106 to fit specific needs.
As shown in Fig. 2, each thread pool has a plurality of characteristics (i.e. columns) associated therewith. These characteristics include but are not limited to: a pool ID 202, a thread type 204. an initial number of threads 206, a maximum number of threads 208, a throttle 210, a stack size 212, a maximum queue length 214, a maximum time out period 216, additional private storage 218, and a reference to an evaluation function 220.
The pool ID column 202 specifies a unique ID associated with a particular thread pool. This ID allows the thread pool to be uniquely identified in the server 106. The thread type column 204 specifies the type of thread that belongs within a particular thread pool. The types of thread that can be in a system vary depending upon the system, but typical thread types include operating system scheduled threads, user or application scheduled threads, user-level operating system scheduled threads, and user-level threads. The next three columns 206, 208, 210 control the number of threads in a particular thread pool. The initial number of threads specifies the number of threads that are allocated to a thread pool when the thread pool is first created by the initialization mechanism 140 (Fig. 1). The maximum number of threads specifies the maximum number of threads that can be within a particular thread pool, and the throttle specifies the increment by which the number of threads in the thread pool may be increased.
The stack size column 212 specifies the size of the stacks allocated to threads within a particular thread pool. All of the threads within a particular thread pool have the same stack size. The maximum queue length 214 and the maximum time out 216 columns determine the behavior of the system when all of the threads in a thread pool are currently being used. The maximum queue length specifies the maximum number of requests that can be waiting for a thread from the thread pool at any one time, and the maximum time out specifies the maximum period of time that a request can wait for a thread from the thread pool before being "timed out". If a request is timed out, then the request is dropped and a "server busy" message is returned to the client 102 submitting the request.
The additional storage column 220 specifies whether the threads in a particular thread pool have additional private storage associated therewith in addition to a stack. Some services (such as JAVA type services) require threads with additional private storage. For thread pools designed to accommodate such service types, additional private storage may be specified. The evaluation function column 220 specifies a reference to a user-provided function that may be invoked when a certain condition is met, for example, when all of the threads in a particular thread pool are currently being used. This column 220 provides a hook that a user can use to execute a special function. The use of column 220 will be elaborated upon in a later section.
Table 200 allows a user to freely define any and all thread pools used in the server 106. Given this ability, a user may define and customize a thread pool for any type of service provided by the server 106. If, for example, the server 106 provides a lightweight service, such as an HTML page retrieval service 122, the user can customize a thread pool for that service type. For such a service type, the user most likely will set the maximum number of threads in the thread pool to a relatively large number so that a large number of requests can be serviced at a time, and the stack size to a relatively small value since only a relatively small amount of memory is needed to provide the lightweight service. Likewise, if the server 106 also provides a heavyweight service, such as a JAVA type service, then the user can customize another thread pool for that service type. For this service type, the user most likely will set the maximum number of threads in the thread pool to a relatively small number so as not to overburden the server 106 with too many concurrent heavyweight requests, and the stack size to a relatively large value to provide sufficient memory for processing the requests. In addition, since JAVA type requests require additional private storage, the user will most likely also specify that the threads in the thread pool have additional private storage associate therewith. Using table 200, the user may define and customize a thread pool for any type of service. Thus far, only efficiency considerations have been given as reasons for implementing multiple thread pools. It should be noted, though, that there are many other reasons. Some of these will be discussed in a later section.
The thread pool configuration table 200 is just one of the sets of information contained within the thread pool information 1 14. Another set of information is the association table 300 shown in Fig. 3. It is in this table 300 that the user specifies the association between particular thread pools and other components in the server 106. Table 300 may take on many different forms. Two possible forms 300a, 300b are shown in Fig. 3. Table 300a is the form used when it is desirable to associate a particular thread pool with one or more types of service. Table 300b is the form used when it is desirable to associate a particular thread pool with a set of indication information, such as the universal resource identifier (URI) prefix of a web request.
In one embodiment, the association table 300a compπses a column 302 for the service type (e.g. JAVA, CGI, HTML, etc.), a column 304 for the pool ID of a particular associated thread pool, and a column tor the pointer 306 to the thread pool. The pointer to the thread pool column 306 provides a reference to the list of threads currently allocated to a particular thread pool At the time that a user specifies the association between a thread pool and a particular type of service, this column is left blank. When the thread pool is actually created by the initialization mechanism 140 (Fig. 1) upon system start-up, the mechanism 140 will insert a reference into column 306. This reference may thereafter be used to access the threads in the thread pool.
Each row of table 302 specifies one association between a particular thread pool and a particular type of service. Like table 200, the contents of table 300a are specifiable by a user. Thus, with tables 200 and 300a, the user can freely define any thread pool, and associate any type of service with any defined thread pool. For example, a separate thread pool may be defined and associated with the HTML service 122 of Fig. 1. Likewise, a separate thread pool may be defined and associated with the CGI service 130, and a separate thread pool may be defined and associated with the JAVA type service 126. Each thread pool may be customized for its associated service type. In addition, a thread pool may be defined and associated with the request processing mechanism 110. In one embodiment, this thread pool is a default thread pool that is created upon system start-up, and is given a relatively large number of threads and a relatively small stack size. Overall, any thread pool may be defined and associated with any type of service, and a single thread pool may be associated with more than one type of service With tables 200 and 300, a user is given great flexibility in managing the thread pools of the server 106 to optimize system performance In one embodiment, all of the tables 200, 300 of the thread pool information 1 14 are modifiable by a user using a simple user interface (not shown), such as a text editor or a simple graphical user interface
In addition to or in lieu of table 300a, there is table 300b, which may be used by a user to specify associations between thread pools and specific sets of indication information. Indication information may include but is not limited to any information particularly identifying a specific application or set of code to be invoked. In one embodiment, the indication information takes the form of a URI prefix In contrast to table 300a. which specifies associations between thread pools and types of service, table 300 sets forth associations between thread pools and specific applications. This type of association may be useful, for example, when it is desirable to associate a thread pool with a very specific application, such as a set of legacy code 120 (Fig. 1). This will be discussed in greater detail in a later section. To enable the associations to be made, there is provided in table 300b a URI prefix column 308, a pool ID column 310, and a pointer to thread pool column 312. The URI prefix column 308 accepts a URI prefix associated with a specific application or set of code, the pool ID column 310 accepts the ID of a particular thread pool, and the pointer to thread pool column 312 provides a pointer to the actual threads of a thread pool. Like column 306 of table 300a, column 312 is left blank at the time that a user specifies the association between a thread pool and a URI prefix. When the thread pool is actually created by the initialization mechanism 140 (Fig. 1) upon system start-up, the mechanism 140 will insert a reference into column 312. This reference may thereafter be used to access the threads in the thread pool.
The structure of the server 106 has been disclosed. With reference to Figs. 1-3, and the flow diagram of Fig. 4, the operation of the server 106 will now be described. Operation of the server 106 begins with system start-up. At start-up, computer code constituting the server 106 is executed, and this gives rise to a server process space. As part of the start-up process, the initialization mechanism 140 of the server 106 is invoked. When invoked, the initialization mechanism 140 performs a number of initialization functions, one of which is to allocate (402) all of the thread pools that will be used by the server 106. These thread pools are allocated within the server process space that has just been created.
In allocating the thread pools, the initialization mechanism 140 consults the thread pool definitions contained in the thread pool configuration table 200. More specifically, the initialization mechanism 140 inspects each row of the table 200, and for each row, determines the values in the initial number of threads column 206, the stack size column 212, and the additional private storage column 218. Based upon these values, the initialization mechanism 140 allocates a thread pool accordingly. That is, the mechanism 140 creates a number of threads equal to the initial number of threads specified in column 206. Each of these threads is assigned a unique thread ID, and each thread is allocated a stack having the stack size specified in column 212, and additional private storage if so specified in column 218. In addition, each thread is associated with the pool ID of the thread pool that is currently being allocated. Once the threads are allocated, information pertaining to the threads is stored in a structure, such as a list structure or a table, which enables the threads to be easily referenced and accessed. A reference to this structure is then derived and stored. In one embodiment, the reference or pointer to the structure is stored in the pointer to thread pool column 306, 312 of the association tables 300. The reference is stored in the row or rows corresponding to the appropriate pool ID. The thread pool is thus allocated. This process is repeated for each row of the thread pool configuration table 200 to allocate all of the thread pools defined by a user. In one embodiment, there is at least one default thread pool that is allocated, and this thread pool is the one associated with the request processing mechanism 1 10. These threads will be used to process incoming requests.
Once all of the thread pools are allocated, the server 106 is ready to service requests from clients 102. When a request is received (404) (at a communications port by a low level mechanism such as an operating system), a thread from the request processing mechanism thread pool is selected. This thread is assigned to and associated with the request, and is thereafter used to execute the computer code constituting the request processing mechanism 1 10 to process the request. Multiple requests can be processed concurrently; thus, if another request is received, then another thread is selected from the request processing mechanism thread pool, and that thread is used to execute the computer code constituting the request processing mechanism 1 10 to process the request.
In processing a request, the request processing mechanism 1 10 first determines (406) with which thread pool the request is to be associated. In one embodiment, this determination involves up to two inquiries. The first inquiry is whether the request is requesting a functionality that has a specific thread pool associated therewith. In one embodiment, this inquiry is made by consulting the association table 300b. More specifically, the request processing mechanism 1 10 extracts from the request a set of indication information, such as a URI prefix, and then compares that information with the information stored in the URI prefix column 308 of the association table 300b. If a matching entry is found, then the pool ID corresponding to the matching entry is the ID of the pool with which the request is to be associated. The associated thread pool is thus determined. On the other hand, if no matching entry is found in table 300b, then a second inquiry is made. The second inquiry involves making a determination as to the type of service that is being requested by the request. In one embodiment, this is done by executing one or more name translation functions. More specifically, in one embodiment, there is a name translation function associated with most if not all of the services 1 12 provided by the server 106. The name translation functions determine, based upon the URI of the request, which of the services 1 12 needs to be invoked to service the request. These name translation functions are executed in turn. For example, the name translation function associated with the HTML engine 122 is invoked first to determine whether the HTML engine 122 needs to be invoked to service the request. If not, then the name translation function associated with the CGI engine 124 is invoked to determine whether the CGI service engine 124 needs to be invoked to respond to the request. This process of executing the name translation functions continues until it is determined which type of service is being requested by the request. Once the service type is determined, it is compared with the service types stored in the service type column 302 of the association table 300a. Unless there is an error, there should be a match between the service type of the request and one of the entπes in table 300a. Once a matching entry is found, the pool ID corresponding to the matching entry is the ID of the pool with which the request is to be associated. The associated thread pool is thus determined.
However the associated thread pool is determined (using the first or the second inquiry), once the request processing mechanism 1 10 knows the associated thread pool, it references the threads in the thread pool using a pointer. Depending upon how the associated thread pool was determined, this pointer may come from column 306 of table 300a, or from column 312 of table 300b. Once the threads in the thread pool are accessed, a free thread from the thread pool is assigned (408) and associated with the request. The assigned thread is thereafter used to service the request. More specifically, the request processing mechanism 1 10 invokes (410) the service 1 12 requested by the request, and passes the request to the service 1 12. In invoking the service 1 12, the request processing mechanism 1 10 provides to the service 1 12 the assigned thread. The invoked service 112 thereafter uses the assigned thread to execute the computer code necessary for servicing the request. As an example, suppose that the request is for a JAVA type service. In such a case, the request processing mechanism 1 10 invokes the JAVA engine 126, passes the request to the JAVA engine 126, and provides to the engine 126 a thread from the thread pool associated with JAVA type services. That thread is thereafter used to execute the computer code constituting the JAVA engine 126. That same thread may also be used to execute whatever application(s) 132 are needed to provide the desired functionality.
Note that in passing the request from the request processing mechanism 1 10 to one of the services 1 12, there is a thread transition. Originally, the request was being processed using a thread from the request processing mechanism thread pool. After the pass-off, the request is being processed using a thread from another thread pool (e.g. the JAVA services thread pool). To facilitate this transition, the request processing mechanism 1 10 performs all of the necessary context switching. Once the request is passed on to one of the services 1 12, the request processing mechanism 110 has completed its task. Therefore, the request processing mechanism thread used to execute the request processing mechanism code 1 10 is no longer needed. Accordingly, this thread is returned (412) to the request processing mechanism thread pool, where is it free to process another request.
Upon receiving the request and the assigned thread from the request processing mechanism 110, one of the services 112 proceeds to service the request. The servicing of the request may involve executing one or more applications 130, 132. This execution is carried out using the assigned thread and its associated stack and private storage (if any). At some point, servicing of the request by the service 1 12 is completed. When that occurs, one or more response pages are generated and sent (414) by the service 112 to the client 102 submitting the request. At that point, the request is fully serviced. After the request is serviced, there is no longer any need for the assigned thread. Thus, it is returned (416) to the thread pool from which it was obtained, where it is free to be used to service other requests. In the manner described, processing of a request is achieved.
As noted above, one of the thread pools that may be defined and allocated within the server 1 10 is the thread pool associated with JAVA type services. In one embodiment, threads from this thread pool are used whenever JAVA type services are provided, and these threads are used to execute both the code constituting the JAVA services engine 126 and the code constituting the one or more JAVA applications 132. Because threads from this thread pool are used to execute JAVA type applications 132, they are subject to more processing than most other threads.
In particular, they are subject to regular attachment and detachment from the JAVA virtual machine (JVM) 1 16. Typically, a thread is attached to the JVM 1 16 prior to being used to execute any JAVA application 132, and that same thread is detached from the JVM 1 16 once execution of the JAVA application 132 is completed. As will be elaborated upon below, the process of attaching and detaching a thread from the JVM 116 is an expensive process. Thus, this constant attaching and detaching of threads is quite costly in terms of processing time. To minimize the amount of attachment/detachment that needs to be done, the present invention further provides a "sticky attach" mechanism. With this mechanism, it is possible to return a thread to the JAVA associated thread pool without detaching the thread from the JVM 1 16. Because the thread is returned to the thread pool without detaching, it can be retπeved from the thread pool and used again to execute a JAVA application 132 without reattaching. This allows a thread to be attached to the JVM 1 16 just once and used to execute an unlimited number of JAVA applications 132. By eliminating the need to attach and detach a thread each time a JAVA application 132 is executed, the present invention significantly reduces the amount of overhead incurred by the server 106. This in turn significantly increases the efficiency of the server 106. The manner of implementing sticky attach will be described in detail below, but before that, some background information will be provided to facilitate a complete understanding of the invention.
Typically, a thread is attached to the JVM 116 by invoking the JNI AttachThread procedure of the JVM 1 16. When invoked, this procedure performs two major functions. First, it registers the thread with the JVM 1 16. More specifically, it stores the thread ID of the thread being attached into the internal structures of the JVM 1 16. This causes the JVM 1 16 to recognize the thread as a JAVA attached thread so that when the JVM 116 performs any action involving attached threads, it will perform the action on this thread. For example, when the JVM 1 16 performs garbage collection, it will suspend this thread before it removes objects from the heap.
A second function performed by the JNI AttachThread procedure is allocation of a structure, referred to herein as a JAVA environment, to the thread being attached. This structure is stored within the pπvate local storage of the thread (recall that threads used to execute JAVA applications have pπvate local storage), and contains all of the resources needed by the thread to invoke the services of the JVM 116. More specifically, the JAVA environment compπses a method table portion and a local data portion. The method table contains a list of references to the methods of the JVM 1 16. These references may be used by the thread to invoke the services of the JVM 1 16 duπng execution of JAVA applications The local data portion provides storage tor stoπng local references to objects. As the services of the JVM 1 16 are invoked duπng execution of a JAVA application 132, local references to objects will be created These local references will be stored in the local data portion of the JAVA environment Overall, the JAVA environment provides the thread with everything that it needs to execute JAVA applications Once the thread is registered with the JVM 1 16 and the JAVA environment is allocated to the thread, the thread is attached to the JVM 1 16 and is ready for use.
To detach a thread from the JVM 1 16, a reverse process is taken. Namely, a JNI_DetachThread procedure of the JVM 116 is called, and this procedure removes the thread ID of the detaching thread from the JVM, and deletes the JAVA environment from the pπvate local storage of the detaching thread. Once that is done, the thread is fully detached from the JVM 1 16, and may be returned to its associated thread pool
As noted above, a typical execution of a JAVA application is earned out as follows First, a thread from a thread pool is attached to the JVM 1 16. Then, the attached thread is used to execute the JAVA application. Once the execution of the JAVA application is completed, the thread is detached from the JVM 1 16 and returned to the thread pool. This is repeated for every JAVA application. As can be seen, this process is inefficient because it requires constant attachment and detachment from the JVM 116.
To eliminate this inefficiency, the sticky attach mechanism of the present invention takes a different approach. Rather than attaching and detaching a thread each time a JAVA application is executed, a thread is attached to the JVM 1 16 only once: the first time it is used to execute a JAVA application Once attached, it is used to execute a JAVA application When it completes execution of the JAVA application, the thread is not detached from the JVM 1 16. Instead, a small amount of clean up is performed (this will be descπbed in more detail in a later section), and then the thread is returned to the thread pool still attached to the JVM 1 16. The next time that same thread is obtained from the thread pool, there will be no need to reattach to the JVM 1 16. Instead, the thread may be used as is to execute a JAVA application In this manner, the sticky attach mechanism of the present invention enables a thread to be attached only once, and be used to execute an unlimited number of JAVA applications
With reference to Fig. 1 , and the flow diagram of Fig 5, the sticky attach process will now be descπbed in greater detail The process of initializing the server 106 and processing requests using the request processing mechanism 1 10 is the same in this process as in the general case descπbed above; thus, these will not be repeated. For purposes of this discussion, it will be assumed that a request has already been processed by the request processing mechanism 1 10 and determined to involve a JAVA type service. In such a case, the request processing mechanism 1 10 obtains a free thread from the thread pool associated with JAVA type services, and assigns (502) this thread to the request. Thereafter, the request processing mechanism 1 10 invokes (504) the JAVA services engine 126, and passes to it the request and the assigned thread. In addition, the request processing mechanism 1 10 performs all of the necessary context switching. Thereafter, the assigned thread and its associated stack are used to execute the code constituting the JAVA services engine 126. At this point, processing by the request processing mechanism 1 10 is completed; hence, the thread used to execute the mechanism 1 10 is returned to the request processing mechanism thread pool.
In servicing the request, the JAVA services engine 126 will most likely invoke one or more of the JAVA applications 132. When invoked, these applications 132 will execute using the assigned thread. Before this occurs, however, the JAVA services engine 126 first determines (506) whether the assigned thread is already attached to the JVM 1 16. In one embodiment, this is done by checking the pπvate local storage of the assigned thread for the JAVA environment discussed above. If this environment is present, then the thread is already attached to the JVM 116. In such a case, the JAVA services engine 126 may proceed to execute (510) one or more of the JAVA applications 132 using the assigned thread. On the other hand, if the JAVA environment is not present, then the engine 126 proceeds to (508) to attach the thread to the JVM 1 16.
In one embodiment, the engine 126 attaches the thread to the JVM 1 16 by invoking the JNI AttachThread procedure of the JVM 1 16. As descπbed previously, this procedure stores the ID of the assigned thread into the internal structures of the JVM 116. In addition, it allocates and stores withm the pπvate local storage of the assigned thread a JAVA environment structure, comprising a method table portion and a local data portion. Once attached to the JVM 1 16, the assigned thread may be used to execute (510) one or more JAVA applications 132.
In executing the JAVA applications 132, services of the JVM 1 16 will be invoked. These services are invoked using the references contained in the method table of the JAVA environment. As the JVM 1 16 services are invoked, one or more local object references will be created These references are stored with the local data portion of the JAVA environment. As this discussion shows, the JAVA environment provides a self- contained structure for facilitating execution of JAVA code by the thread. Execution of the JAVA applications 132 continues until the request is fully serviced. At that point, one or more response pages are generated and sent to the client 102 that submitted the request. Servicing of the request is thus completed.
After the request is serviced, it is time to return the assigned thread to the JAVA associated thread pool. Before this is done, however, some clean up operations are performed (512). Among others, these operations include clearing out the assigned thread's stack. In addition, any obsolete object references still stored in the data portion of the JAVA environment are removed. Removal of these references releases the referenced objects, which in turn enables the JVM 116 to perform garbage collection on the objects if there are no other references to them. After the clean up operations are performed, the assigned thread is returned (514) to the JAVA associated thread pool. Notice that the thread is returned to the thread pool without detaching from the JVM 116. That is, the ID of the thread is not removed from the internal structures of the JVM 116. In addition, the JAVA environment is not removed from the private local storage of the thread. Thus, for all intents and purposes, the thread is still attached to the JVM 116. That being the case, the next time that same thread is assigned from the thread pool, no reattachment will be necessary. Rather, the thread may be used as is to immediately execute JAVA applications 132. By removing the need to constantly attach and detach threads, the sticky attach mechanism of the present invention significantly decreases the overhead incurred by the server 106. This in turn increases the efficiency of the overall system.
It was mentioned previously that optimizing system performance is just one of the reasons for implementing multiple thread pools in a system. There are many more reasons, one of which is to enable one or more thread-unaware or non-thread-safe applications to be executed safely in a multi -threaded environment. To elaborate, in many current systems, there exist sets of computer code commonly referred to as legacy code that were developed years ago using old or outdated programming languages or techniques. A majority of this legacy code was written either before multi-threaded technology was developed or was developed without multi-threading in mind, and thus, is thread-unaware. When code is thread-unaware, it does not take into account the possibility that there may be more than one instance of that set of code running in the same process space at the same time. As a result, if multiple instances of the thread- unaware code are executed in the same process space concurrently, as is often the case in multi -threaded systems, serious eπors can occur. One of the possible errors is that one instance may modify or overwrite the information used by another instance (this occurs because each instance assumes that it is the only instance in the process space). When this happens, proper execution of the instances can be completely undermined. As a result, it is potentially hazardous to execute thread-unaware legacy code in a multithreaded environment. While legacy code is often outdated and potentially hazardous, it is still desirable in some implementations to make use of the code. In such implementations, the challenge is to execute the code in such a way that it does not cause system errors or data corruption.
The present invention provides a solution to this problem. Basically, when it is desirable to execute a thread-unaware or non-thread-safe application in a multi-threaded environment, a separate thread pool is defined and associated with the thread-unaware application. Unlike other thread pools, however, this thread pool has its maximum number of threads parameter set to " 1 ". By limiting the number of threads in this thread pool to 1, it is guaranteed that there will be no more than one instance of the application executing at any one time. This in turn precludes any possibility of one instance of the application overwriting or modifying the information used by another. In effect, limiting the number of threads in the thread pool to 1 serializes the execution of the application. If two executed instances of the application are needed, the second instance will start executing only after the first instance has completed execution. By serializing execution, potential problems caused by concurrently executing multiple instances of the thread- unaware application are prevented. In this manner, execution of the thread-unaware application in a multi -threaded environment is made safe.
To enable one or more thread-unaware applications to be executed safely in system 100 of Fig. 1, a user first defines for each thread-unaware application a separate thread pool. This is done by creating an entry in the thread pool configuration table 200 of Fig. 2. In addition to specifying other parameters for this thread pool, such as the pool ID and stack size, the user sets the values in the initial number of threads column 206 and the maximum number of threads column 208 to "1 ". This guarantees that there will be no more than one thread in the thread pool at any one time. Thereafter, the user associates the thread pool with the thread-unaware application (for the sake of example, it will be assumed that the thread-unaware application is the legacy code 120 of Fig. 1 ). This may be done by associating a specific URI prefix with the application 120. and then storing this URI prefix into an entry of table 300b in column 308. Also stored into table 300b is the pool ID of the thread pool that the user has just defined (the pool ID is stored in column 310). Once that is done, the application 120 is ready for execution.
At system start-up, the initialization mechanism 140 of the server 106 is invoked. Among other functions, the initialization mechanism 140 creates the thread pool associated with the thread-unaware application 120, and allocates one thread to it. This thread is assigned a thread ID and the proper pool ID, and information pertaining to the thread is stored in a structure, such as a list structure or a table, which enables the thread to be easily referenced and accessed. A reference to this structure is then derived and stored in the pointer to thread pool column 312 of the association table 300b in the row corresponding to the appropriate pool ID. The thread pool is thus allocated. The server 106 is now ready to service requests.
When a request for the thread-unaware application 120 is received, the request is processed by the request processing mechanism 1 10. In processing the request, the mechanism 1 10 extracts a URI prefix from the request, and compares it with the URI prefixes stored in column 308 of table 300b. Unless there is an error, a matching entry will be found in table 300b for the URI prefix. Once the matching entry is found, the thread pool associated with the thread-unaware application 120 is accessed using the corresponding pointer in column 312. Once the thread pool is accessed, the request processing mechanism 1 10 determines whether the thread pool has a free thread available. If not, then it means that the one thread in the thread pool is currently being used to execute an instance of the application 120. In such a case, the request processing mechanism 1 10 waits for the thread to become available. Thus, only one instance of the thread-unaware application 120 can be executed at a time.
On the other hand, if a free thread is available, then the free thread is assigned to the request, and is thereafter used to service the request. More specifically, the request processing mechanism 1 10 invokes the thread-unaware application 120, passes the request on to the application 120, and provides the free thread to the application 120 for use in servicing the request. In passing the request on to the application 120, the request processing mechanism 1 10 performs all of the necessary context switching. Once that is done, the thread used to execute the request processing mechanism 1 10 is returned to the request processing mechanism thread pool. Thereafter, the code associated with the thread-unaware application 120 is executed using the assigned thread to service the request. When the application 120 completes execution, it generates and sends one or more response pages to the client 102 that submitted the request. The request is thus fully serviced. Once that is done, the assigned thread is returned to the thread pool associated with the thread-unaware application 120, where it can be used for another request. In the manner described, the thread-unaware application 120 is executed safely in a multithreaded environment.
As noted previously in connection with table 200 of Fig. 2, one of the parameters that can be specified for each thread pool is a reference to an evaluation function (column 220). This evaluation function is user-provided, and may be invoked to evaluate a request when one or more conditions are satisfied. When invoked, this function determines, based upon the request, how that request should be processed. Since the evaluation function is user-provided, it may be programmed to perform any desired operations, and to take any desired considerations and factors into account. This ability to define and associate an evaluation function with a thread pool gives a user great flexibility in controlling the functionality of the server 106. When used properly, the evaluation function can be an advantageous and powerful tool.
The advantages offered by the evaluation function are best illustrated with reference to an example. Suppose that the server 106 of Fig. 1 is used to implement an Internet shopping website. On such a site, various types of requests may be received from the clients 102. These requests may range from requests to start a shopping transaction, to requests to add additional items to ongoing transactions, to requests to finalize transactions. To the server 106, these requests are all equal, that is, they are all requests for services. From a business standpoint, however, these requests are not all equal. Rather, the request to finalize a transaction is much more valuable than a request to start a transaction because the request to finalize a transaction is much more likely to produce immediate revenue. In addition, not all requests to finalize transactions are made equal. A request to finalize a $5,000 transaction is much more important than a request to finalize a $5 transaction. As this discussion shows, there are business and other types of reasons for treating different requests differently. The evaluation function allows these considerations to be taken into account in processing requests.
To show one possible use of the evaluation function, suppose that a request is received by the server 106. Suppose further: ( 1 ) that the request is processed by the request processing mechanism 1 10 and determined to be a request for a JAVA type service; and (2) that the thread pool associated with JAVA type services currently has no free threads available. In this situation, the request is typically put onto the queue associated with the thread pool. If the queue is full, or if the maximum time out period has elapsed, then the request is dropped and a "server busy" message is sent back to the client 102 that submitted the request. This may be the desired result for some requests, but if the request is a request to finalize a transaction, especially one with a large transaction amount, it most likely is not the desired result. To prevent this result from being realized, the evaluation function may be used.
To illustrate, when the request processing mechanism 1 10 determines that there are no free threads in the thread pool, rather than processing the request like any other request, the mechanism 1 10 may invoke the evaluation function referenced in the evaluation function column 220 of the thread pool. When invoked, the evaluation function analyzes and evaluates the request to determine how the request should be handled. If the evaluation function determines, for example, that the request is a request to finalize a $5,000 transaction, then the evaluation function may place the request at the head rather than the tail of the queue. Alternatively, the evaluation function may ensure that the request will not dropped from the queue but rather, will be serviced even if the maximum time out peπod has elapsed. On the other hand, if the evaluation function determines that the request is a request to start a transaction, then the evaluation function may simply put the request at the tail of the queue. The evaluation function is user- provided; thus, it may implement any logic desired by the user. In the example given, the condition prompting the execution of the evaluation function is the unavailability of a free thread, the considerations taken into account in evaluating the request are the type of request and the amount of the request, and the actions taken involve placement of the request on the queue and insurance that the request will not be dropped from the queue. It should be noted that these are illustrative only. For purposes of the invention, any conditions (or none at all) may prompt the execution of the evaluation function, any critena may be taken into account in evaluating the request, and any actions may be taken in disposing of the request. All such possibilities are withm the scope of the present invention.
In system 100 shown in Fig. 1, it is the request processing mechanism 1 10 that is pπmaπly responsible for invoking the evaluation function associated with a thread pool. Thus, it is the request processing mechanism 1 10 that checks for the conditions (whatever they might be) prompting the execution of the evaluation function. Accordingly, the code constituting the request processing mechanism 1 10 comprises logic for checking for these conditions. It should be noted that this is just one possible embodiment. Alternatively, the tasks of checking for the proper conditions and invoking the evaluation function may be performed by one or more other components. This and other modifications are within the scope of the present invention.
Thus far, the invocation of the evaluation function has been described in the context of multiple thread pools. However, it should be noted that the invention is not so limited. Rather, if so desired, the concept of the evaluation function may be implemented effectively in a single thread pool environment. This and other modifications are within the scope of the invention.
Hardware Overview
In one embodiment, the server 106 of the present invention and its various components are implemented as a set of instructions executable by one or more processors. The invention may be implemented as part of an object oriented programming system, including but not limited to the JAVA™ programming system manufactured by Sun Microsystems, Inc. of Mountain View, California. Fig. 6 shows a hardware block diagram of a computer system 600 in which an embodiment of the invention may be implemented. Computer system 600 includes a bus 602 or other communication mechanism for communicating information, and a processor 604 coupled with bus 602 for processing information. Computer system 600 also includes a main memory 606, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 602 for storing information and instructions to be executed by processor 604. Main memory 606 may also be further used to store temporary variables or other intermediate information during execution of instructions by processor 604. Computer system 600 further includes a read only memory (ROM) 608 or other static storage device coupled to bus 602 for storing static information and instructions for processor 604. A storage device 610, such as a magnetic disk or optical disk, is provided and coupled to bus 602 for storing information and instructions.
Computer system 600 may be coupled via bus 602 to a display 612, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 614. including alphanumeric and other keys, is coupled to bus 602 for communicating information and command selections to processor 604. Another type of user input device is cursor control 616, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 604 and for controlling cursor movement on display 612. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
According to one embodiment, the functionality of the present invention is provided by computer system 600 in response to processor 604 executing one or more sequences of one or more instructions contained in main memory 606. Such instructions may be read into main memory 606 from another computer-readable medium, such as storage device 610. Execution of the sequences of instructions contained in main memory 606 causes processor 604 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processor 604 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 610. Volatile media includes dynamic memory, such as main memory 606. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 602. Transmission media can also take the form of acoustic or electromagnetic waves, such as those generated during radio-wave, infra-red, and optical data communications.
Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 604 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 600 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 602. Bus 602 carries the data to main memory 606, from which processor 604 retrieves and executes the instructions. The instructions received by main memory 606 may optionally be stored on storage device 610 either before or after execution by processor 604.
Computer system 600 also includes a communication interface 618 coupled to bus 602. Communication interface 618 provides a two-way data communication coupling to a network link 620 that is connected to a local network 622. For example, communication interface 618 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 618 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 618 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
Network link 620 typically provides data communication through one or more networks to other data devices. For example, network link 620 may provide a connection through local network 622 to a host computer 624 or to data equipment operated by an Internet Service Provider (ISP) 626. ISP 626 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 628. Local network 622 and Internet 628 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 620 and through communication interface 618, which carry the digital data to and from computer system 600, are exemplary forms of carrier waves transporting the information.
Computer system 600 can send messages and receive data, including program code, through the network(s), network link 620 and communication interface 618. In the Internet example, a server 630 might transmit a requested code for an application program through Internet 628. ISP 626. local network 622 and communication interface 618. The received code may be executed by processor 604 as it is received, and/or stored in storage device 610, or other non-volatile storage for later execution. In this manner, computer system 600 may obtain application code in the form of a carrier wave.
At this point, it should be noted that although the invention has been described with reference to a specific embodiment, it should not be construed to be so limited. Various modifications may be made by those of ordinary skill in the art with the benefit of this disclosure without departing from the spirit of the invention. Thus, the invention should not be limited by the specific embodiments used to illustrate it but only by the scope of the appended claims.

Claims

What is claimed is:
1. A computer implemented method for servicing a request, comprising: allocating a plurality of different thread pools, each thread pool comprising one or more threads; receiving a request; processing said request to determine a particular thread pool with which to associate said request, wherein said particular thread pool is one of said plurality of different thread pools; and using a thread from said particular thread pool to carry out servicing of said request.
2. The method of claim 1 , wherein said plurality of different thread pools are allocated within a single process space.
3. The method of claim 2, wherein said particular thread pool has a set of characteristics, wherein said request is a request for a particular type of service, and wherein said set of characteristics for said particular thread pool are customized for said particular type of service.
4. The method of claim 3, wherein said set of characteristics comprises a value for a maximum number of threads in said particular thread pool.
5. The method of claim 3, wherein said set of characteristics comprises a stack size.
6. The method of claim 3, wherein said particular type of service is a JAVA- type service, and wherein said particular thread pool is dedicated to JAVA-type services.
7. The method of claim 3, further comprising: receiving a second request; processing said second request to determine a second particular thread pool with which to associate said second request, wherein said second particular thread pool is one of said plurality of different thread pools; and using a thread from said second particular thread pool to carry out servicing of said second request.
8. The method of claim 7, wherein said second particular thread pool has a second set of characteristics, wherein said second request is a request for a second particular type of service, and wherein said second set of characteristics for said second particular thread pool are customized for said second particular type of service.
9. The method of claim 1 , wherein processing said request comprises: determining, based upon said request, a type of service requested by said request; and determining which one of said plurality of different thread pools is associated with said type of service.
10. The method of claim 1, wherein processing said request comprises: extracting from said request a set of indication information; and determining which one of said plurality of different thread pools is associated with said set of indication information.
11. The method of claim 10, wherein said set of indication information comprises a universal resource identifier (URI).
12. The method of claim 1 , wherein allocating comprises: accessing a set of configuration information, said configuration information specifying a set of characteristics associated with each thread pool of said plurality of different thread pools; and allocating each thread pool of said plurality of different thread pools in accordance with the set of characteristics associated with the thread pool being allocated.
13. The method of claim 12, wherein said set of configuration information is specifiable by a user.
14. An apparatus for servicing a request, comprising: a mechanism for allocating a plurality of different thread pools, each thread pool comprising one or more threads; a mechanism for receiving a request; a mechanism for processing said request to determine a particular thread pool with which to associate said request, wherein said particular thread pool is one of said plurality of different thread pools; and a mechanism for using a thread from said particular thread pool to carry out servicing of said request.
15. The apparatus of claim 14, wherein said plurality of different thread pools are allocated within a single process space.
16. The apparatus of claim 15, wherein said particular thread pool has a set of characteristics, wherein said request is a request for a particular type of service, and wherein said set of characteristics for said particular thread pool are customized for said particular type of service.
17. The apparatus of claim 16, wherein said set of characteristics comprises a value for a maximum number of threads in said particular thread pool.
18. The apparatus of claim 16, wherein said set of characteristics comprises a stack size.
19. The apparatus of claim 16, wherein said particular type of service is a JAVA-type service, and wherein said particular thread pool is dedicated to JAVA-type services.
20. The apparatus of claim 16, further comprising: a mechanism for receiving a second request; a mechanism for processing said second request to determine a second particular thread pool with which to associate said second request, wherein said second particular thread pool is one of said plurality of different thread pools; and a mechanism for using a thread from said second particular thread pool to carry out servicing of said second request.
21. The apparatus of claim 20, wherein said second particular thread pool has a second set of characteristics, wherein said second request is a request for a second particular type of service, and wherein said second set of characteristics for said second particular thread pool are customized for said second particular type of service.
22. The apparatus of claim 14, wherein the mechanism for processing said request comprises: a mechanism for determining, based upon said request, a type of service requested by said request; and a mechanism for determining which one of said plurality of different thread pools is associated with said type of service.
23. The apparatus of claim 14, wherein the mechanism for processing said request comprises: a mechanism for extracting from said request a set of indication information; and a mechanism for determining which one of said plurality of different thread pools is associated with said set of indication information.
24. The apparatus of claim 23, wherein said set of indication information comprises a universal resource identifier (URI).
25. The apparatus of claim 14, wherein the mechanism for allocating comprises: a mechanism for accessing a set of configuration information, said configuration information specifying a set of characteristics associated with each thread pool of said plurality of different thread pools; and a mechanism for allocating each thread pool of said plurality of different thread pools in accordance with the set of characteristics associated with the thread pool being allocated.
26. The apparatus of claim 25. wherein said set of configuration information is specifiable by a user.
27. A computer readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to service a request, said computer readable medium comprising: instructions for causing one or more processors to allocate a plurality of different thread pools, each thread pool comprising one or more threads; instructions for causing one or more processors to receive a request; instructions for causing one or more processors to process said request to determine a particular thread pool with which to associate said request, wherein said particular thread pool is one of said plurality of different thread pools; and instructions for causing one or more processors to use a thread from said particular thread pool to carry out servicing of said request.
28. The computer readable medium of claim 27, wherein said plurality of different thread pools are allocated within a single process space.
29. The computer readable medium of claim 28, wherein said particular thread pool has a set of characteristics, wherein said request is a request for a particular type of service, and wherein said set of characteristics for said particular thread pool are customized for said particular type of service.
30. The computer readable medium of claim 29, wherein said set of characteristics comprises a value for a maximum number of threads in said particular thread pool.
31. The computer readable medium of claim 29, wherein said set of characteristics comprises a stack size.
32. The computer readable medium of claim 29, wherein said particular type of service is a JAVA-type service, and wherein said particular thread pool is dedicated to JAVA-type services.
33. The computer readable medium of claim 29, further comprising: instructions for causing one or more processors to receive a second request; instructions for causing one or more processors to process said second request to determine a second particular thread pool with which to associate said second request, wherein said second particular thread pool is one of said plurality of different thread pools; and instructions for causing one or more processors to use a thread from said second particular thread pool to carry out servicing of said second request.
34. The computer readable medium of claim 33, wherein said second particular thread pool has a second set of characteristics, wherein said second request is a request for a second particular type of service, and wherein said second set of characteristics for said second particular thread pool are customized for said second particular type of service.
35. The computer readable medium of claim 27, wherein the instructions for causing one or more processors to process said request comprises: instructions for causing one or more processors to determine, based upon said request, a type of service requested by said request; and instructions for causing one or more processors to determine which one of said plurality of different thread pools is associated with said type of service.
36. The computer readable medium of claim 27, wherein the instructions for causing one or more processors to process said request comprises: instructions for causing one or more processors to extract from said request a set of indication information; and instructions for causing one or more processors to determine which one of said plurality of different thread pools is associated with said set of indication information.
37. The computer readable medium of claim 36, wherein said set of indication information comprises a universal resource identifier (URI).
38 The computer readable medium of claim 27. wherein the instructions for causing one or more processors to allocate compnses: instructions for causing one or more processors to access a set of configuration information, said configuration information specifying a set of charactenstics associated with each thread pool of said plurality of different thread pools; and instructions for causing one or more processors to allocate each thread pool of said plurality of different thread pools in accordance with the set of charactenstics associated with the thread pool being allocated.
39. The computer readable medium of claim 38, wherein said set of configuration information is specifiable by a user.
PCT/US2000/026151 1999-09-24 2000-09-21 Mechanism for implementing multiple thread pools in a computer system to optimize system performance WO2001022215A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
AU76090/00A AU7609000A (en) 1999-09-24 2000-09-21 Mechanism for implementing multiple thread pools in a computer system to optimize system performance
EP00965364A EP1242871A4 (en) 1999-09-24 2000-09-21 Mechanism for implementing multiple thread pools in a computer system to optimize system performance

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US15571199P 1999-09-24 1999-09-24
US15630599P 1999-09-24 1999-09-24
US60/155,711 1999-09-24
US60/156,305 1999-09-24
US09/574,302 US6542920B1 (en) 1999-09-24 2000-05-19 Mechanism for implementing multiple thread pools in a computer system to optimize system performance
US09/574,302 2000-05-19

Publications (2)

Publication Number Publication Date
WO2001022215A1 true WO2001022215A1 (en) 2001-03-29
WO2001022215A8 WO2001022215A8 (en) 2001-07-19

Family

ID=27387750

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/026151 WO2001022215A1 (en) 1999-09-24 2000-09-21 Mechanism for implementing multiple thread pools in a computer system to optimize system performance

Country Status (4)

Country Link
US (1) US6542920B1 (en)
EP (1) EP1242871A4 (en)
AU (1) AU7609000A (en)
WO (1) WO2001022215A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1274011A1 (en) * 2001-07-06 2003-01-08 Alcatel A method and system for routing and logging a request
WO2004046923A1 (en) * 2002-11-20 2004-06-03 Nokia Corporation Concurrent operation of a state machine family
WO2005048009A2 (en) * 2003-09-22 2005-05-26 Codito Technologies Pvt. Ltd. Method and system for multithreaded processing using errands
WO2006037212A1 (en) * 2004-10-04 2006-04-13 Research In Motion Limited Allocation of threads to user objects in a computer system
US8127010B2 (en) 2004-10-04 2012-02-28 Research In Motion Limited System and method for adaptive allocation of threads to user objects in a computer system
WO2014125314A1 (en) * 2013-02-15 2014-08-21 B-K Medical Aps On demand ultrasound performance
WO2017041398A1 (en) * 2015-09-08 2017-03-16 安一恒通(北京)科技有限公司 Data transmission method and device
CN109992414A (en) * 2019-03-12 2019-07-09 平安普惠企业管理有限公司 A kind of task processing method and device based on thread pool

Families Citing this family (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5850518A (en) * 1994-12-12 1998-12-15 Northrup; Charles J. Access-method-independent exchange
EP1224568A4 (en) 1999-09-24 2006-11-08 Sun Microsystems Inc Mechanism for enabling session information to be shared across multiple processes
US6766349B1 (en) * 1999-09-24 2004-07-20 Sun Microsystems, Inc. Mechanism for obtaining a thread from, and returning a thread to, a thread pool without attaching and detaching
US6701367B1 (en) 1999-09-24 2004-03-02 Sun Microsystems, Inc. Mechanism for enabling customized session managers to interact with a network server
US6898617B2 (en) * 1999-11-18 2005-05-24 International Business Machines Corporation Method, system and program products for managing thread pools of a computing environment to avoid deadlock situations by dynamically altering eligible thread pools
US6836888B1 (en) * 2000-03-17 2004-12-28 Lucent Technologies Inc. System for reverse sandboxing
US6874027B1 (en) * 2000-04-07 2005-03-29 Network Appliance, Inc. Low-overhead threads in a high-concurrency system
US7000028B1 (en) * 2000-06-02 2006-02-14 Verisign, Inc. Automated domain name registration
US6763520B1 (en) * 2000-08-24 2004-07-13 Cognos Incorporated Fair assignment of processing resources to queued requests
US6684262B1 (en) * 2000-10-25 2004-01-27 International Business Machines Corporation Method and system for controlling peripheral device interface behavior using thread registration
US6968557B1 (en) * 2000-12-18 2005-11-22 Stratum8 Corporation Reducing stack memory resources in a threaded computer system
US7188343B2 (en) * 2001-05-18 2007-03-06 Hewlett-Packard Development Company, L.P. Distributable multi-daemon configuration for multi-system management
US7594230B2 (en) 2001-06-11 2009-09-22 Microsoft Corporation Web server architecture
US7225362B2 (en) 2001-06-11 2007-05-29 Microsoft Corporation Ensuring the health and availability of web applications
US7430738B1 (en) 2001-06-11 2008-09-30 Microsoft Corporation Methods and arrangements for routing server requests to worker processes based on URL
US7228551B2 (en) * 2001-06-11 2007-06-05 Microsoft Corporation Web garden application pools having a plurality of user-mode web applications
US7334228B2 (en) * 2001-07-27 2008-02-19 International Business Machines Corporation Runtime-resource management
JP4054572B2 (en) * 2001-12-17 2008-02-27 キヤノン株式会社 Application execution system
US8037153B2 (en) * 2001-12-21 2011-10-11 International Business Machines Corporation Dynamic partitioning of messaging system topics
US7305397B2 (en) * 2002-01-31 2007-12-04 Tririga Llc Caching data communications to reduce latency
US7159025B2 (en) * 2002-03-22 2007-01-02 Microsoft Corporation System for selectively caching content data in a server based on gathered information and type of memory in the server
US6915384B2 (en) * 2002-03-22 2005-07-05 Microsoft Corporation Multiple-level persisted template caching
US7490137B2 (en) * 2002-03-22 2009-02-10 Microsoft Corporation Vector-based sending of web content
US7269752B2 (en) * 2002-06-04 2007-09-11 Lucent Technologies Inc. Dynamically controlling power consumption within a network node
US20030235194A1 (en) * 2002-06-04 2003-12-25 Mike Morrison Network processor with multiple multi-threaded packet-type specific engines
JP2004062446A (en) * 2002-07-26 2004-02-26 Ibm Japan Ltd Information gathering system, application server, information gathering method, and program
US7275247B2 (en) * 2002-09-19 2007-09-25 International Business Machines Corporation Method and apparatus for handling threads in a data processing system
US7478398B2 (en) * 2002-10-31 2009-01-13 Hewlett-Packard Development Company, L.P. Management apparatus and method for data collection including accumulating messages and determining message handlers for processing the accumulated messages
US7237242B2 (en) * 2002-12-31 2007-06-26 International Business Machines Corporation Dynamic thread pool tuning techniques
US7207043B2 (en) * 2002-12-31 2007-04-17 International Business Machines Corporation Programmatic response-time based workload distribution techniques
US7599912B2 (en) * 2003-01-14 2009-10-06 At&T Intellectual Property I, L.P. Structured query language (SQL) query via common object request broker architecture (CORBA) interface
US7181741B2 (en) * 2003-01-30 2007-02-20 Hewlett-Packard Development Company, L.P. Apparatus and method to minimize blocking overhead in upcall based MxN threads
WO2005008546A1 (en) * 2003-07-11 2005-01-27 Computer Associates Think, Inc. Event processor for job scheduling and management
US7373640B1 (en) 2003-07-31 2008-05-13 Network Appliance, Inc. Technique for dynamically restricting thread concurrency without rewriting thread code
US20050050113A1 (en) * 2003-08-21 2005-03-03 International Business Machines Corporation System and method for facilitating data flow between synchronous and asynchronous processes
US7496916B2 (en) * 2003-09-18 2009-02-24 International Business Machines Corporation Service and recovery using multi-flow redundant request processing
US7360064B1 (en) 2003-12-10 2008-04-15 Cisco Technology, Inc. Thread interleaving in a multithreaded embedded processor
US7441101B1 (en) 2003-12-10 2008-10-21 Cisco Technology, Inc. Thread-aware instruction fetching in a multithreaded embedded processor
US7206922B1 (en) 2003-12-30 2007-04-17 Cisco Systems, Inc. Instruction memory hierarchy for an embedded processor
US7703101B2 (en) * 2004-02-13 2010-04-20 International Business Machines Corporation Autonomic workload classification using predictive assertion for wait queue and thread pool selection
US20050198464A1 (en) * 2004-03-04 2005-09-08 Savaje Technologies, Inc. Lazy stack memory allocation in systems with virtual memory
US7784054B2 (en) * 2004-04-14 2010-08-24 Wm Software Inc. Systems and methods for CPU throttling utilizing processes
US7559063B2 (en) * 2004-06-03 2009-07-07 International Business Machines Corporation Program flow control in computer systems
US7418719B2 (en) * 2004-08-31 2008-08-26 Microsoft Corporation Method and system to support a unified process model for handling messages sent in different protocols
US7418712B2 (en) * 2004-08-31 2008-08-26 Microsoft Corporation Method and system to support multiple-protocol processing within worker processes
US7418709B2 (en) * 2004-08-31 2008-08-26 Microsoft Corporation URL namespace to support multiple-protocol processing within worker processes
US7617498B1 (en) * 2004-09-15 2009-11-10 Nortel Networks Limited Resource conflict management using predefined XML schemas
US7987225B2 (en) * 2004-12-22 2011-07-26 International Business Machines Corporation Method for remembering resource allocation in grids
WO2006128925A1 (en) * 2005-05-31 2006-12-07 Imbert Management Consulting Solutions Computational expansion system
US8656402B2 (en) * 2005-08-26 2014-02-18 International Business Machines Corporation Incremental web container growth to control startup request flooding
US8356284B2 (en) * 2006-12-28 2013-01-15 International Business Machines Corporation Threading model analysis system and method
US7899637B2 (en) * 2007-06-13 2011-03-01 Tokyo Electron Limited Method and apparatus for creating a gate optimization evaluation library
US7713758B2 (en) * 2007-06-13 2010-05-11 Tokyo Electon Limited Method and apparatus for optimizing a gate channel
US8209701B1 (en) * 2007-09-27 2012-06-26 Emc Corporation Task management using multiple processing threads
US8209702B1 (en) * 2007-09-27 2012-06-26 Emc Corporation Task execution using multiple pools of processing threads, each pool dedicated to execute different types of sub-tasks
US9043801B2 (en) * 2008-01-15 2015-05-26 International Business Machines Corporation Two-tiered dynamic load balancing using sets of distributed thread pools
DE202008004678U1 (en) 2008-04-02 2008-07-31 Werth, Ulrich, Dr. Implant for long-term acupuncture
DE102008017389A1 (en) 2008-04-02 2009-11-05 Werth Implantat Therapie Center S.L.U. Implant for long-term acupuncture, has two points arranged on implant base body or joined together or bundled to form implant body
US8799397B2 (en) * 2008-05-22 2014-08-05 Red Hat, Inc. Shared transport
US8527866B2 (en) 2010-04-30 2013-09-03 Microsoft Corporation Multi-threaded sort of data items in spreadsheet tables
US20110276868A1 (en) * 2010-05-05 2011-11-10 Microsoft Corporation Multi-Threaded Adjustment of Column Widths or Row Heights
US9886315B2 (en) * 2010-08-27 2018-02-06 Ebay Inc. Identity and semaphore-based quality of service
US8856764B2 (en) * 2011-01-25 2014-10-07 International Business Machines Corporation Distributed static analysis of computer software applications
US9069598B2 (en) 2012-01-06 2015-06-30 International Business Machines Corporation Providing logical partions with hardware-thread specific information reflective of exclusive use of a processor core
WO2015180667A1 (en) * 2014-05-28 2015-12-03 Mediatek Inc. Computing system with reduced data exchange overhead and related data exchange method thereof
US10885023B1 (en) * 2014-09-08 2021-01-05 Amazon Technologies, Inc. Asynchronous processing for synchronous requests in a database
US11301142B2 (en) * 2016-06-06 2022-04-12 Vmware, Inc. Non-blocking flow control in multi-processing-entity systems
CN107798111B (en) * 2017-11-01 2021-04-06 四川长虹电器股份有限公司 Method for exporting data in large batch in distributed environment
US11340955B2 (en) 2020-01-02 2022-05-24 International Business Machines Corporation Thread pool management for multiple applications

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5872963A (en) * 1997-02-18 1999-02-16 Silicon Graphics, Inc. Resumption of preempted non-privileged threads with no kernel intervention
US5961584A (en) * 1994-12-09 1999-10-05 Telefonaktiebolaget Lm Ericsson System for managing internal execution threads
US5991792A (en) * 1998-01-02 1999-11-23 International Business Machines Corporation Method, apparatus and computer program product for dynamically managing a thread pool of reusable threads in a computer system
US6112196A (en) * 1998-06-25 2000-08-29 International Business Machines Corporation Method and system for managing connections to a database management system by reusing connections to a database subsystem

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE69230905T2 (en) 1991-06-28 2000-08-17 Ericsson Telefon Ab L M USER MODULES IN TELECOMMUNICATION SYSTEMS
US5410698A (en) 1993-10-12 1995-04-25 Intel Corporation Method and system for dynamic loading of software libraries
EP0694837A1 (en) * 1994-07-25 1996-01-31 International Business Machines Corporation Dynamic workload balancing
US5793415A (en) 1995-05-15 1998-08-11 Imagetel International Inc. Videoconferencing and multimedia system
US5809248A (en) 1995-07-05 1998-09-15 Sun Microsystems, Inc. Method and apparatus for front end navigator and network architecture for performing functions on distributed files in a computer network
US6182109B1 (en) * 1996-03-08 2001-01-30 International Business Machines Corporation Dynamic execution unit management for high performance user level network server system
US5928323A (en) * 1996-05-30 1999-07-27 Sun Microsystems, Inc. Apparatus and method for dynamically generating information with server-side software objects
US5809145A (en) 1996-06-28 1998-09-15 Paradata Systems Inc. System for distributing digital information
US6058460A (en) * 1996-06-28 2000-05-02 Sun Microsystems, Inc. Memory allocation in a multithreaded environment
US6052711A (en) 1996-07-01 2000-04-18 Sun Microsystems, Inc. Object-oriented system, method and article of manufacture for a client-server session web access in an interprise computing framework system.
US5835724A (en) 1996-07-03 1998-11-10 Electronic Data Systems Corporation System and method for communication information using the internet that receives and maintains information concerning the client and generates and conveys the session data to the client
US5790790A (en) 1996-10-24 1998-08-04 Tumbleweed Software Corporation Electronic document delivery system in which notification of said electronic document is sent to a recipient thereof
US5918228A (en) 1997-01-28 1999-06-29 International Business Machines Corporation Method and apparatus for enabling a web server to impersonate a user of a distributed file system to obtain secure access to supported web documents
US6012090A (en) 1997-03-14 2000-01-04 At&T Corp. Client-side parallel requests for network services using group name association
US6418458B1 (en) 1998-10-02 2002-07-09 Ncr Corporation Object-oriented prioritized work thread pool

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5961584A (en) * 1994-12-09 1999-10-05 Telefonaktiebolaget Lm Ericsson System for managing internal execution threads
US5872963A (en) * 1997-02-18 1999-02-16 Silicon Graphics, Inc. Resumption of preempted non-privileged threads with no kernel intervention
US5991792A (en) * 1998-01-02 1999-11-23 International Business Machines Corporation Method, apparatus and computer program product for dynamically managing a thread pool of reusable threads in a computer system
US6112196A (en) * 1998-06-25 2000-08-29 International Business Machines Corporation Method and system for managing connections to a database management system by reusing connections to a database subsystem

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1242871A4 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1274011A1 (en) * 2001-07-06 2003-01-08 Alcatel A method and system for routing and logging a request
WO2004046923A1 (en) * 2002-11-20 2004-06-03 Nokia Corporation Concurrent operation of a state machine family
WO2005048009A2 (en) * 2003-09-22 2005-05-26 Codito Technologies Pvt. Ltd. Method and system for multithreaded processing using errands
WO2005048009A3 (en) * 2003-09-22 2008-05-29 Codito Technologies Pvt Ltd Method and system for multithreaded processing using errands
WO2006037212A1 (en) * 2004-10-04 2006-04-13 Research In Motion Limited Allocation of threads to user objects in a computer system
US8127010B2 (en) 2004-10-04 2012-02-28 Research In Motion Limited System and method for adaptive allocation of threads to user objects in a computer system
US8671410B2 (en) 2004-10-04 2014-03-11 Blackberry Limited System and method for allocation of threads to user objects in a computer system
WO2014125314A1 (en) * 2013-02-15 2014-08-21 B-K Medical Aps On demand ultrasound performance
CN105027082A (en) * 2013-02-15 2015-11-04 B-K医疗公司 On demand ultrasound performance
WO2017041398A1 (en) * 2015-09-08 2017-03-16 安一恒通(北京)科技有限公司 Data transmission method and device
CN109992414A (en) * 2019-03-12 2019-07-09 平安普惠企业管理有限公司 A kind of task processing method and device based on thread pool

Also Published As

Publication number Publication date
WO2001022215A8 (en) 2001-07-19
US6542920B1 (en) 2003-04-01
EP1242871A4 (en) 2006-11-15
AU7609000A (en) 2001-04-24
EP1242871A1 (en) 2002-09-25

Similar Documents

Publication Publication Date Title
US6542920B1 (en) Mechanism for implementing multiple thread pools in a computer system to optimize system performance
US6766349B1 (en) Mechanism for obtaining a thread from, and returning a thread to, a thread pool without attaching and detaching
US6604125B1 (en) Mechanism for enabling a thread unaware or non thread safe application to be executed safely in a multi-threaded environment
US6895584B1 (en) Mechanism for evaluating requests prior to disposition in a multi-threaded environment
JP3853592B2 (en) Distributed web application server
US6882999B2 (en) URL mapping methods and systems
JP4729172B2 (en) Method and apparatus for performing transactions in a stateless web environment that supports a declarative paradigm
EP1076290B1 (en) Method for on-demand network application download and execution
US8103779B2 (en) Mechanism for enabling session information to be shared across multiple processes
EP1309914B1 (en) Accessing legacy applications from the internet
US6950866B1 (en) XML-based integrated services parsing
US7222152B1 (en) Generic communications framework
US20050198079A1 (en) Process and system for real-time relocation of objects during garbage collection
US6163812A (en) Adaptive fast path architecture for commercial operating systems and information server applications
US20020156897A1 (en) Mechanism for servicing connections by disassociating processing resources from idle connections and monitoring the idle connections for activity
US20060129512A1 (en) Socket-like communication API for C
US20060130063A1 (en) Fast platform independent inter-process communication
US7593930B2 (en) Fast channel architecture
US7500080B2 (en) Facilitating non-contiguous allocation of a large object within a java heap
JPH07281974A (en) Communication system for exchange of data between computers in network
US7536688B2 (en) Segmented virtual machine
KR20010041297A (en) Method and apparatus for the suspension and continuation of remote processes
US20060248130A1 (en) Process and system for real-time relocation of objects during garbage collection
US7266622B2 (en) Method, computer program product, and system for automatic application buffering
JP2001056767A (en) Method for cleaning up internal state by using transaction service synchronous interface

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: C1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: C1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

CFP Corrected version of a pamphlet front page

Free format text: REVISED ABSTRACT RECEIVED BY THE INTERNATIONAL BUREAU AFTER COMPLETION OF THE TECHNICAL PREPARATIONS FOR INTERNATIONAL PUBLICATION

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2000965364

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWP Wipo information: published in national office

Ref document number: 2000965364

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: JP