CA2461015A1 - A system and method for a multi-node environment with shared storage - Google Patents
A system and method for a multi-node environment with shared storage Download PDFInfo
- Publication number
- CA2461015A1 CA2461015A1 CA002461015A CA2461015A CA2461015A1 CA 2461015 A1 CA2461015 A1 CA 2461015A1 CA 002461015 A CA002461015 A CA 002461015A CA 2461015 A CA2461015 A CA 2461015A CA 2461015 A1 CA2461015 A1 CA 2461015A1
- Authority
- CA
- Canada
- Prior art keywords
- node
- storage
- operating system
- membership group
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0793—Remedial or corrective actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
- G06F11/0709—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/526—Mutual exclusion algorithms
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99931—Database or file accessing
- Y10S707/99938—Concurrency, e.g. lock management in shared database
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
Abstract
A system and method are disclosed for providing multi-node environment comprising a first node (306a) associated with a first operating system (300a); a second node associated with a second operating system, wherein the second operating system is independent of the first operating system; a storage; and an interconnect (304) coupling the second node with the storage;
and wherein the first node directly accesses the storage and the second node directly accesses the storage.
and wherein the first node directly accesses the storage and the second node directly accesses the storage.
Description
MULTI-NODE ENVIRONMENT WITH SHARED
STORAGE
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. Provisional Patent Application No.
60/324,196 (Attorney Docket No. POLYP001+) entitled SHARED STORAGE
LOCK: A NEW SOFTWARE SYNCHRONIZATION MECHANISM FOR
ENFORCING MUTUAL EXCLUSION AMONG MULTIPLE NEGOTIATORS
filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,226 (Attorney Docket No. POLYP002+) entitled JOUNALING
MECHANISM WITH EFFICIENT, SELECTIVE RECOVERY FOR MULTI-NODE
ENVIRONMENTS filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,224 (Attorney Docket No. POLYP003+) entitled COLLABORATIVE
CACHING IN A MULTI-NODE FILESYSTEM filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No 60/324,242 (Attorney Docket No. POLYP005+) entitled DISTRIBUTED
MANAGEMENT OF A STORAGE AREA NETWORK filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,195 (Attorney Docket No. POLYP006+) entitled METHOD FOR
IMPLEMENTING JOURNALING AND DISTRIBUTED LOCK MANAGEMENT
filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,243 (Attorney Docket No. POLYP007+) entitled MATRIX SERVER: A
HIGHLY AVAILABLE MATRIX PROCESSING SYSTEM WITH COHERENT
SHARED FILE STORAGE filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,787 (Attorney Docket No. POLYP008+) entitled A METHOD FOR
PROCESSING SYSTEM filed September 24, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/327,191 (Attorney Docket No. POLYP009+) entitled FAST LOCK RECOVERY:
AVAILABLE MATRIX PROCESSING SYSTEM filed October 1, 2001, which is incorporated herein by reference for all purposes.
This application is related to co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP001) entitled A SYSTEM AND METHOD
FOR SYNCHRONIZATION FOR ENFORCING MUTUAL EXCLUSION AMONG
MULTIPLE NEGOTIATORS filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP002) entitled SYSTEM AND METHOD
FOR JOURNAL RECOVERY FOR MULTINODE ENVIRONMENTS filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No. (Attorney Docket No. POLYP003) entitled A SYSTEM AND METHOD FOR COLLABORATIVE CACHING IN A
MULTINODE SYSTEM filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP005) entitled A SYSTEM AND METHOD FOR
MANAGEMENT OF A STORAGE AREA NETWORK filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No. (Attorney Docket No. POLYP006) entitled SYSTEM
AND METHOD FOR IMPLEMENTING JOURNALING IN A MULTI-NODE
ENVIRONMENT filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP009) entitled A SYSTEM AND METHOD FOR
EFFICIENT LOCK RECOVERY filed concurrently herewith, which is incorporated herein by reference for all purposes.
FIELD OF THE INVENTION
The present invention relates generally to computer systems. In particular, the present invention relates to computer systems that share resources such as storage.
BACKGROUND OF THE INVENTION
Servers are typically used for big applications and work loads such as those used in conjunction with large web services and manufacturing. Often, a single server does not have enough power to perform the required application. Several servers may be used in conjunction with several storage devices in a storage area network (SAN) to accommodate heavy traffic. As systems get larger, applications often become important enough to avoid shutting off access to perform maintenance.
A typical server management system uses a single management control station that manages the servers and the shared storage. A potential problem of such a system is that it may have a single point of failure which can cause a shut-down of the entire storage area network to perform maintenance. Another potential problem is that there is typically no dynamic cooperation between the servers in case a change to the system occurs.
Such systems typically use large mainframes. A problem with the mainframes is that they are very expensive. Another possible system may use smaller computers but this solution typically requires customized hardware as well as a customized operating system that coordinates the computers to work as though it was one large machine with one operating system between them. Obtaining and maintaining customized hardware can be very costly.
What is needed is a system and method for a multi-node environment with shared storage provided at a lower price. The present invention addresses such a need.
BRIEF DESCRIPTION OF THE DRAWINGS
STORAGE
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. Provisional Patent Application No.
60/324,196 (Attorney Docket No. POLYP001+) entitled SHARED STORAGE
LOCK: A NEW SOFTWARE SYNCHRONIZATION MECHANISM FOR
ENFORCING MUTUAL EXCLUSION AMONG MULTIPLE NEGOTIATORS
filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,226 (Attorney Docket No. POLYP002+) entitled JOUNALING
MECHANISM WITH EFFICIENT, SELECTIVE RECOVERY FOR MULTI-NODE
ENVIRONMENTS filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,224 (Attorney Docket No. POLYP003+) entitled COLLABORATIVE
CACHING IN A MULTI-NODE FILESYSTEM filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No 60/324,242 (Attorney Docket No. POLYP005+) entitled DISTRIBUTED
MANAGEMENT OF A STORAGE AREA NETWORK filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,195 (Attorney Docket No. POLYP006+) entitled METHOD FOR
IMPLEMENTING JOURNALING AND DISTRIBUTED LOCK MANAGEMENT
filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,243 (Attorney Docket No. POLYP007+) entitled MATRIX SERVER: A
HIGHLY AVAILABLE MATRIX PROCESSING SYSTEM WITH COHERENT
SHARED FILE STORAGE filed September 21, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/324,787 (Attorney Docket No. POLYP008+) entitled A METHOD FOR
PROCESSING SYSTEM filed September 24, 2001, which is incorporated herein by reference for all purposes.
This application claims priority to U.S. Provisional Patent Application No.
60/327,191 (Attorney Docket No. POLYP009+) entitled FAST LOCK RECOVERY:
AVAILABLE MATRIX PROCESSING SYSTEM filed October 1, 2001, which is incorporated herein by reference for all purposes.
This application is related to co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP001) entitled A SYSTEM AND METHOD
FOR SYNCHRONIZATION FOR ENFORCING MUTUAL EXCLUSION AMONG
MULTIPLE NEGOTIATORS filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP002) entitled SYSTEM AND METHOD
FOR JOURNAL RECOVERY FOR MULTINODE ENVIRONMENTS filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No. (Attorney Docket No. POLYP003) entitled A SYSTEM AND METHOD FOR COLLABORATIVE CACHING IN A
MULTINODE SYSTEM filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP005) entitled A SYSTEM AND METHOD FOR
MANAGEMENT OF A STORAGE AREA NETWORK filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No. (Attorney Docket No. POLYP006) entitled SYSTEM
AND METHOD FOR IMPLEMENTING JOURNALING IN A MULTI-NODE
ENVIRONMENT filed concurrently herewith, which is incorporated herein by reference for all purposes; and co-pending U.S. Patent Application No.
(Attorney Docket No. POLYP009) entitled A SYSTEM AND METHOD FOR
EFFICIENT LOCK RECOVERY filed concurrently herewith, which is incorporated herein by reference for all purposes.
FIELD OF THE INVENTION
The present invention relates generally to computer systems. In particular, the present invention relates to computer systems that share resources such as storage.
BACKGROUND OF THE INVENTION
Servers are typically used for big applications and work loads such as those used in conjunction with large web services and manufacturing. Often, a single server does not have enough power to perform the required application. Several servers may be used in conjunction with several storage devices in a storage area network (SAN) to accommodate heavy traffic. As systems get larger, applications often become important enough to avoid shutting off access to perform maintenance.
A typical server management system uses a single management control station that manages the servers and the shared storage. A potential problem of such a system is that it may have a single point of failure which can cause a shut-down of the entire storage area network to perform maintenance. Another potential problem is that there is typically no dynamic cooperation between the servers in case a change to the system occurs.
Such systems typically use large mainframes. A problem with the mainframes is that they are very expensive. Another possible system may use smaller computers but this solution typically requires customized hardware as well as a customized operating system that coordinates the computers to work as though it was one large machine with one operating system between them. Obtaining and maintaining customized hardware can be very costly.
What is needed is a system and method for a multi-node environment with shared storage provided at a lower price. The present invention addresses such a need.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like structural elements, and in which:
Figure 1 is a block diagram of an example of a typical server system.
Figure 2 is a block diagram of another example of a typical server system.
Figure 3 is a block diagram of a system for a multi-node environment according to an embodiment of the present invention.
Figure 4 is another block diagram of a system according to an embodiment of the present invention.
Figure 5 is a block diagram of the software components of server 300 according to an embodiment of the present invention.
Figure 6 is a flow diagram of a method for a multi-node environment according to an embodiment of the present invention.
Figures 7A-7C are other flow diagrams of a method according to an embodiment of the present invention for a mufti-node environment.
Figure 8 is a flow diagram of a method according to an embodiment of the present invention for reading a file.
Figures 9A-9B are flow diagrams of a method according to an embodiment of the present invention for writing to a file.
Figs. l0A-lOB are flow diagrams of a method according to an embodiment of the present invention for adding a node to a cluster of servers sharing storage such as a disk.
Figs. 11A-11C are flow diagrams of a method according to the present invention for handling a server failure.
Fig. 12 is flow diagram of a method according to an embodiment of the present invention for adding or removing shared storage.
DETAILED DESCRIPTION
It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, or a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or electronic communication links.
It should be noted that the order of the steps of disclosed processes may be altered within the scope of the invention.
A detailed description of one or more preferred embodiments of the invention are provided below along with accompanying figures that illustrate by way of example the principles of the invention. While the invention is described in connection with such embodiments, it should be understood that the invention is not limited to any embodiment. On the contrary, the scope of the invention is limited only by the appended claims and the invention encompasses numerous alternatives, modifications and equivalents. For the purpose of example, numerous specific details are set forth in the following description in order to provide a thorough understanding of the present invention. The present invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the present invention is not unnecessarily obscured.
Figure 1 is a block diagram of a conventional server system. In this example, computers 100A-100D are networked together by network 102. Each of the computers 100A-100D have their own locally connected storage 104A-104D.
Business computing typically requires regular sharing of information but this type of system can be slow to access shared information or it might require very expensive customized hardware.
Figure 2 is a block diagram of another conventional system. In this example, the various servers 200A-200D all use a single operating system across all of the servers 200A-200D. This type of system can be very expensive since it typically requires customized hardware. Another problem with this type of system is that if there is a fault in a single computer 200A-200D, the entire multi-server system would likely need to shut down. Accordingly, each computer needs to be more reliable than the standard computer, thus further raising costs.
What is needed is a system and method for a multi-node environment that does not require customized hardware components. The present invention addresses such a need.
Figure 3 is a block diagram of a system for a mufti-node environment according to an embodiment of the present invention. In this example, servers 300D are coupled via network interconnects 302. The network interconnects 302 can represent any network infrastructure such as an Ethernet, InfiniBand network or Fibre Channel network capable of host-to-host communication. The servers 300A-300D
are also coupled to the data storage interconnect 304, which in turn is coupled to shared storage 306A-306D. The data storage interconnect 304 can be any S interconnect that can allow access to the shared storage 306A-306D by servers 300A-300D. An example of the data storage interconnect 304 is a Fibre Channel switch, such as a Brocade 3200 Fibre Channel switch. Alternately, the data storage network might be an iSCSI or other IP storage network, InfiniBand network, or another kind of host-to-storage network. In addition, the network interconnects 302 and the data storage interconnect 304 may be embodied in a single interconnect.
Servers 300A-300D can be any computer, preferable an off the-shelf computer or server or any equivalent thereof. Servers 300A-300D can each run operating systems that are independent of each other. Accordingly, each server 300A-300D
can, but does not need to, run a different operating system. For example, server 300A
may run Microsoft windows, while server 300B runs Linux, and server 300C can simultaneously run a Unix operating system. An advantage of running independent operating systems for the servers 300A-300D is that the entire mufti-node system can be dynamic. For example, one of the servers 300A-300D can fail while the other servers 300A-300D continue to operate.
The shared storage 306A-306D can be any storage device, such as hard drive disks, compact disks, tape, and random access memory. A filesystem is a logical entity built on the shared storage. Although the shared storage 306A-306D is typically considered a physical device while the filesystem is typically considered a logical structure overlaid on part of the storage, the filesystem is sometimes referred to herein as shared storage for simplicity. For example, when it is stated that shared storage fails, it can be a failure of a part of a filesystem, one or more filesystems, or the physical storage device on which the filesystem is overlaid. Accordingly, shared storage, as used herein, can mean the physical storage device, a portion of a filesystem, a filesystem, filesystems, or any combination thereof.
Figure 4 is another block diagram of a system according to an embodiment of the present invention. In this example, the system preferably has no single point of failure. Accordingly, servers 300A' - 300D' are coupled with multiple network interconnects 302A-302D. The servers 300A'-300D' are also shown to be coupled with multiple storage interconnects 304A-304B. The storage interconnects 304A-304B are each coupled to a plurality of data storage 306A'-306D'.
In this manner, there is redundancy in the system such that if any of the components or connections fail, the entire system can continue to operate.
In the example shown in Figure 4, as well as the example shown in Figure 3, the number of servers 300A'-300D', the number of storage interconnects 304A-304B, and the number of data storage 306A'-306D' can be as many as the customer requires and is not physically limited by the system. Likewise, the operating systems used by servers 300A'-300D' can also be as many independent operating systems as the customer requires.
Figure 5 is a block diagram of the software components of server 300 according to an embodiment of the present invention. In an embodiment of the present invention, each server 300A-300D of Fig 3 includes these software components.
In this embodiment, the following components are shown:
The Distributed Lock Manager (DLM) 500 manages matrix-wide locks for the filesystem image 306a-306d, including the management of lock state during crash recovery. The Matrix Filesystem 504 uses DLM S00-managed locks to implement matrix-wide mutual exclusion and matrix-wide filesystem 306a-306d metadata and data cache consistency. The DLM 500 is a distributed symmetric lock manager.
Preferably, there is an instance of the DLM 500 resident on every server in the matrix.
Every instance is a peer to every other instance; there is no master/slave relationship among the instances.
The lock-caching layer ("LCL") 502 is a component internal to the operating system kernel that interfaces between the Matrix Filesystem 504 and the application-level DLM 500. The purposes of the LCL 502 include the following:
1. It hides the details of the DLM 500 from kernel-resident clients that need to obtain distributed locks.
2. It caches DLM 500 locks (that is, it may hold on to DLM 500 locks after clients have released all references to them), sometimes obviating the need for kernel components to communicate with an application-level process (the DLM S00) to obtain matrix-wide locks.
3. It provides the ability to obtain locks in both process and server scopes (where a process lock ensures that the corresponding DLM (500) lock is held, and also excludes local processes attempting to obtain the lock in conflicting modes, whereas a server lock only ensures that the DLM (500) lock is held, without excluding other local processes).
4. It allows clients to define callouts for different types of locks when certain events related to locks occur, particularly the acquisition and surrender of level locks. This ability is a requirement for cache-coherency, which depends on callouts to flush modified cached data to permanent storage when corresponding DLM 500 write locks are downgraded or released, and to purge cached data when DLM 500 read locks are released.
The LCL 502 is the only kernel component that makes lock requests from the user-level DLM 500. It partitions DLM 500 locks among kernel clients, so that a single DLM 500 lock has at most one kernel client on each node, namely, the LCL
502 itself. Each DLM 500 lock is the product of an LCL 502 request, which was induced by a client's request of an LCL 502 lock, and each LCL 502 lock is backed by a DLM 500 lock.
The Matrix Filesystem 504 is the shared filesystem component of The Matrix Server. The Matrix Filesystem 504 allows multiple servers to simultaneously mount, in read/write mode, filesystems living on physically shared storage devices 306a-306d. The Matrix Filesystem 504 is a distributed symmetric matrixed filesystem;
there is no single server that filesystem activity must pass through to perform filesystem activities. The Matrix Filesystem 504 provides normal local filesystem semantics and interfaces for clients of the filesystem.
SAN (Storage Area Network) Membership Service 506 provides the group membership services infrastructure for the Matrix Filesystem 504, including managing filesystem membership, health monitoring, coordinating mounts and unmounts of shared filesystems 306a-306d, and coordinating crash recovery.
Matrix Membership Service 508 provides the Local, matrix-style matrix membership support, including virtual host management, service monitoring, notification services, data replication, etc. The Matrix Filesystem 504 does not interface directly with the MMS 508, but the Matrix Filesystem 504 does interface with the SAN Membership Service 506, which interfaces with the MMS 508 in order to provide the filesystem 504 with the matrix group services infrastructure.
The Shared Disk Monitor Probe S 10 maintains and monitors the membership of the various shared storage devices in the matrix. It acquires and maintains leases on the various shared storage devices in the matrix as a protection against rogue server "split-brain" conditions. It communicates with the SMS 506 to coordinate recovery activities on occurrence of a device membership transition.
Filesystem monitors 512 are used by the SAN Membership Service 508 to 1 S initiate Matrix Filesystem 504 mounts and unmounts, according to the matrix configuration put in place by the Matrix Server user interface.
The Service Monitor 514 tracks the state (health & availability) of various services on each server in the matrix so that the matrix server may take automatic remedial action when the state of any monitored service transitions. Services monitored include HTTP, FTP, Telnet, SMTP, etc. The remedial actions include service restart on the same server or service fail-over and restart on another server.
The Device Monitor 516 tracks the state (health & availability) of various storage-related devices in the matrix so that the matrix server may take automatic remedial action when the state of any monitored device transitions. Devices monitored may include data storage devices 306a-306d (such as storage device drives, solid state storage devices, ram storage devices, JOBDs, RAID arrays, etc.)and storage network devices 304' (such as FibreChannel Switches, Infiniband Switches, iSCSI switches, etc.). The remedial actions include initiation of Matrix Filesystem 504 recovery, storage network path failover, and device reset.
The Application Monitor 518 tracks the state (health & availability) of various applications on each server in the matrix so that the matrix server may take automatic remedial action when the state of any monitored application transitions.
Applications monitored may include databases, mail routers, CRM apps, etc. The remedial actions include application restart on the same server or application fail-over and restart on another server.
The Notifier Agent 520 tracks events associated with specified objects in the matrix and executes supplied scripts of commands on occurrence of any tracked event.
The Replicator Agent 522 monitors the content of any filesystem subtree and periodically replicates any data which has not yet been replicated from a source tree to a destination tree. Replication is preferably used for subtrees not placed in shared storage The Matrix Communication Service 524 provides the network communication infrastructure for the DLM 500, Matrix Membership Service 508, and SAN
Membership Service 506. The Matrix Filesystem 504 does not use the MCS 524 directly, but it does use it indirectly through these other components.
The Storage Control Layber (SCL) 526 provides matrix-wide device identification, used to identify the Matrix Filesystems 504 at mount time. The SCL
526 also manages storage fabric configuration and low level I/O device fencing of rogue servers from the shared storage devices 306a-306d containing the Matrix Filesystems 504. It also provides the ability for a server in the matrix to voluntarily intercede during normal device operations to fence itself when communication with rest of the matrix has been lost.
The Storage Control Layer 526 is the Matrix Server module responsible for managing shared storage devices 306a-306d. Management in this context consists of two primary functions. The first is to enforce I/O fencing at the hardware SAN
level by enabling/disabling host access to the set of shared storage devices 306a-306d. And the second is to generate global (matrix-wide) unique device names (or "labels") for all matrix storage devices 306a-306d and ensure that all hosts in the matrix have access to those global device names. The SCL module also includes utilities and library routines needed to provide device information to the UI.
The Pseudo Storage Driver 528 is a layered driver that "hides" a target storage device 306a-306d so that all references to the underlying target device must pass through the PSD layered driver. Thus, the PSD provides the ability to "fence"
a device, blocking all I/O from the host server to the underlying target device until it is unfenced again. The PSD also provides an application-level interface to lock a storage partition across the matrix. It also has the ability to provide common matrix-wide 'handles', or paths, to devices such that all servers accessing shared storage in the Matrix Server can use the same path to access a given shared device.
Figure 6 is a flow diagram of a method for a mufti-node environment according to an embodiment of the present invention. In this example, a first operating system is provided (600), and a second operating system is also provided (602). The second operating system is preferably independent of the first operating system. They can be the same operating system, such as both operating systems being Linux, or different operating systems, such as different versions of Windows or a Unix and a Linux, but each running a separate OS rather than a combined OS. A
storage is also provided (604). An interconnect coupling the first operating system with the storage and coupling the second operating system with the storage is then provided (606). The storage is then directly accessed by the first operating system (608), and the storage is also directly accessed by the second operating system (610).
Figures 7A-7C are other flow diagrams of a method according to an embodiment of the present invention for a mufti-node environment. The following terms are used herein.
Group membership is used herein to refer to a set of cooperating processes (programs) that form a communication group. For example, a group membership can be formed between servers. SAN membership is used herein to refer to a storage area membership. SAN membership refers to the group of servers that are communicating which are allowed to read/write the Storage area network resources such as disks and switches. Shared storage group membership is used herein to refer to the set of servers actively using a single filesystem located on the SAN. An example is the set of servers that have "mounted" the filesystem to make it accessible to user programs.
Cluster membership refers to the set of servers forming a single cluster which actively share filesystems and network resources as a logical whole.
In this example, a membership of a cluster of servers is determined (700). As previously mentioned, servers are used as an example, however, any node, computer or processor can be used. A cluster, as used herein, can be any two or more servers, computers, processors, or any combination thereof, that is associated with a membership.
It is then determined whether the membership of the cluster has changed (702). At time zero, there will be no change to the membership since there is no history of the membership. At some later time, the membership may be changed, for example, if a server is taken off line, or a new server is added. In addition to the change in the numbers of servers that are utilized, determining whether the membership has changed can also accommodate a situation where a server has failed, a server has been added, or has been taken out of the cluster.
If the cluster membership has not changed (702), then each server sends messages to the other servers and monitors the messages to see if the other servers are active (704). It is then determined whether all servers respond (706). If all the other servers in the cluster respond to the messages, then the storage area network (Sale membership service (SMS) is informed of the active members of the cluster (708). If, however, all servers in the cluster do not respond (706), then message activity is blocked (716). Blocking message activity avoids stale messages between membership changes. Once message activity is blocked, the membership of the cluster is again determined (700).
If the cluster membership has changed (702), then new locks are no longer granted (710). It is then determined whether there is an administrator (ADM) in this cluster (712). If there is no administrator in this cluster, then one of the members of the cluster is elected as administrator (714). In the example shown in Figure 5, the SAN membership service (SMS) 506 can be used as an administrator.
The administrator verifies that the other servers in this cluster are part of this storage area membership (720) of Figure 7B. Step 720 accommodates both when all of the servers are part of the cluster, or when there are servers outside the cluster.
There may be separate clusters operating in the storage area network either by design or by accident. In either case servers misconfigured software may result in a server attaching to the shared storage and attempting to access it, without knowledge of the valid cluster. If nodes are outside the cluster then the administrator excludes (fences) those servers to prevent corruption of data on shared storage.
Servers that successfully gain membership to the network cluster are then allowed access to the shared storage and are then part of the SAN membership.
All cluster non-members are then excluded and all cluster members are allowed into the shared storage group (722). Accordingly, servers outside the cluster membership are excluded from accessing the discs 306A-306D of Figure 5.
Shared storage is then set to equal one (724). It is determined by the ADM
whether recovery is necessary for a member of this shared storage group (726).
If no recovery is required for a member of this shared storage group, then it is determined whether this is the last shared storage (732). If it is not the last shared storage, then the shared storage is then set to shared storage plus one (730), and the next shared storage is evaluated by having the administrator determine whether recovery is necessary for a member of that shared storage (726). If, however, this shared storage was the last shared storage, then the granting of new locks are resumed for shared storages that are accessible (734).
If the administrator determines that recovery is necessary for a member of this shared storage (726), then it is determined which members) of this shared storage needs recovery (728). For example, the SAN membership service (SMS) of each server can tell their distributed lock manager (DLM) whether that server needs recovery.
The server that needs recovery is analyzed. In this example, the DLM for the member that needs recovery is set equal to one (750). Locks which can trigger journal recovery in the shared storage by the matrix file system (MFS) and the lock caching layer (LCL) are then recovered (752).
It is then determined whether recovery was successful (754). If the recovery was successful then it is determined whether this server was the last server that required recovery (762). If this server was not the last server that required recovery, then the next server that needed recovery is analyzed by setting DLM equals DLM
plus one (764). The locks are then recovered for this next server (752). If, however, this server was the last server that needed recovery (762), then it is determined whether this shared storage is the last shared storage that needed to be analyzed (732 of Figure 7B).
If recovery of the locks was not successful (754 of Figure 7C), then that failed shared storage is marked as error (756), and an alert is sent to a human system operator (758).
The failed shared storage is then isolated and access to that shared storage is prevented (760). It is then determined whether this shared storage is the last shared storage that needs to be analyzed (732 of Figure 7B).
Figure 8 is a flow diagram of a method according to an embodiment of the present invention for reading a file from shared storage. The operating system of a server in a cluster requests to read a part of a file (800).
A shared lock request is provided (802). For example, the matrix file system (MFS) 504 of Figure S asks the lock caching layer (LCL) 502 of Figure S for a shared lock. It is then determined whether the request is granted (804). If the request for the 1 S shared lock is not granted, then it is determined whether there is a shared storage failure (806).
If there is a share storage failure, then the shared storage is isolated if necessary (808). If, however, there is no shared storage failure then the MFS
asks the LCL for a shared lock (802). It is then determined whether the shared lock request is granted (804).
If the shared lock request is granted (804), then it is determined whether the server is able to read a piece of the requested file from the shared storage into the operating system buffer cache (810). If the server is able to read the file into the OS
buffer cache then the process is complete. If, however, the server is unable to read into the OS buffer cache then the shared lock is dropped and an error is returned to the operating system or application (812). An example of when such an error might occur is if there is a disk failure or a connection failure.
Figures 9A-9B are flow diagrams of a method according to an embodiment of the present invention of writing to a file in a shared storage.
In this example, the operating system of a server requests to write to a part of a file (900). The matrix file system (MFS) 504 of Figure 5 asks the lock caching layer (LCL) for an exclusive lock (902). It is then determined whether the request for an exclusive lock is granted (904). If the request for the exclusive lock is not granted then it is determined whether there is a shared storage failure (920). If there is a shared storage failure, then that shared storage is isolated, if necessary (922). If, however, there is no shared storage failure (920), then there is another request for the 1 S exclusive lock (902).
If the request for an exclusive lock is granted (904), then it is determined whether there is a stable copy of the requested part of the file in the local cache (906).
If there is a stable copy in the cache then the piece of file is modified in the local cache (934 of Figure 9B). If however, there is no stable copy in the local cache (906), then it is determined if the piece of the requested file can be successfully read from the shared storage into the operating system local cache (930). If the server could not read the file into the local cash, then an error is returned to the operating system or the application (932). If, however, the file was successfully read into the local cache then the piece of file is modified in the local cache (934).
It is then determined whether a request for a lock for this particular file has been received (936). If a request for a lock for this file has been received, and the file has been modified, then the modified data is written to the shared storage (940). The server then gives up the exclusive lock on this file (938).
If no request has been received for a lock for this file (936), then it is determined whether the operating system is cleaning out the local cache (942).
If the cache is being cleaned, then the modified data is written to the shared storage (940) and any locks are maintained unless there is an outstanding request from another server. Otherwise, the modified file is kept in the local cache (944). By keeping the modified file in the local cache until it is needed by another server, access to the file is made faster for the server that maintains the file in its local cash.
Figs. l0A-lOB are flow diagrams of a method according to an embodiment of the present invention for adding a node to a cluster of servers sharing storage such as a disk.
In this example, it is determined whether there is an administrator (ADM) in the cluster (1400). The cluster includes the set of servers that cooperate to share a shared resource such as the shared storage. One of the servers in the cluster act as an administrator to manage the cluster. If there is no administrator in the cluster, then it is determined whether this server can try to become the administrator (1408).
If this server can try to become the administrator then the server reads the device information from the database and it is now the new administrator (1410).
If there is an administrator in the cluster (1400), or if this server can not become the new administrator (1408), then it asks the existing administrator to be imported into the cluster (1402). An example of how this server can not become the administrator (1408) is if another server became the administrator during the time this S server established that there was no administrator and then tried to become the administrator.
It is then determined whether it is permissible for this server to be imported into the cluster (1404). If it is not permissible then the process of adding this server to the cluster has failed (1412). Examples of reasons why adding the server would fail include this server not being healthy or having a storage area network generation number mismatch with the generation number used by the administrator.
If this server can be imported (1404), then it receives device names from the administrator (1406). Examples of device names include names of shared storage.
The administrator grants physical storage area network access to this server (1410 of Fig. 10B). The administrator then commands the physical hardware to allow this server storage area network (SAN) access (1412). This server now has access to the SAN (1414).
Figs. 11A-11C are flow diagrams of a method according to the present invention for handling a server failure. In this example, it is determined that a server or communication with a server has failed (1700). It is then determined whether there is still an administrator (1702). For example, the server that failed may have been the administrator. If there is still an administrator then the failed server is physically isolated (1708). An example of physically isolating the failed server is to disable the port associated with the failed server.
The storage area network generation number is then updated and stored to the database (1710). Thereafter, normal operation continues (1712).
If there is no longer an administrator (1702), then a server is picked to try to become the new administrator (1704). There are several ways to select a server to try to become the new administrator. One example is a random selection of one of the servers. The elected server is then told to try to become the new administrator (1706).
One example of how the server is selected and told to become the new administrator is through the use of a group coordinator.
In one embodiment, the group coordinator is elected during the formation of a process communication group using an algorithm that can uniquely identify the coordinator of the group with no communication with any server or node except that required to agree on the membership of the group. For example, the server with the lowest numbered Internet Protocol (IP) address of the members can be selected.
The coorindator can then make global decisions for the group of servers, such as the selection of a possible administrator. The server selected as administrator is preferably one which has a high probability of success of actually becoming the administrator. The group coordinator attempts to place the administrator on a node which might be able to connect the SAN hardware and has not recently failed in an attempt to become the SAN administrator.
The selected server then attempts to acquire the storage area network locks (1720). If it cannot acquire the SAN locks, then it has failed to become the administrator (1724). If it succeeds in acquiring the SAN locks (1720), then it attempts to read the SAN generation number from the membership database (1722).
The database can be maintained in one of the membership partitions on a shared storage and can be co-resident with the SAN locks If the server fails to read the SAN generation number from the database (1722), then it drops the SAN locks (1726), and it has failed to become the administrator (1724). Once the server has failed to become the administrator (1724), the group coordinator tells a server to try to become the new administrator (1706 Fig.
5A).
If the server can read the SAN generation number from the database, then it increments the SAN generation number and stores it back into the database (1728). It also informs the group coordinator that this server is now the administrator (1730).
The group coordinator receives the administrator update (1732). It is then determined if it is permissible for this server to be the new administrator (1750). If it is not okay, then a message to undo the administrator status is sent to the current server trying to become the administrator (1752). Thereafter, the group coordinator tells another server to try to become the new administrator (1706 of Fig. 11A).
If it is okay for this server to be the new administrator, the administrator is told to commit (1754), and the administrator is committed (1756). The coordinator then informs the other servers in the cluster about the new administrator (1758).
Fig. 12 is flow diagram of a method according to an embodiment of the present invention for adding or removing shared storage. In this example, a request is sent from a server to the administrator to add or remove a shared storage (1600), such as a disk. The disk is then added or removed to the naming database (1602).
The naming database can be maintained on the shared storage accessible by all servers and known by all servers in advance when they join the cluster. Servers with no knowledge of the location of a naming database are preferably not eligible to become a SAN administrator.
The SAN generation number is then incremented (1604). Each server in the cluster is then informed of the SAN generation number and the addition or deletion of the new disk (1606). When all the servers in the cluster acknowledge, the new SAN
generation number is written to the database (1608). The requesting server is then notified that the addition/removal of the disk is complete (1610).
Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. It should be noted that there are many alternative ways of implementing both the process and apparatus of the present invention. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
WHAT IS CLAIMED IS:
Figure 1 is a block diagram of an example of a typical server system.
Figure 2 is a block diagram of another example of a typical server system.
Figure 3 is a block diagram of a system for a multi-node environment according to an embodiment of the present invention.
Figure 4 is another block diagram of a system according to an embodiment of the present invention.
Figure 5 is a block diagram of the software components of server 300 according to an embodiment of the present invention.
Figure 6 is a flow diagram of a method for a multi-node environment according to an embodiment of the present invention.
Figures 7A-7C are other flow diagrams of a method according to an embodiment of the present invention for a mufti-node environment.
Figure 8 is a flow diagram of a method according to an embodiment of the present invention for reading a file.
Figures 9A-9B are flow diagrams of a method according to an embodiment of the present invention for writing to a file.
Figs. l0A-lOB are flow diagrams of a method according to an embodiment of the present invention for adding a node to a cluster of servers sharing storage such as a disk.
Figs. 11A-11C are flow diagrams of a method according to the present invention for handling a server failure.
Fig. 12 is flow diagram of a method according to an embodiment of the present invention for adding or removing shared storage.
DETAILED DESCRIPTION
It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, or a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or electronic communication links.
It should be noted that the order of the steps of disclosed processes may be altered within the scope of the invention.
A detailed description of one or more preferred embodiments of the invention are provided below along with accompanying figures that illustrate by way of example the principles of the invention. While the invention is described in connection with such embodiments, it should be understood that the invention is not limited to any embodiment. On the contrary, the scope of the invention is limited only by the appended claims and the invention encompasses numerous alternatives, modifications and equivalents. For the purpose of example, numerous specific details are set forth in the following description in order to provide a thorough understanding of the present invention. The present invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the present invention is not unnecessarily obscured.
Figure 1 is a block diagram of a conventional server system. In this example, computers 100A-100D are networked together by network 102. Each of the computers 100A-100D have their own locally connected storage 104A-104D.
Business computing typically requires regular sharing of information but this type of system can be slow to access shared information or it might require very expensive customized hardware.
Figure 2 is a block diagram of another conventional system. In this example, the various servers 200A-200D all use a single operating system across all of the servers 200A-200D. This type of system can be very expensive since it typically requires customized hardware. Another problem with this type of system is that if there is a fault in a single computer 200A-200D, the entire multi-server system would likely need to shut down. Accordingly, each computer needs to be more reliable than the standard computer, thus further raising costs.
What is needed is a system and method for a multi-node environment that does not require customized hardware components. The present invention addresses such a need.
Figure 3 is a block diagram of a system for a mufti-node environment according to an embodiment of the present invention. In this example, servers 300D are coupled via network interconnects 302. The network interconnects 302 can represent any network infrastructure such as an Ethernet, InfiniBand network or Fibre Channel network capable of host-to-host communication. The servers 300A-300D
are also coupled to the data storage interconnect 304, which in turn is coupled to shared storage 306A-306D. The data storage interconnect 304 can be any S interconnect that can allow access to the shared storage 306A-306D by servers 300A-300D. An example of the data storage interconnect 304 is a Fibre Channel switch, such as a Brocade 3200 Fibre Channel switch. Alternately, the data storage network might be an iSCSI or other IP storage network, InfiniBand network, or another kind of host-to-storage network. In addition, the network interconnects 302 and the data storage interconnect 304 may be embodied in a single interconnect.
Servers 300A-300D can be any computer, preferable an off the-shelf computer or server or any equivalent thereof. Servers 300A-300D can each run operating systems that are independent of each other. Accordingly, each server 300A-300D
can, but does not need to, run a different operating system. For example, server 300A
may run Microsoft windows, while server 300B runs Linux, and server 300C can simultaneously run a Unix operating system. An advantage of running independent operating systems for the servers 300A-300D is that the entire mufti-node system can be dynamic. For example, one of the servers 300A-300D can fail while the other servers 300A-300D continue to operate.
The shared storage 306A-306D can be any storage device, such as hard drive disks, compact disks, tape, and random access memory. A filesystem is a logical entity built on the shared storage. Although the shared storage 306A-306D is typically considered a physical device while the filesystem is typically considered a logical structure overlaid on part of the storage, the filesystem is sometimes referred to herein as shared storage for simplicity. For example, when it is stated that shared storage fails, it can be a failure of a part of a filesystem, one or more filesystems, or the physical storage device on which the filesystem is overlaid. Accordingly, shared storage, as used herein, can mean the physical storage device, a portion of a filesystem, a filesystem, filesystems, or any combination thereof.
Figure 4 is another block diagram of a system according to an embodiment of the present invention. In this example, the system preferably has no single point of failure. Accordingly, servers 300A' - 300D' are coupled with multiple network interconnects 302A-302D. The servers 300A'-300D' are also shown to be coupled with multiple storage interconnects 304A-304B. The storage interconnects 304A-304B are each coupled to a plurality of data storage 306A'-306D'.
In this manner, there is redundancy in the system such that if any of the components or connections fail, the entire system can continue to operate.
In the example shown in Figure 4, as well as the example shown in Figure 3, the number of servers 300A'-300D', the number of storage interconnects 304A-304B, and the number of data storage 306A'-306D' can be as many as the customer requires and is not physically limited by the system. Likewise, the operating systems used by servers 300A'-300D' can also be as many independent operating systems as the customer requires.
Figure 5 is a block diagram of the software components of server 300 according to an embodiment of the present invention. In an embodiment of the present invention, each server 300A-300D of Fig 3 includes these software components.
In this embodiment, the following components are shown:
The Distributed Lock Manager (DLM) 500 manages matrix-wide locks for the filesystem image 306a-306d, including the management of lock state during crash recovery. The Matrix Filesystem 504 uses DLM S00-managed locks to implement matrix-wide mutual exclusion and matrix-wide filesystem 306a-306d metadata and data cache consistency. The DLM 500 is a distributed symmetric lock manager.
Preferably, there is an instance of the DLM 500 resident on every server in the matrix.
Every instance is a peer to every other instance; there is no master/slave relationship among the instances.
The lock-caching layer ("LCL") 502 is a component internal to the operating system kernel that interfaces between the Matrix Filesystem 504 and the application-level DLM 500. The purposes of the LCL 502 include the following:
1. It hides the details of the DLM 500 from kernel-resident clients that need to obtain distributed locks.
2. It caches DLM 500 locks (that is, it may hold on to DLM 500 locks after clients have released all references to them), sometimes obviating the need for kernel components to communicate with an application-level process (the DLM S00) to obtain matrix-wide locks.
3. It provides the ability to obtain locks in both process and server scopes (where a process lock ensures that the corresponding DLM (500) lock is held, and also excludes local processes attempting to obtain the lock in conflicting modes, whereas a server lock only ensures that the DLM (500) lock is held, without excluding other local processes).
4. It allows clients to define callouts for different types of locks when certain events related to locks occur, particularly the acquisition and surrender of level locks. This ability is a requirement for cache-coherency, which depends on callouts to flush modified cached data to permanent storage when corresponding DLM 500 write locks are downgraded or released, and to purge cached data when DLM 500 read locks are released.
The LCL 502 is the only kernel component that makes lock requests from the user-level DLM 500. It partitions DLM 500 locks among kernel clients, so that a single DLM 500 lock has at most one kernel client on each node, namely, the LCL
502 itself. Each DLM 500 lock is the product of an LCL 502 request, which was induced by a client's request of an LCL 502 lock, and each LCL 502 lock is backed by a DLM 500 lock.
The Matrix Filesystem 504 is the shared filesystem component of The Matrix Server. The Matrix Filesystem 504 allows multiple servers to simultaneously mount, in read/write mode, filesystems living on physically shared storage devices 306a-306d. The Matrix Filesystem 504 is a distributed symmetric matrixed filesystem;
there is no single server that filesystem activity must pass through to perform filesystem activities. The Matrix Filesystem 504 provides normal local filesystem semantics and interfaces for clients of the filesystem.
SAN (Storage Area Network) Membership Service 506 provides the group membership services infrastructure for the Matrix Filesystem 504, including managing filesystem membership, health monitoring, coordinating mounts and unmounts of shared filesystems 306a-306d, and coordinating crash recovery.
Matrix Membership Service 508 provides the Local, matrix-style matrix membership support, including virtual host management, service monitoring, notification services, data replication, etc. The Matrix Filesystem 504 does not interface directly with the MMS 508, but the Matrix Filesystem 504 does interface with the SAN Membership Service 506, which interfaces with the MMS 508 in order to provide the filesystem 504 with the matrix group services infrastructure.
The Shared Disk Monitor Probe S 10 maintains and monitors the membership of the various shared storage devices in the matrix. It acquires and maintains leases on the various shared storage devices in the matrix as a protection against rogue server "split-brain" conditions. It communicates with the SMS 506 to coordinate recovery activities on occurrence of a device membership transition.
Filesystem monitors 512 are used by the SAN Membership Service 508 to 1 S initiate Matrix Filesystem 504 mounts and unmounts, according to the matrix configuration put in place by the Matrix Server user interface.
The Service Monitor 514 tracks the state (health & availability) of various services on each server in the matrix so that the matrix server may take automatic remedial action when the state of any monitored service transitions. Services monitored include HTTP, FTP, Telnet, SMTP, etc. The remedial actions include service restart on the same server or service fail-over and restart on another server.
The Device Monitor 516 tracks the state (health & availability) of various storage-related devices in the matrix so that the matrix server may take automatic remedial action when the state of any monitored device transitions. Devices monitored may include data storage devices 306a-306d (such as storage device drives, solid state storage devices, ram storage devices, JOBDs, RAID arrays, etc.)and storage network devices 304' (such as FibreChannel Switches, Infiniband Switches, iSCSI switches, etc.). The remedial actions include initiation of Matrix Filesystem 504 recovery, storage network path failover, and device reset.
The Application Monitor 518 tracks the state (health & availability) of various applications on each server in the matrix so that the matrix server may take automatic remedial action when the state of any monitored application transitions.
Applications monitored may include databases, mail routers, CRM apps, etc. The remedial actions include application restart on the same server or application fail-over and restart on another server.
The Notifier Agent 520 tracks events associated with specified objects in the matrix and executes supplied scripts of commands on occurrence of any tracked event.
The Replicator Agent 522 monitors the content of any filesystem subtree and periodically replicates any data which has not yet been replicated from a source tree to a destination tree. Replication is preferably used for subtrees not placed in shared storage The Matrix Communication Service 524 provides the network communication infrastructure for the DLM 500, Matrix Membership Service 508, and SAN
Membership Service 506. The Matrix Filesystem 504 does not use the MCS 524 directly, but it does use it indirectly through these other components.
The Storage Control Layber (SCL) 526 provides matrix-wide device identification, used to identify the Matrix Filesystems 504 at mount time. The SCL
526 also manages storage fabric configuration and low level I/O device fencing of rogue servers from the shared storage devices 306a-306d containing the Matrix Filesystems 504. It also provides the ability for a server in the matrix to voluntarily intercede during normal device operations to fence itself when communication with rest of the matrix has been lost.
The Storage Control Layer 526 is the Matrix Server module responsible for managing shared storage devices 306a-306d. Management in this context consists of two primary functions. The first is to enforce I/O fencing at the hardware SAN
level by enabling/disabling host access to the set of shared storage devices 306a-306d. And the second is to generate global (matrix-wide) unique device names (or "labels") for all matrix storage devices 306a-306d and ensure that all hosts in the matrix have access to those global device names. The SCL module also includes utilities and library routines needed to provide device information to the UI.
The Pseudo Storage Driver 528 is a layered driver that "hides" a target storage device 306a-306d so that all references to the underlying target device must pass through the PSD layered driver. Thus, the PSD provides the ability to "fence"
a device, blocking all I/O from the host server to the underlying target device until it is unfenced again. The PSD also provides an application-level interface to lock a storage partition across the matrix. It also has the ability to provide common matrix-wide 'handles', or paths, to devices such that all servers accessing shared storage in the Matrix Server can use the same path to access a given shared device.
Figure 6 is a flow diagram of a method for a mufti-node environment according to an embodiment of the present invention. In this example, a first operating system is provided (600), and a second operating system is also provided (602). The second operating system is preferably independent of the first operating system. They can be the same operating system, such as both operating systems being Linux, or different operating systems, such as different versions of Windows or a Unix and a Linux, but each running a separate OS rather than a combined OS. A
storage is also provided (604). An interconnect coupling the first operating system with the storage and coupling the second operating system with the storage is then provided (606). The storage is then directly accessed by the first operating system (608), and the storage is also directly accessed by the second operating system (610).
Figures 7A-7C are other flow diagrams of a method according to an embodiment of the present invention for a mufti-node environment. The following terms are used herein.
Group membership is used herein to refer to a set of cooperating processes (programs) that form a communication group. For example, a group membership can be formed between servers. SAN membership is used herein to refer to a storage area membership. SAN membership refers to the group of servers that are communicating which are allowed to read/write the Storage area network resources such as disks and switches. Shared storage group membership is used herein to refer to the set of servers actively using a single filesystem located on the SAN. An example is the set of servers that have "mounted" the filesystem to make it accessible to user programs.
Cluster membership refers to the set of servers forming a single cluster which actively share filesystems and network resources as a logical whole.
In this example, a membership of a cluster of servers is determined (700). As previously mentioned, servers are used as an example, however, any node, computer or processor can be used. A cluster, as used herein, can be any two or more servers, computers, processors, or any combination thereof, that is associated with a membership.
It is then determined whether the membership of the cluster has changed (702). At time zero, there will be no change to the membership since there is no history of the membership. At some later time, the membership may be changed, for example, if a server is taken off line, or a new server is added. In addition to the change in the numbers of servers that are utilized, determining whether the membership has changed can also accommodate a situation where a server has failed, a server has been added, or has been taken out of the cluster.
If the cluster membership has not changed (702), then each server sends messages to the other servers and monitors the messages to see if the other servers are active (704). It is then determined whether all servers respond (706). If all the other servers in the cluster respond to the messages, then the storage area network (Sale membership service (SMS) is informed of the active members of the cluster (708). If, however, all servers in the cluster do not respond (706), then message activity is blocked (716). Blocking message activity avoids stale messages between membership changes. Once message activity is blocked, the membership of the cluster is again determined (700).
If the cluster membership has changed (702), then new locks are no longer granted (710). It is then determined whether there is an administrator (ADM) in this cluster (712). If there is no administrator in this cluster, then one of the members of the cluster is elected as administrator (714). In the example shown in Figure 5, the SAN membership service (SMS) 506 can be used as an administrator.
The administrator verifies that the other servers in this cluster are part of this storage area membership (720) of Figure 7B. Step 720 accommodates both when all of the servers are part of the cluster, or when there are servers outside the cluster.
There may be separate clusters operating in the storage area network either by design or by accident. In either case servers misconfigured software may result in a server attaching to the shared storage and attempting to access it, without knowledge of the valid cluster. If nodes are outside the cluster then the administrator excludes (fences) those servers to prevent corruption of data on shared storage.
Servers that successfully gain membership to the network cluster are then allowed access to the shared storage and are then part of the SAN membership.
All cluster non-members are then excluded and all cluster members are allowed into the shared storage group (722). Accordingly, servers outside the cluster membership are excluded from accessing the discs 306A-306D of Figure 5.
Shared storage is then set to equal one (724). It is determined by the ADM
whether recovery is necessary for a member of this shared storage group (726).
If no recovery is required for a member of this shared storage group, then it is determined whether this is the last shared storage (732). If it is not the last shared storage, then the shared storage is then set to shared storage plus one (730), and the next shared storage is evaluated by having the administrator determine whether recovery is necessary for a member of that shared storage (726). If, however, this shared storage was the last shared storage, then the granting of new locks are resumed for shared storages that are accessible (734).
If the administrator determines that recovery is necessary for a member of this shared storage (726), then it is determined which members) of this shared storage needs recovery (728). For example, the SAN membership service (SMS) of each server can tell their distributed lock manager (DLM) whether that server needs recovery.
The server that needs recovery is analyzed. In this example, the DLM for the member that needs recovery is set equal to one (750). Locks which can trigger journal recovery in the shared storage by the matrix file system (MFS) and the lock caching layer (LCL) are then recovered (752).
It is then determined whether recovery was successful (754). If the recovery was successful then it is determined whether this server was the last server that required recovery (762). If this server was not the last server that required recovery, then the next server that needed recovery is analyzed by setting DLM equals DLM
plus one (764). The locks are then recovered for this next server (752). If, however, this server was the last server that needed recovery (762), then it is determined whether this shared storage is the last shared storage that needed to be analyzed (732 of Figure 7B).
If recovery of the locks was not successful (754 of Figure 7C), then that failed shared storage is marked as error (756), and an alert is sent to a human system operator (758).
The failed shared storage is then isolated and access to that shared storage is prevented (760). It is then determined whether this shared storage is the last shared storage that needs to be analyzed (732 of Figure 7B).
Figure 8 is a flow diagram of a method according to an embodiment of the present invention for reading a file from shared storage. The operating system of a server in a cluster requests to read a part of a file (800).
A shared lock request is provided (802). For example, the matrix file system (MFS) 504 of Figure S asks the lock caching layer (LCL) 502 of Figure S for a shared lock. It is then determined whether the request is granted (804). If the request for the 1 S shared lock is not granted, then it is determined whether there is a shared storage failure (806).
If there is a share storage failure, then the shared storage is isolated if necessary (808). If, however, there is no shared storage failure then the MFS
asks the LCL for a shared lock (802). It is then determined whether the shared lock request is granted (804).
If the shared lock request is granted (804), then it is determined whether the server is able to read a piece of the requested file from the shared storage into the operating system buffer cache (810). If the server is able to read the file into the OS
buffer cache then the process is complete. If, however, the server is unable to read into the OS buffer cache then the shared lock is dropped and an error is returned to the operating system or application (812). An example of when such an error might occur is if there is a disk failure or a connection failure.
Figures 9A-9B are flow diagrams of a method according to an embodiment of the present invention of writing to a file in a shared storage.
In this example, the operating system of a server requests to write to a part of a file (900). The matrix file system (MFS) 504 of Figure 5 asks the lock caching layer (LCL) for an exclusive lock (902). It is then determined whether the request for an exclusive lock is granted (904). If the request for the exclusive lock is not granted then it is determined whether there is a shared storage failure (920). If there is a shared storage failure, then that shared storage is isolated, if necessary (922). If, however, there is no shared storage failure (920), then there is another request for the 1 S exclusive lock (902).
If the request for an exclusive lock is granted (904), then it is determined whether there is a stable copy of the requested part of the file in the local cache (906).
If there is a stable copy in the cache then the piece of file is modified in the local cache (934 of Figure 9B). If however, there is no stable copy in the local cache (906), then it is determined if the piece of the requested file can be successfully read from the shared storage into the operating system local cache (930). If the server could not read the file into the local cash, then an error is returned to the operating system or the application (932). If, however, the file was successfully read into the local cache then the piece of file is modified in the local cache (934).
It is then determined whether a request for a lock for this particular file has been received (936). If a request for a lock for this file has been received, and the file has been modified, then the modified data is written to the shared storage (940). The server then gives up the exclusive lock on this file (938).
If no request has been received for a lock for this file (936), then it is determined whether the operating system is cleaning out the local cache (942).
If the cache is being cleaned, then the modified data is written to the shared storage (940) and any locks are maintained unless there is an outstanding request from another server. Otherwise, the modified file is kept in the local cache (944). By keeping the modified file in the local cache until it is needed by another server, access to the file is made faster for the server that maintains the file in its local cash.
Figs. l0A-lOB are flow diagrams of a method according to an embodiment of the present invention for adding a node to a cluster of servers sharing storage such as a disk.
In this example, it is determined whether there is an administrator (ADM) in the cluster (1400). The cluster includes the set of servers that cooperate to share a shared resource such as the shared storage. One of the servers in the cluster act as an administrator to manage the cluster. If there is no administrator in the cluster, then it is determined whether this server can try to become the administrator (1408).
If this server can try to become the administrator then the server reads the device information from the database and it is now the new administrator (1410).
If there is an administrator in the cluster (1400), or if this server can not become the new administrator (1408), then it asks the existing administrator to be imported into the cluster (1402). An example of how this server can not become the administrator (1408) is if another server became the administrator during the time this S server established that there was no administrator and then tried to become the administrator.
It is then determined whether it is permissible for this server to be imported into the cluster (1404). If it is not permissible then the process of adding this server to the cluster has failed (1412). Examples of reasons why adding the server would fail include this server not being healthy or having a storage area network generation number mismatch with the generation number used by the administrator.
If this server can be imported (1404), then it receives device names from the administrator (1406). Examples of device names include names of shared storage.
The administrator grants physical storage area network access to this server (1410 of Fig. 10B). The administrator then commands the physical hardware to allow this server storage area network (SAN) access (1412). This server now has access to the SAN (1414).
Figs. 11A-11C are flow diagrams of a method according to the present invention for handling a server failure. In this example, it is determined that a server or communication with a server has failed (1700). It is then determined whether there is still an administrator (1702). For example, the server that failed may have been the administrator. If there is still an administrator then the failed server is physically isolated (1708). An example of physically isolating the failed server is to disable the port associated with the failed server.
The storage area network generation number is then updated and stored to the database (1710). Thereafter, normal operation continues (1712).
If there is no longer an administrator (1702), then a server is picked to try to become the new administrator (1704). There are several ways to select a server to try to become the new administrator. One example is a random selection of one of the servers. The elected server is then told to try to become the new administrator (1706).
One example of how the server is selected and told to become the new administrator is through the use of a group coordinator.
In one embodiment, the group coordinator is elected during the formation of a process communication group using an algorithm that can uniquely identify the coordinator of the group with no communication with any server or node except that required to agree on the membership of the group. For example, the server with the lowest numbered Internet Protocol (IP) address of the members can be selected.
The coorindator can then make global decisions for the group of servers, such as the selection of a possible administrator. The server selected as administrator is preferably one which has a high probability of success of actually becoming the administrator. The group coordinator attempts to place the administrator on a node which might be able to connect the SAN hardware and has not recently failed in an attempt to become the SAN administrator.
The selected server then attempts to acquire the storage area network locks (1720). If it cannot acquire the SAN locks, then it has failed to become the administrator (1724). If it succeeds in acquiring the SAN locks (1720), then it attempts to read the SAN generation number from the membership database (1722).
The database can be maintained in one of the membership partitions on a shared storage and can be co-resident with the SAN locks If the server fails to read the SAN generation number from the database (1722), then it drops the SAN locks (1726), and it has failed to become the administrator (1724). Once the server has failed to become the administrator (1724), the group coordinator tells a server to try to become the new administrator (1706 Fig.
5A).
If the server can read the SAN generation number from the database, then it increments the SAN generation number and stores it back into the database (1728). It also informs the group coordinator that this server is now the administrator (1730).
The group coordinator receives the administrator update (1732). It is then determined if it is permissible for this server to be the new administrator (1750). If it is not okay, then a message to undo the administrator status is sent to the current server trying to become the administrator (1752). Thereafter, the group coordinator tells another server to try to become the new administrator (1706 of Fig. 11A).
If it is okay for this server to be the new administrator, the administrator is told to commit (1754), and the administrator is committed (1756). The coordinator then informs the other servers in the cluster about the new administrator (1758).
Fig. 12 is flow diagram of a method according to an embodiment of the present invention for adding or removing shared storage. In this example, a request is sent from a server to the administrator to add or remove a shared storage (1600), such as a disk. The disk is then added or removed to the naming database (1602).
The naming database can be maintained on the shared storage accessible by all servers and known by all servers in advance when they join the cluster. Servers with no knowledge of the location of a naming database are preferably not eligible to become a SAN administrator.
The SAN generation number is then incremented (1604). Each server in the cluster is then informed of the SAN generation number and the addition or deletion of the new disk (1606). When all the servers in the cluster acknowledge, the new SAN
generation number is written to the database (1608). The requesting server is then notified that the addition/removal of the disk is complete (1610).
Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. It should be noted that there are many alternative ways of implementing both the process and apparatus of the present invention. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.
WHAT IS CLAIMED IS:
Claims (19)
1. A system for a multi-node environment comprising:
a first node associated with a first operating system;
a second node associated with a second operating system, wherein the second operating system is independent of the first operating system;
a storage; and an interconnect coupling the first node with the storage and coupling the second node with the storage; and wherein the first node directly accesses the storage and the second node directly accesses the storage.
a first node associated with a first operating system;
a second node associated with a second operating system, wherein the second operating system is independent of the first operating system;
a storage; and an interconnect coupling the first node with the storage and coupling the second node with the storage; and wherein the first node directly accesses the storage and the second node directly accesses the storage.
2. The system of claim 1, wherein the first operating system is a different operating system than the second operating system.
3. The system of claim 1, wherein the storage is a coherent shared file storage.
4. The system of claim 1, wherein the interconnect is a shared disk interconnect.
5. The system of claim 1, further comprising a third node that is dynamically added such that the third node directly accesses the storage.
6. The system of claim 1, wherein the second node is dynamically decoupled wherein the second node no longer directly accesses the storage.
7. The system of claim 1, further comprising a second storage that is dynamically added such that the second storage is directly accessible by the first and second nodes.
8. The system of claim 1, further comprising a second storage wherein the second storage is directly accessible by the first and second nodes, and wherein the second storage is dynamically decoupled from the first and second nodes such that the second storage is no longer directly accessible by the first and second nodes.
9. A system for a multi-node environment comprising:
a first node associated with a membership group, wherein the first node is associated with a first operating system;
a second node associated with the membership group, wherein the second node is associated with a second operating system, and wherein the second operating system is independent of the first operating system; and storage coupled with the first and second nodes, the storage being directly accessible by the first node and the second node; and and wherein the first node is configured to determine whether the membership group has changed.
a first node associated with a membership group, wherein the first node is associated with a first operating system;
a second node associated with the membership group, wherein the second node is associated with a second operating system, and wherein the second operating system is independent of the first operating system; and storage coupled with the first and second nodes, the storage being directly accessible by the first node and the second node; and and wherein the first node is configured to determine whether the membership group has changed.
10. The system of claim 9, wherein the first node dynamically adjusts to the change in the membership group.
11. A method for managing a multi-node environment comprising:
associating a first node with a membership group, wherein the first node is associated with a first operating system, and wherein the first node directly accesses a storage;
associating a second node with the membership group, wherein the second node is associated with a second operating system, and wherein the second operating system is independent of the first operating system, and wherein the second node directly accesses the storage;
determining whether the membership group has changed; and dynamically adjusting to the change in the membership group.
associating a first node with a membership group, wherein the first node is associated with a first operating system, and wherein the first node directly accesses a storage;
associating a second node with the membership group, wherein the second node is associated with a second operating system, and wherein the second operating system is independent of the first operating system, and wherein the second node directly accesses the storage;
determining whether the membership group has changed; and dynamically adjusting to the change in the membership group.
12. The method of claim 11, wherein the first operating system is a different operating system than the second operating system.
13. The method of claim 11, wherein the membership group changed due to adding a third node to the membership group.
14. The method of claim 11, wherein a third node is associated with the membership group and wherein the membership group changed due to subtracting the third node from the membership group.
15. The method of claim 11, further comprising determining whether recovery needs to be performed if the membership group has changed.
16. The method of claim 11, further comprising recovering locks if the membership group has changed and if a recovery is necessary.
17. The method of claim 11, further comprising stopping granting of new locks if the membership group has changed.
18. A system for a first node configured to be associated with a multi-node environment comprising:
27~
a processor configured to be associated with a first operating system, wherein the first operating system is independent of a second operating system of a second node; the processor also configured to be capable of determining whether a membership group has changed, wherein the membership group is associated with the first node and the second node; and also configured to be capable of dynamically adjusting to the change in the membership group; and a storage coupled to the processor, wherein the storage is configured to be directly accessed by the processor and also configured to be directly accessed by the second node.
27~
a processor configured to be associated with a first operating system, wherein the first operating system is independent of a second operating system of a second node; the processor also configured to be capable of determining whether a membership group has changed, wherein the membership group is associated with the first node and the second node; and also configured to be capable of dynamically adjusting to the change in the membership group; and a storage coupled to the processor, wherein the storage is configured to be directly accessed by the processor and also configured to be directly accessed by the second node.
19. A computer program product for a first node configured to be associated with a multi-node environment, the computer program product being embodied in a computer readable medium and comprising computer instructions for:
determining whether a membership group has changed, wherein the membership group is associated with the first node and a second node, and wherein the first node is associated with a first operating system and the second node is associated with a second operating system, and wherein the first operating system is independent of the second operating system;
dynamically adjusting to the change in the membership group; and directly accessing a storage, wherein the storage is also directly accessible by the second node.
determining whether a membership group has changed, wherein the membership group is associated with the first node and a second node, and wherein the first node is associated with a first operating system and the second node is associated with a second operating system, and wherein the first operating system is independent of the second operating system;
dynamically adjusting to the change in the membership group; and directly accessing a storage, wherein the storage is also directly accessible by the second node.
Applications Claiming Priority (17)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US32419601P | 2001-09-21 | 2001-09-21 | |
US32422601P | 2001-09-21 | 2001-09-21 | |
US32424301P | 2001-09-21 | 2001-09-21 | |
US32424201P | 2001-09-21 | 2001-09-21 | |
US32419501P | 2001-09-21 | 2001-09-21 | |
US32422401P | 2001-09-21 | 2001-09-21 | |
US60/324,243 | 2001-09-21 | ||
US60/324,195 | 2001-09-21 | ||
US60/324,226 | 2001-09-21 | ||
US60/324,224 | 2001-09-21 | ||
US60/324,196 | 2001-09-21 | ||
US60/324,242 | 2001-09-21 | ||
US32478701P | 2001-09-24 | 2001-09-24 | |
US60/324,787 | 2001-09-24 | ||
US32719101P | 2001-10-01 | 2001-10-01 | |
US60/327,191 | 2001-10-01 | ||
PCT/US2002/029859 WO2003027903A1 (en) | 2001-09-21 | 2002-09-20 | A system and method for a multi-node environment with shared storage |
Publications (1)
Publication Number | Publication Date |
---|---|
CA2461015A1 true CA2461015A1 (en) | 2003-04-03 |
Family
ID=27575390
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA2460833A Expired - Lifetime CA2460833C (en) | 2001-09-21 | 2002-09-20 | System and method for implementing journaling in a multi-node environment |
CA002461015A Abandoned CA2461015A1 (en) | 2001-09-21 | 2002-09-20 | A system and method for a multi-node environment with shared storage |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA2460833A Expired - Lifetime CA2460833C (en) | 2001-09-21 | 2002-09-20 | System and method for implementing journaling in a multi-node environment |
Country Status (7)
Country | Link |
---|---|
US (8) | US20040202013A1 (en) |
EP (2) | EP1428149B1 (en) |
JP (2) | JP2005504369A (en) |
CN (2) | CN1302419C (en) |
AU (1) | AU2002341784A1 (en) |
CA (2) | CA2460833C (en) |
WO (5) | WO2003027853A1 (en) |
Families Citing this family (191)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7412462B2 (en) * | 2000-02-18 | 2008-08-12 | Burnside Acquisition, Llc | Data repository and method for promoting network storage of data |
US6890968B2 (en) * | 2001-05-16 | 2005-05-10 | Kerr Corporation | Prepolymerized filler in dental restorative composite |
US8010558B2 (en) | 2001-06-05 | 2011-08-30 | Silicon Graphics International | Relocation of metadata server with outstanding DMAPI requests |
US20040139125A1 (en) | 2001-06-05 | 2004-07-15 | Roger Strassburg | Snapshot copy of data volume during data access |
US7640582B2 (en) | 2003-04-16 | 2009-12-29 | Silicon Graphics International | Clustered filesystem for mix of trusted and untrusted nodes |
US7617292B2 (en) * | 2001-06-05 | 2009-11-10 | Silicon Graphics International | Multi-class heterogeneous clients in a clustered filesystem |
US7702791B2 (en) | 2001-07-16 | 2010-04-20 | Bea Systems, Inc. | Hardware load-balancing apparatus for session replication |
US7571215B2 (en) * | 2001-07-16 | 2009-08-04 | Bea Systems, Inc. | Data replication protocol |
US7409420B2 (en) * | 2001-07-16 | 2008-08-05 | Bea Systems, Inc. | Method and apparatus for session replication and failover |
US7113980B2 (en) | 2001-09-06 | 2006-09-26 | Bea Systems, Inc. | Exactly once JMS communication |
US6826601B2 (en) * | 2001-09-06 | 2004-11-30 | Bea Systems, Inc. | Exactly one cache framework |
WO2003027853A1 (en) * | 2001-09-21 | 2003-04-03 | Polyserve, Inc. | A system and method for synchronisation for enforcing mutual exclusion among multiple negotiators |
US7392302B2 (en) | 2002-02-21 | 2008-06-24 | Bea Systems, Inc. | Systems and methods for automated service migration |
US7178050B2 (en) * | 2002-02-22 | 2007-02-13 | Bea Systems, Inc. | System for highly available transaction recovery for transaction processing systems |
US7096213B2 (en) * | 2002-04-08 | 2006-08-22 | Oracle International Corporation | Persistent key-value repository with a pluggable architecture to abstract physical storage |
WO2003092166A1 (en) * | 2002-04-25 | 2003-11-06 | Kashya Israel Ltd. | An apparatus for continuous compression of large volumes of data |
US20030220943A1 (en) * | 2002-05-23 | 2003-11-27 | International Business Machines Corporation | Recovery of a single metadata controller failure in a storage area network environment |
US7774325B2 (en) * | 2002-10-17 | 2010-08-10 | Intel Corporation | Distributed network attached storage system |
EP1568031A1 (en) * | 2002-11-07 | 2005-08-31 | Koninklijke Philips Electronics N.V. | Record carrier having a main file system area and a virtual file system area |
US7457906B2 (en) * | 2003-01-21 | 2008-11-25 | Nextio, Inc. | Method and apparatus for shared I/O in a load/store fabric |
US7613797B2 (en) * | 2003-03-19 | 2009-11-03 | Unisys Corporation | Remote discovery and system architecture |
GB0308923D0 (en) * | 2003-04-17 | 2003-05-28 | Ibm | Low-overhead storage cluster configuration locking |
US7409389B2 (en) | 2003-04-29 | 2008-08-05 | International Business Machines Corporation | Managing access to objects of a computing environment |
US7376744B2 (en) * | 2003-05-09 | 2008-05-20 | Oracle International Corporation | Using local locks for global synchronization in multi-node systems |
US7849401B2 (en) * | 2003-05-16 | 2010-12-07 | Justsystems Canada Inc. | Method and system for enabling collaborative authoring of hierarchical documents with locking |
CA2429375A1 (en) * | 2003-05-22 | 2004-11-22 | Cognos Incorporated | Model action logging |
WO2005008434A2 (en) * | 2003-07-11 | 2005-01-27 | Computer Associates Think, Inc. | A distributed locking method and system for networked device management |
US7356531B1 (en) * | 2003-07-25 | 2008-04-08 | Symantec Operating Corporation | Network file system record lock recovery in a highly available environment |
US7739541B1 (en) | 2003-07-25 | 2010-06-15 | Symantec Operating Corporation | System and method for resolving cluster partitions in out-of-band storage virtualization environments |
US8234517B2 (en) * | 2003-08-01 | 2012-07-31 | Oracle International Corporation | Parallel recovery by non-failed nodes |
US7584454B1 (en) * | 2003-09-10 | 2009-09-01 | Nextaxiom Technology, Inc. | Semantic-based transactional support and recovery for nested composite software services |
US20050091215A1 (en) * | 2003-09-29 | 2005-04-28 | Chandra Tushar D. | Technique for provisioning storage for servers in an on-demand environment |
US7581205B1 (en) | 2003-09-30 | 2009-08-25 | Nextaxiom Technology, Inc. | System and method of implementing a customizable software platform |
US7234073B1 (en) * | 2003-09-30 | 2007-06-19 | Emc Corporation | System and methods for failover management of manageable entity agents |
US8225282B1 (en) | 2003-11-25 | 2012-07-17 | Nextaxiom Technology, Inc. | Semantic-based, service-oriented system and method of developing, programming and managing software modules and software solutions |
US7155546B2 (en) * | 2003-12-18 | 2006-12-26 | Intel Corporation | Multiple physical interfaces in a slot of a storage enclosure to support different storage interconnect architectures |
US20050138154A1 (en) * | 2003-12-18 | 2005-06-23 | Intel Corporation | Enclosure management device |
US7376147B2 (en) * | 2003-12-18 | 2008-05-20 | Intel Corporation | Adaptor supporting different protocols |
US8560747B1 (en) | 2007-02-16 | 2013-10-15 | Vmware, Inc. | Associating heartbeat data with access to shared resources of a computer system |
US10776206B1 (en) * | 2004-02-06 | 2020-09-15 | Vmware, Inc. | Distributed transaction system |
US20110179082A1 (en) * | 2004-02-06 | 2011-07-21 | Vmware, Inc. | Managing concurrent file system accesses by multiple servers using locks |
US8543781B2 (en) | 2004-02-06 | 2013-09-24 | Vmware, Inc. | Hybrid locking using network and on-disk based schemes |
US8700585B2 (en) * | 2004-02-06 | 2014-04-15 | Vmware, Inc. | Optimistic locking method and system for committing transactions on a file system |
US7849098B1 (en) * | 2004-02-06 | 2010-12-07 | Vmware, Inc. | Providing multiple concurrent access to a file system |
JP4485256B2 (en) * | 2004-05-20 | 2010-06-16 | 株式会社日立製作所 | Storage area management method and management system |
US7962449B2 (en) * | 2004-06-25 | 2011-06-14 | Apple Inc. | Trusted index structure in a network environment |
US7730012B2 (en) | 2004-06-25 | 2010-06-01 | Apple Inc. | Methods and systems for managing data |
US8131674B2 (en) | 2004-06-25 | 2012-03-06 | Apple Inc. | Methods and systems for managing data |
US7386752B1 (en) * | 2004-06-30 | 2008-06-10 | Symantec Operating Corporation | Using asset dependencies to identify the recovery set and optionally automate and/or optimize the recovery |
US7769734B2 (en) * | 2004-07-26 | 2010-08-03 | International Business Machines Corporation | Managing long-lived resource locks in a multi-system mail infrastructure |
WO2006015536A1 (en) * | 2004-08-08 | 2006-02-16 | Huawei Technologies Co. Ltd. | A method for realizing notification log operation |
US20060041559A1 (en) * | 2004-08-17 | 2006-02-23 | International Business Machines Corporation | Innovation for managing virtual storage area networks |
US20060059269A1 (en) * | 2004-09-13 | 2006-03-16 | Chien Chen | Transparent recovery of switch device |
US7310711B2 (en) * | 2004-10-29 | 2007-12-18 | Hitachi Global Storage Technologies Netherlands B.V. | Hard disk drive with support for atomic transactions |
US7496701B2 (en) * | 2004-11-18 | 2009-02-24 | International Business Machines Corporation | Managing virtual server control of computer support systems with heartbeat message |
JP4462024B2 (en) | 2004-12-09 | 2010-05-12 | 株式会社日立製作所 | Failover method by disk takeover |
US8495266B2 (en) * | 2004-12-10 | 2013-07-23 | Hewlett-Packard Development Company, L.P. | Distributed lock |
US20060242453A1 (en) * | 2005-04-25 | 2006-10-26 | Dell Products L.P. | System and method for managing hung cluster nodes |
US7506204B2 (en) * | 2005-04-25 | 2009-03-17 | Microsoft Corporation | Dedicated connection to a database server for alternative failure recovery |
JP4648751B2 (en) * | 2005-05-02 | 2011-03-09 | 株式会社日立製作所 | Storage control system and storage control method |
US7631016B2 (en) * | 2005-05-04 | 2009-12-08 | Oracle International Corporation | Providing the latest version of a data item from an N-replica set |
US7356653B2 (en) * | 2005-06-03 | 2008-04-08 | International Business Machines Corporation | Reader-initiated shared memory synchronization |
US7437426B2 (en) * | 2005-09-27 | 2008-10-14 | Oracle International Corporation | Detecting and correcting node misconfiguration of information about the location of shared storage resources |
US7774565B2 (en) * | 2005-12-21 | 2010-08-10 | Emc Israel Development Center, Ltd. | Methods and apparatus for point in time data access and recovery |
US8060713B1 (en) | 2005-12-21 | 2011-11-15 | Emc (Benelux) B.V., S.A.R.L. | Consolidating snapshots in a continuous data protection system using journaling |
US7849361B2 (en) * | 2005-12-22 | 2010-12-07 | Emc Corporation | Methods and apparatus for multiple point in time data access |
US7836033B1 (en) * | 2006-01-24 | 2010-11-16 | Network Appliance, Inc. | Method and apparatus for parallel updates to global state in a multi-processor system |
US20070180287A1 (en) * | 2006-01-31 | 2007-08-02 | Dell Products L. P. | System and method for managing node resets in a cluster |
US7577867B2 (en) * | 2006-02-17 | 2009-08-18 | Emc Corporation | Cross tagging to data for consistent recovery |
US7552148B2 (en) * | 2006-02-28 | 2009-06-23 | Microsoft Corporation | Shutdown recovery |
US7899780B1 (en) * | 2006-03-30 | 2011-03-01 | Emc Corporation | Methods and apparatus for structured partitioning of management information |
CN100383750C (en) * | 2006-06-07 | 2008-04-23 | 中国科学院计算技术研究所 | High-reliable journal system realizing method facing to large-scale computing system |
US7734960B2 (en) * | 2006-08-14 | 2010-06-08 | Hewlett-Packard Development Company, L.P. | Method of managing nodes in computer cluster |
US7886034B1 (en) * | 2006-09-27 | 2011-02-08 | Symantec Corporation | Adaptive liveness management for robust and efficient peer-to-peer storage |
US20080082533A1 (en) * | 2006-09-28 | 2008-04-03 | Tak Fung Wang | Persistent locks/resources for concurrency control |
US7627687B2 (en) * | 2006-09-28 | 2009-12-01 | Emc Israel Development Center, Ltd. | Methods and apparatus for managing data flow in a continuous data replication system having journaling |
US7627612B2 (en) * | 2006-09-28 | 2009-12-01 | Emc Israel Development Center, Ltd. | Methods and apparatus for optimal journaling for continuous data replication |
US8024521B2 (en) * | 2007-03-13 | 2011-09-20 | Sony Computer Entertainment Inc. | Atomic operation on non-standard sized data using external cache |
US7778986B2 (en) * | 2007-08-29 | 2010-08-17 | International Business Machines Corporation | Securing transfer of ownership of a storage object from an unavailable owner node to another node |
US8055855B2 (en) * | 2007-10-05 | 2011-11-08 | International Business Machines Corporation | Varying access parameters for processes to access memory addresses in response to detecting a condition related to a pattern of processes access to memory addresses |
US7856536B2 (en) * | 2007-10-05 | 2010-12-21 | International Business Machines Corporation | Providing a process exclusive access to a page including a memory address to which a lock is granted to the process |
US7770064B2 (en) * | 2007-10-05 | 2010-08-03 | International Business Machines Corporation | Recovery of application faults in a mirrored application environment |
US7921272B2 (en) * | 2007-10-05 | 2011-04-05 | International Business Machines Corporation | Monitoring patterns of processes accessing addresses in a storage device to determine access parameters to apply |
US7860836B1 (en) | 2007-12-26 | 2010-12-28 | Emc (Benelux) B.V., S.A.R.L. | Method and apparatus to recover data in a continuous data protection environment using a journal |
US7840536B1 (en) | 2007-12-26 | 2010-11-23 | Emc (Benelux) B.V., S.A.R.L. | Methods and apparatus for dynamic journal expansion |
US7958372B1 (en) | 2007-12-26 | 2011-06-07 | Emc (Benelux) B.V., S.A.R.L. | Method and apparatus to convert a logical unit from a first encryption state to a second encryption state using a journal in a continuous data protection environment |
US8041940B1 (en) | 2007-12-26 | 2011-10-18 | Emc Corporation | Offloading encryption processing in a storage area network |
US9178785B1 (en) | 2008-01-24 | 2015-11-03 | NextAxiom Technology, Inc | Accounting for usage and usage-based pricing of runtime engine |
US9501542B1 (en) | 2008-03-11 | 2016-11-22 | Emc Corporation | Methods and apparatus for volume synchronization |
US8108634B1 (en) | 2008-06-27 | 2012-01-31 | Emc B.V., S.A.R.L. | Replicating a thin logical unit |
US7719443B1 (en) | 2008-06-27 | 2010-05-18 | Emc Corporation | Compressing data in a continuous data protection environment |
US7840730B2 (en) | 2008-06-27 | 2010-11-23 | Microsoft Corporation | Cluster shared volumes |
US8719473B2 (en) * | 2008-09-19 | 2014-05-06 | Microsoft Corporation | Resource arbitration for shared-write access via persistent reservation |
US7882286B1 (en) | 2008-09-26 | 2011-02-01 | EMC (Benelux)B.V., S.A.R.L. | Synchronizing volumes for replication |
US8060714B1 (en) | 2008-09-26 | 2011-11-15 | Emc (Benelux) B.V., S.A.R.L. | Initializing volumes in a replication system |
EP2352090B1 (en) | 2008-10-06 | 2019-09-25 | International Business Machines Corporation | System accessing shared data by a plurality of application servers |
US8171337B2 (en) * | 2009-03-30 | 2012-05-01 | The Boeing Company | Computer architectures using shared storage |
US8296358B2 (en) * | 2009-05-14 | 2012-10-23 | Hewlett-Packard Development Company, L.P. | Method and system for journaling data updates in a distributed file system |
US20110055494A1 (en) * | 2009-08-25 | 2011-03-03 | Yahoo! Inc. | Method for distributed direct object access storage |
US8055615B2 (en) * | 2009-08-25 | 2011-11-08 | Yahoo! Inc. | Method for efficient storage node replacement |
US9311319B2 (en) * | 2009-08-27 | 2016-04-12 | Hewlett Packard Enterprise Development Lp | Method and system for administration of storage objects |
US20110093745A1 (en) * | 2009-10-20 | 2011-04-21 | Aviad Zlotnick | Systems and methods for implementing test applications for systems using locks |
US8510334B2 (en) | 2009-11-05 | 2013-08-13 | Oracle International Corporation | Lock manager on disk |
US8392680B1 (en) | 2010-03-30 | 2013-03-05 | Emc International Company | Accessing a volume in a distributed environment |
US8103937B1 (en) * | 2010-03-31 | 2012-01-24 | Emc Corporation | Cas command network replication |
US8381014B2 (en) | 2010-05-06 | 2013-02-19 | International Business Machines Corporation | Node controller first failure error management for a distributed system |
US20110276728A1 (en) * | 2010-05-06 | 2011-11-10 | Hitachi, Ltd. | Method and apparatus for storage i/o path configuration |
US8332687B1 (en) | 2010-06-23 | 2012-12-11 | Emc Corporation | Splitter used in a continuous data protection environment |
US9098462B1 (en) | 2010-09-14 | 2015-08-04 | The Boeing Company | Communications via shared memory |
US8433869B1 (en) | 2010-09-27 | 2013-04-30 | Emc International Company | Virtualized consistency group using an enhanced splitter |
US8478955B1 (en) | 2010-09-27 | 2013-07-02 | Emc International Company | Virtualized consistency group using more than one data protection appliance |
US8694700B1 (en) | 2010-09-29 | 2014-04-08 | Emc Corporation | Using I/O track information for continuous push with splitter for storage device |
US8335771B1 (en) | 2010-09-29 | 2012-12-18 | Emc Corporation | Storage array snapshots for logged access replication in a continuous data protection system |
US8589732B2 (en) | 2010-10-25 | 2013-11-19 | Microsoft Corporation | Consistent messaging with replication |
US8335761B1 (en) | 2010-12-02 | 2012-12-18 | Emc International Company | Replicating in a multi-copy environment |
US8812916B2 (en) | 2011-06-02 | 2014-08-19 | International Business Machines Corporation | Failure data management for a distributed computer system |
US9256605B1 (en) | 2011-08-03 | 2016-02-09 | Emc Corporation | Reading and writing to an unexposed device |
US8973018B2 (en) | 2011-08-23 | 2015-03-03 | International Business Machines Corporation | Configuring and relaying events from a storage controller to a host server |
US8694724B1 (en) * | 2011-09-06 | 2014-04-08 | Emc Corporation | Managing data storage by provisioning cache as a virtual device |
US8898112B1 (en) | 2011-09-07 | 2014-11-25 | Emc Corporation | Write signature command |
US8560662B2 (en) * | 2011-09-12 | 2013-10-15 | Microsoft Corporation | Locking system for cluster updates |
US9170852B2 (en) | 2012-02-02 | 2015-10-27 | Microsoft Technology Licensing, Llc | Self-updating functionality in a distributed system |
US20130290385A1 (en) * | 2012-04-30 | 2013-10-31 | Charles B. Morrey, III | Durably recording events for performing file system operations |
US9223659B1 (en) | 2012-06-28 | 2015-12-29 | Emc International Company | Generating and accessing a virtual volume snapshot in a continuous data protection system |
US9218295B2 (en) * | 2012-07-13 | 2015-12-22 | Ca, Inc. | Methods and systems for implementing time-locks |
US10235145B1 (en) | 2012-09-13 | 2019-03-19 | Emc International Company | Distributed scale-out replication |
US9336094B1 (en) | 2012-09-13 | 2016-05-10 | Emc International Company | Scaleout replication of an application |
US9081840B2 (en) * | 2012-09-21 | 2015-07-14 | Citigroup Technology, Inc. | Methods and systems for modeling a replication topology |
US9696939B1 (en) | 2013-03-14 | 2017-07-04 | EMC IP Holding Company LLC | Replicating data using deduplication-based arrays using network-based replication |
US9383937B1 (en) | 2013-03-14 | 2016-07-05 | Emc Corporation | Journal tiering in a continuous data protection system using deduplication-based storage |
US9110914B1 (en) | 2013-03-14 | 2015-08-18 | Emc Corporation | Continuous data protection using deduplication-based storage |
US8996460B1 (en) | 2013-03-14 | 2015-03-31 | Emc Corporation | Accessing an image in a continuous data protection using deduplication-based storage |
US9244997B1 (en) | 2013-03-15 | 2016-01-26 | Emc Corporation | Asymmetric active-active access of asynchronously-protected data storage |
US9081842B1 (en) | 2013-03-15 | 2015-07-14 | Emc Corporation | Synchronous and asymmetric asynchronous active-active-active data access |
US9152339B1 (en) | 2013-03-15 | 2015-10-06 | Emc Corporation | Synchronization of asymmetric active-active, asynchronously-protected storage |
US9069709B1 (en) | 2013-06-24 | 2015-06-30 | Emc International Company | Dynamic granularity in data replication |
US9087112B1 (en) | 2013-06-24 | 2015-07-21 | Emc International Company | Consistency across snapshot shipping and continuous replication |
US9146878B1 (en) | 2013-06-25 | 2015-09-29 | Emc Corporation | Storage recovery from total cache loss using journal-based replication |
US9454485B2 (en) | 2013-08-01 | 2016-09-27 | Lenovo Enterprise Solutions (Singapore) Pte. Ltd. | Sharing local cache from a failover node |
WO2015061554A1 (en) * | 2013-10-25 | 2015-04-30 | Advanced Micro Devices, Inc. | Method and apparatus for performing a bus lock and translation lookaside buffer invalidation |
US9367260B1 (en) | 2013-12-13 | 2016-06-14 | Emc Corporation | Dynamic replication system |
US9405765B1 (en) | 2013-12-17 | 2016-08-02 | Emc Corporation | Replication of virtual machines |
US9158630B1 (en) | 2013-12-19 | 2015-10-13 | Emc Corporation | Testing integrity of replicated storage |
US9372752B2 (en) * | 2013-12-27 | 2016-06-21 | Intel Corporation | Assisted coherent shared memory |
CN106104502B (en) | 2014-03-20 | 2019-03-22 | 慧与发展有限责任合伙企业 | System, method and medium for storage system affairs |
US9189339B1 (en) | 2014-03-28 | 2015-11-17 | Emc Corporation | Replication of a virtual distributed volume with virtual machine granualarity |
US9686206B2 (en) * | 2014-04-29 | 2017-06-20 | Silicon Graphics International Corp. | Temporal based collaborative mutual exclusion control of a shared resource |
US9497140B2 (en) | 2014-05-14 | 2016-11-15 | International Business Machines Corporation | Autonomous multi-node network configuration and self-awareness through establishment of a switch port group |
US9274718B1 (en) | 2014-06-20 | 2016-03-01 | Emc Corporation | Migration in replication system |
US10082980B1 (en) | 2014-06-20 | 2018-09-25 | EMC IP Holding Company LLC | Migration of snapshot in replication system using a log |
US9619543B1 (en) | 2014-06-23 | 2017-04-11 | EMC IP Holding Company LLC | Replicating in virtual desktop infrastructure |
US10237342B2 (en) * | 2014-09-17 | 2019-03-19 | Dh2I Company | Coordinated and high availability storage access |
US10101943B1 (en) | 2014-09-25 | 2018-10-16 | EMC IP Holding Company LLC | Realigning data in replication system |
US10437783B1 (en) | 2014-09-25 | 2019-10-08 | EMC IP Holding Company LLC | Recover storage array using remote deduplication device |
US10324798B1 (en) | 2014-09-25 | 2019-06-18 | EMC IP Holding Company LLC | Restoring active areas of a logical unit |
US9529885B1 (en) | 2014-09-29 | 2016-12-27 | EMC IP Holding Company LLC | Maintaining consistent point-in-time in asynchronous replication during virtual machine relocation |
US9910621B1 (en) | 2014-09-29 | 2018-03-06 | EMC IP Holding Company LLC | Backlogging I/O metadata utilizing counters to monitor write acknowledgements and no acknowledgements |
US9600377B1 (en) | 2014-12-03 | 2017-03-21 | EMC IP Holding Company LLC | Providing data protection using point-in-time images from multiple types of storage devices |
US10496487B1 (en) | 2014-12-03 | 2019-12-03 | EMC IP Holding Company LLC | Storing snapshot changes with snapshots |
US9405481B1 (en) | 2014-12-17 | 2016-08-02 | Emc Corporation | Replicating using volume multiplexing with consistency group file |
US9632881B1 (en) | 2015-03-24 | 2017-04-25 | EMC IP Holding Company LLC | Replication of a virtual distributed volume |
US9411535B1 (en) | 2015-03-27 | 2016-08-09 | Emc Corporation | Accessing multiple virtual devices |
US10296419B1 (en) | 2015-03-27 | 2019-05-21 | EMC IP Holding Company LLC | Accessing a virtual device using a kernel |
US9678680B1 (en) | 2015-03-30 | 2017-06-13 | EMC IP Holding Company LLC | Forming a protection domain in a storage architecture |
US10853181B1 (en) | 2015-06-29 | 2020-12-01 | EMC IP Holding Company LLC | Backing up volumes using fragment files |
US10496538B2 (en) * | 2015-06-30 | 2019-12-03 | Veritas Technologies Llc | System, method and mechanism to efficiently coordinate cache sharing between cluster nodes operating on the same regions of a file or the file system blocks shared among multiple files |
US10360236B2 (en) * | 2015-09-25 | 2019-07-23 | International Business Machines Corporation | Replicating structured query language (SQL) in a heterogeneous replication environment |
US10341252B2 (en) * | 2015-09-30 | 2019-07-02 | Veritas Technologies Llc | Partition arbitration optimization |
US9684576B1 (en) | 2015-12-21 | 2017-06-20 | EMC IP Holding Company LLC | Replication using a virtual distributed volume |
US10235196B1 (en) | 2015-12-28 | 2019-03-19 | EMC IP Holding Company LLC | Virtual machine joining or separating |
US10067837B1 (en) | 2015-12-28 | 2018-09-04 | EMC IP Holding Company LLC | Continuous data protection with cloud resources |
US10133874B1 (en) | 2015-12-28 | 2018-11-20 | EMC IP Holding Company LLC | Performing snapshot replication on a storage system not configured to support snapshot replication |
US10152267B1 (en) | 2016-03-30 | 2018-12-11 | Emc Corporation | Replication data pull |
US10579282B1 (en) | 2016-03-30 | 2020-03-03 | EMC IP Holding Company LLC | Distributed copy in multi-copy replication where offset and size of I/O requests to replication site is half offset and size of I/O request to production volume |
US10235087B1 (en) | 2016-03-30 | 2019-03-19 | EMC IP Holding Company LLC | Distributing journal data over multiple journals |
US10235060B1 (en) | 2016-04-14 | 2019-03-19 | EMC IP Holding Company, LLC | Multilevel snapshot replication for hot and cold regions of a storage system |
CN106055417B (en) * | 2016-06-02 | 2018-09-11 | 北京百度网讯科技有限公司 | Method for message transmission and device for robot operating system |
US10210073B1 (en) | 2016-09-23 | 2019-02-19 | EMC IP Holding Company, LLC | Real time debugging of production replicated data with data obfuscation in a storage system |
US10805238B1 (en) | 2016-09-23 | 2020-10-13 | Amazon Technologies, Inc. | Management of alternative resources |
US10146961B1 (en) | 2016-09-23 | 2018-12-04 | EMC IP Holding Company LLC | Encrypting replication journals in a storage system |
US10019194B1 (en) | 2016-09-23 | 2018-07-10 | EMC IP Holding Company LLC | Eventually consistent synchronous data replication in a storage system |
US10235091B1 (en) | 2016-09-23 | 2019-03-19 | EMC IP Holding Company LLC | Full sweep disk synchronization in a storage system |
US10423459B1 (en) | 2016-09-23 | 2019-09-24 | Amazon Technologies, Inc. | Resource manager |
US10346366B1 (en) | 2016-09-23 | 2019-07-09 | Amazon Technologies, Inc. | Management of a data processing pipeline |
US10666569B1 (en) * | 2016-09-23 | 2020-05-26 | Amazon Technologies, Inc. | Journal service with named clients |
US10235090B1 (en) | 2016-09-23 | 2019-03-19 | EMC IP Holding Company LLC | Validating replication copy consistency using a hash function in a storage system |
US10725915B1 (en) | 2017-03-31 | 2020-07-28 | Veritas Technologies Llc | Methods and systems for maintaining cache coherency between caches of nodes in a clustered environment |
US10459810B2 (en) | 2017-07-06 | 2019-10-29 | Oracle International Corporation | Technique for higher availability in a multi-node system using replicated lock information to determine a set of data blocks for recovery |
US11144493B1 (en) | 2018-05-02 | 2021-10-12 | Ecosense Lighting Inc. | Composite interface circuit |
CN109376014B (en) * | 2018-10-19 | 2021-07-02 | 郑州云海信息技术有限公司 | Distributed lock manager implementation method and system |
US11880350B2 (en) * | 2021-06-08 | 2024-01-23 | International Business Machines Corporation | Identifying resource lock ownership across a clustered computing environment |
Family Cites Families (74)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0833857B2 (en) * | 1987-02-18 | 1996-03-29 | 株式会社日立製作所 | System database sharing system system system |
JP2667039B2 (en) * | 1990-05-18 | 1997-10-22 | 株式会社東芝 | Data management system and data management method |
JPH0827755B2 (en) * | 1991-02-15 | 1996-03-21 | インターナショナル・ビジネス・マシーンズ・コーポレイション | How to access data units at high speed |
US5276872A (en) | 1991-06-25 | 1994-01-04 | Digital Equipment Corporation | Concurrency and recovery for index trees with nodal updates using multiple atomic actions by which the trees integrity is preserved during undesired system interruptions |
US5438464A (en) | 1993-04-23 | 1995-08-01 | Quantum Corporation | Synchronization of multiple disk drive spindles |
JP3023441B2 (en) * | 1993-11-16 | 2000-03-21 | 株式会社日立製作所 | Database division management method and parallel database system |
DE4341877A1 (en) * | 1993-12-08 | 1995-06-14 | Siemens Ag | Coordination to access multiple processors to common resource |
US5454108A (en) * | 1994-01-26 | 1995-09-26 | International Business Machines Corporation | Distributed lock manager using a passive, state-full control-server |
US5751992A (en) * | 1994-09-23 | 1998-05-12 | International Business Machines Corporation | Computer program product for continuous destaging of changed data from a shared cache in a multisystem shared disk environment wherein castout interest is established in a hierarchical fashion |
JP3516362B2 (en) | 1995-03-01 | 2004-04-05 | 富士通株式会社 | Shared data processing device and shared data processing system |
US5699500A (en) * | 1995-06-01 | 1997-12-16 | Ncr Corporation | Reliable datagram service provider for fast messaging in a clustered environment |
US5594863A (en) * | 1995-06-26 | 1997-01-14 | Novell, Inc. | Method and apparatus for network file recovery |
US6356740B1 (en) * | 1995-06-30 | 2002-03-12 | Hughes Electronics Corporation | Method and system of frequency stabilization in a mobile satellite communication system |
JPH09114721A (en) | 1995-10-19 | 1997-05-02 | Nec Corp | Device sharing method and device sharing system in local area network |
US5678026A (en) | 1995-12-28 | 1997-10-14 | Unisys Corporation | Multi-processor data processing system with control for granting multiple storage locks in parallel and parallel lock priority and second level cache priority queues |
US5850507A (en) * | 1996-03-19 | 1998-12-15 | Oracle Corporation | Method and apparatus for improved transaction recovery |
US6016505A (en) | 1996-04-30 | 2000-01-18 | International Business Machines Corporation | Program product to effect barrier synchronization in a distributed computing environment |
US6026426A (en) * | 1996-04-30 | 2000-02-15 | International Business Machines Corporation | Application programming interface unifying multiple mechanisms |
US5920872A (en) | 1996-06-25 | 1999-07-06 | Oracle Corporation | Resource management using resource domains |
US6044367A (en) | 1996-08-02 | 2000-03-28 | Hewlett-Packard Company | Distributed I/O store |
US5875469A (en) * | 1996-08-26 | 1999-02-23 | International Business Machines Corporation | Apparatus and method of snooping processors and look-aside caches |
US5987506A (en) * | 1996-11-22 | 1999-11-16 | Mangosoft Corporation | Remote access and geographically distributed computers in a globally addressable storage environment |
US5909540A (en) * | 1996-11-22 | 1999-06-01 | Mangosoft Corporation | System and method for providing highly available data storage using globally addressable memory |
US6026474A (en) * | 1996-11-22 | 2000-02-15 | Mangosoft Corporation | Shared client-side web caching using globally addressable memory |
US5974250A (en) * | 1996-12-13 | 1999-10-26 | Compaq Computer Corp. | System and method for secure information transmission over a network |
US6108757A (en) * | 1997-02-28 | 2000-08-22 | Lucent Technologies Inc. | Method for locking a shared resource in multiprocessor system |
US5913227A (en) | 1997-03-24 | 1999-06-15 | Emc Corporation | Agent-implemented locking mechanism |
FR2762418B1 (en) * | 1997-04-17 | 1999-06-11 | Alsthom Cge Alcatel | METHOD FOR MANAGING A SHARED MEMORY |
US6237001B1 (en) * | 1997-04-23 | 2001-05-22 | Oracle Corporation | Managing access to data in a distributed database environment |
US5960446A (en) | 1997-07-11 | 1999-09-28 | International Business Machines Corporation | Parallel file system and method with allocation map |
US6021508A (en) | 1997-07-11 | 2000-02-01 | International Business Machines Corporation | Parallel file system and method for independent metadata loggin |
US5953719A (en) | 1997-09-15 | 1999-09-14 | International Business Machines Corporation | Heterogeneous database system with dynamic commit procedure control |
US6112281A (en) * | 1997-10-07 | 2000-08-29 | Oracle Corporation | I/O forwarding in a cache coherent shared disk computer system |
US6108654A (en) * | 1997-10-31 | 2000-08-22 | Oracle Corporation | Method and system for locking resources in a computer system |
US6009466A (en) | 1997-10-31 | 1999-12-28 | International Business Machines Corporation | Network management system for enabling a user to configure a network of storage devices via a graphical user interface |
JPH11143843A (en) | 1997-11-06 | 1999-05-28 | Hitachi Ltd | Operation condition management method for plural nodes configuration system |
US6199105B1 (en) * | 1997-12-09 | 2001-03-06 | Nec Corporation | Recovery system for system coupling apparatuses, and recording medium recording recovery program |
US6256740B1 (en) * | 1998-02-06 | 2001-07-03 | Ncr Corporation | Name service for multinode system segmented into I/O and compute nodes, generating guid at I/O node and exporting guid to compute nodes via interconnect fabric |
US6173293B1 (en) * | 1998-03-13 | 2001-01-09 | Digital Equipment Corporation | Scalable distributed file system |
US6163855A (en) * | 1998-04-17 | 2000-12-19 | Microsoft Corporation | Method and system for replicated and consistent modifications in a server cluster |
US6438582B1 (en) * | 1998-07-21 | 2002-08-20 | International Business Machines Corporation | Method and system for efficiently coordinating commit processing in a parallel or distributed database system |
US6272491B1 (en) * | 1998-08-24 | 2001-08-07 | Oracle Corporation | Method and system for mastering locks in a multiple server database system |
US6154512A (en) | 1998-11-19 | 2000-11-28 | Nortel Networks Corporation | Digital phase lock loop with control for enabling and disabling synchronization |
US6178519B1 (en) * | 1998-12-10 | 2001-01-23 | Mci Worldcom, Inc. | Cluster-wide database system |
US6757277B1 (en) * | 1999-01-26 | 2004-06-29 | Siemens Information And Communication Networks, Inc. | System and method for coding algorithm policy adjustment in telephony-over-LAN networks |
US6226717B1 (en) | 1999-02-04 | 2001-05-01 | Compaq Computer Corporation | System and method for exclusive access to shared storage |
US6269410B1 (en) * | 1999-02-12 | 2001-07-31 | Hewlett-Packard Co | Method and apparatus for using system traces to characterize workloads in a data storage system |
US6725392B1 (en) * | 1999-03-03 | 2004-04-20 | Adaptec, Inc. | Controller fault recovery system for a distributed file system |
WO2000062502A2 (en) | 1999-04-12 | 2000-10-19 | Rainfinity, Inc. | Distributed server cluster for controlling network traffic |
JP4864210B2 (en) * | 1999-05-20 | 2012-02-01 | イヴァン, チョン−ション ホワン, | Work group server implementation method and apparatus |
US6421723B1 (en) | 1999-06-11 | 2002-07-16 | Dell Products L.P. | Method and system for establishing a storage area network configuration |
JP4057201B2 (en) | 1999-09-16 | 2008-03-05 | 富士通株式会社 | High-speed data exchange method between different computers and extent extraction / conversion program recording medium |
US6598058B2 (en) * | 1999-09-22 | 2003-07-22 | International Business Machines Corporation | Method and apparatus for cross-node sharing of cached dynamic SQL in a multiple relational database management system environment |
US6865549B1 (en) * | 1999-11-15 | 2005-03-08 | Sun Microsystems, Inc. | Method and apparatus for concurrency control in a policy-based management system |
US6473819B1 (en) * | 1999-12-17 | 2002-10-29 | International Business Machines Corporation | Scalable interruptible queue locks for shared-memory multiprocessor |
US6618819B1 (en) * | 1999-12-23 | 2003-09-09 | Nortel Networks Limited | Sparing system and method to accommodate equipment failures in critical systems |
US6370625B1 (en) * | 1999-12-29 | 2002-04-09 | Intel Corporation | Method and apparatus for lock synchronization in a microprocessor system |
US7062648B2 (en) | 2000-02-18 | 2006-06-13 | Avamar Technologies, Inc. | System and method for redundant array network storage |
US6643748B1 (en) * | 2000-04-20 | 2003-11-04 | Microsoft Corporation | Programmatic masking of storage units |
US20030041138A1 (en) * | 2000-05-02 | 2003-02-27 | Sun Microsystems, Inc. | Cluster membership monitor |
US6530004B1 (en) * | 2000-06-20 | 2003-03-04 | International Business Machines Corporation | Efficient fault-tolerant preservation of data integrity during dynamic RAID data migration |
US7844513B2 (en) | 2000-07-17 | 2010-11-30 | Galactic Computing Corporation Bvi/Bc | Method and system for operating a commissioned e-commerce service prover |
US6944249B2 (en) | 2000-08-17 | 2005-09-13 | Broadcom Corporation | Method and system for transmitting isochronous voice in a wireless network |
US6665814B2 (en) * | 2000-11-29 | 2003-12-16 | International Business Machines Corporation | Method and apparatus for providing serialization support for a computer system |
US6976060B2 (en) * | 2000-12-05 | 2005-12-13 | Agami Sytems, Inc. | Symmetric shared file storage system |
US8219662B2 (en) | 2000-12-06 | 2012-07-10 | International Business Machines Corporation | Redirecting data generated by network devices |
US20040213239A1 (en) * | 2000-12-15 | 2004-10-28 | Lin Xinming A. | Implementation of IP multicast on ATM network with EMCON links |
US6804794B1 (en) * | 2001-02-28 | 2004-10-12 | Emc Corporation | Error condition handling |
US7130316B2 (en) | 2001-04-11 | 2006-10-31 | Ati Technologies, Inc. | System for frame based audio synchronization and method thereof |
US7107319B2 (en) * | 2001-05-31 | 2006-09-12 | Oracle Corporation | Method and apparatus for reducing latency and message traffic during data and lock transfer in a multi-node system |
US6708175B2 (en) * | 2001-06-06 | 2004-03-16 | International Business Machines Corporation | Program support for disk fencing in a shared disk parallel file system across storage area network |
WO2003027853A1 (en) * | 2001-09-21 | 2003-04-03 | Polyserve, Inc. | A system and method for synchronisation for enforcing mutual exclusion among multiple negotiators |
US6871268B2 (en) * | 2002-03-07 | 2005-03-22 | International Business Machines Corporation | Methods and systems for distributed caching in presence of updates and in accordance with holding times |
US6862666B2 (en) * | 2002-05-16 | 2005-03-01 | Sun Microsystems, Inc. | Hardware assisted lease-based access to memory |
-
2002
- 2002-09-20 WO PCT/US2002/029857 patent/WO2003027853A1/en not_active Application Discontinuation
- 2002-09-20 US US10/251,645 patent/US20040202013A1/en not_active Abandoned
- 2002-09-20 US US10/251,690 patent/US7496646B2/en active Active
- 2002-09-20 JP JP2003529357A patent/JP2005504369A/en active Pending
- 2002-09-20 US US10/251,689 patent/US7149853B2/en not_active Expired - Fee Related
- 2002-09-20 JP JP2003531367A patent/JP4249622B2/en not_active Expired - Lifetime
- 2002-09-20 CA CA2460833A patent/CA2460833C/en not_active Expired - Lifetime
- 2002-09-20 US US10/251,894 patent/US7240057B2/en not_active Expired - Lifetime
- 2002-09-20 CN CNB028230981A patent/CN1302419C/en not_active Expired - Lifetime
- 2002-09-20 CA CA002461015A patent/CA2461015A1/en not_active Abandoned
- 2002-09-20 CN CNB028232313A patent/CN1320483C/en not_active Expired - Lifetime
- 2002-09-20 US US10/251,626 patent/US7111197B2/en active Active
- 2002-09-20 WO PCT/US2002/030083 patent/WO2003025780A1/en not_active Application Discontinuation
- 2002-09-20 US US10/251,893 patent/US7266722B2/en active Active
- 2002-09-20 US US10/251,895 patent/US7437386B2/en active Active
- 2002-09-20 EP EP02768873A patent/EP1428149B1/en not_active Expired - Lifetime
- 2002-09-20 AU AU2002341784A patent/AU2002341784A1/en not_active Abandoned
- 2002-09-20 WO PCT/US2002/030085 patent/WO2003025751A1/en not_active Application Discontinuation
- 2002-09-20 WO PCT/US2002/029859 patent/WO2003027903A1/en active Application Filing
- 2002-09-20 WO PCT/US2002/030082 patent/WO2003025801A1/en active Application Filing
- 2002-09-20 EP EP02775933A patent/EP1428151A4/en not_active Withdrawn
-
2006
- 2006-08-04 US US11/499,907 patent/US7467330B2/en not_active Expired - Lifetime
Also Published As
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7437386B2 (en) | System and method for a multi-node environment with shared storage | |
CA2284376C (en) | Method and apparatus for managing clustered computer systems | |
US7406473B1 (en) | Distributed file system using disk servers, lock servers and file servers | |
US7739677B1 (en) | System and method to prevent data corruption due to split brain in shared data clusters | |
US6360331B2 (en) | Method and system for transparently failing over application configuration information in a server cluster | |
EP1024428B1 (en) | Managing a clustered computer system | |
US6449734B1 (en) | Method and system for discarding locally committed transactions to ensure consistency in a server cluster | |
EP1171817B1 (en) | Data distribution in a server cluster | |
US8495131B2 (en) | Method, system, and program for managing locks enabling access to a shared resource | |
US20050283641A1 (en) | Apparatus, system, and method for verified fencing of a rogue node within a cluster | |
US20040153749A1 (en) | Redundant multi-processor and logical processor configuration for a file server | |
US8316110B1 (en) | System and method for clustering standalone server applications and extending cluster functionality | |
JP2002229837A (en) | Method for controlling access to data in shared disc parallel data file | |
US7702757B2 (en) | Method, apparatus and program storage device for providing control to a networked storage architecture | |
WO2003054711A1 (en) | A system and method for management of a storage area network | |
Kishida et al. | SSDLM: architecture of a distributed lock manager with high degree of locality for clustered file systems | |
JP2009282776A (en) | Computer system | |
Youn et al. | The approaches for high available and fault-tolerant cluster systems | |
Dyke et al. | RAC Concepts | |
Kyne et al. | Clustering Solutions Overview: Parallel Sysplex and Other Platforms | |
Clusterware | RAC Concepts | |
Farazdel et al. | GPFS on AIX Clusters |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
FZDE | Discontinued |