WO2017008687A1

WO2017008687A1 - Message queuing system and method of realizing message communication

Info

Publication number: WO2017008687A1
Application number: PCT/CN2016/089308
Authority: WO
Inventors: 张俊; 何乐; 张�杰
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2015-07-14
Filing date: 2016-07-08
Publication date: 2017-01-19
Also published as: CN107038075A; CN107038075B

Abstract

Provided are a message queuing system and a method of realizing message communication. The system comprises: a first buffer area (21), a second buffer area (22) and a persistent storage (23). The first buffer area (21) and the second buffer area (22) are located in a memory of a server in which a message producer and a message consumer are located. The second buffer area (22) comprises a primary buffer area (221) and a secondary buffer area (222). The first buffer area (21) is configured to buffer a message pushed by the message producer, store the message in the persistent storage (23), and send the message stored in the persistent storage (23) to the second buffer area (22). The persistent storage (23) is configured to persistently store the message sent by the first buffer area (21). The secondary buffer area (222) is configured to buffer the message sent by the first buffer area (21). The primary buffer area (221) is configured to deliver the message therein to the message consumer, and after all of the messages is delivered to the message consumer, perform primary-secondary switching with the secondary buffer area (222). The system improves the real-time performance of message transmission in the message queuing system.

Description

Message queuing system and method for implementing message communication

The present application claims priority to Chinese Patent Application No. 201510413284.1, the entire disclosure of which is hereby incorporated by reference in its entirety in its entirety in the the the the the the the the the

Technical field

The present application relates to the field of computers, and more particularly to a message queuing system and a method of implementing message communication.

Background technique

In computer systems, message queues are an important way to achieve communication between different processes or between different threads in the same process. That is, message transmission is done in the message queue between different processes or between different threads.

In the related art, various implementations of the message queue system have existed. One solution is that the message queuing system consists of a cache area on the server where the process (or thread) resides and a local disk. The message pushed by the message producer is first stored in the buffer area. When the number of messages in the buffer area reaches the configuration value or the publication time of the message exceeds the threshold, the message in the buffer area is stored in the local disk. In order to ensure the reliability of the message. And only messages stored on the local disk can be further passed to the message consumer. In addition, due to the limited storage space of the local disk, messages stored in the local disk are deleted whether they are delivered to the message consumer, and are deleted after a certain period of time to release the storage space of the local disk. The so-called "reliability" means that the message pushed by the message producer to the message queue system is not lost, even if the message queue system fails, and the message can be saved until it is delivered to the message consumer. In the system until it is passed to the message consumer.

In the first scenario, only messages located on the local disk can be delivered to the message consumer, and the message must be stored in the cache before being stored on the local disk. Therefore, the real-time performance of the message transmission will be Relatively poor. Moreover, the message is delivered from the local disk to the message consumer, and is affected by the speed of the IO operation of the local disk, and the real-time performance of the message transmission is even worse. In addition, because the messages on the local disk are periodically deleted, the reliability of the message is also very poor.

Another solution is that the message queuing system consists of the memory on the server where the process (or thread) is located and the database located outside the server. Wherein, the message pushed by the message producer is first stored in the memory, and then the message is Passed from memory to the message consumer. Moreover, in order to ensure the reliability of the message, the messages in the memory are also stored in the database. Obviously, in the second scenario, the message is passed directly from memory to the message consumer, and the IO operation speed of the memory is faster than the IO operation speed of the local disk, so the real-time performance of the message transmission is compared with the first scheme. better. In addition, because the storage space of the database is relatively large, messages stored in the database are not deleted periodically, so the reliability of the message is better than the first scheme.

However, in the process of implementing the present application, the inventor of the present application found that at least the following problems exist in the prior art: although in the second scheme, the IO operation speed of the memory is relatively fast, so that the message is transmitted in the message queue system. The real-time performance is better than the first one. However, since the memory does not support simultaneous read/write operations, the message pushed by the message producer is stored in the memory, and the message cannot be transferred from the memory to the message consumer, which greatly affects The real-time nature of message transmission.

In addition, in the second scenario, if the message consumer cannot receive the message normally from the message queue system or can only receive the message at a very slow speed, and the message producer produces the message very fast, the more it will be The more messages you get, the more memory you have, and the more serious the memory overflow. When a memory overflow occurs, the overflowed message is discarded, so that the overflowed message cannot be delivered to the message consumer.

Summary of the invention

In order to solve the above technical problem, the embodiment of the present application provides a message queuing system and a method for implementing message communication, so as to solve the real-time problem in the prior art that the message is transmitted in the message queuing system due to the memory not supporting concurrently performing the read/write operation. Sexual problem. In addition, the problem that the overflow message caused by the memory overflow in the message queue system in the prior art cannot be normally transmitted to the message consumer is further solved.

The embodiment of the present application discloses the following technical solutions:

A message queuing system includes: a first buffer, a second buffer, and a persistent storage area; wherein the first buffer and the second buffer are located in a memory of a server where a message producer and a message consumer are located The second buffer includes a main buffer and a backup buffer;

The first buffer is configured to cache a message pushed by a message producer, store a message pushed by the message producer into the persistent storage area, and send a message that has been stored to the persistent storage area to the Said second buffer;

The persistent storage area is configured to persistently store the message sent by the first buffer by using a persistent storage medium;

The backup buffer is configured to buffer a message sent by the first buffer.

The main buffer is configured to deliver a message located in the main buffer to a message consumer, where After all the messages of the main buffer are delivered to the message consumer, the active and standby switches are performed with the backup buffer.

Preferably, the backup buffer is further configured to check whether there is a memory overflow before buffering the message sent by the first buffer, and if there is a memory overflow, discard the message sent by the first buffer, and The current status of itself is marked as an abnormal state, and if there is no memory overflow, the message sent by the first buffer is continuously buffered;

The main buffer is further configured to check whether the current state of the backup buffer is in a normal state before performing the active/standby switchover with the standby buffer, and if not in the normal state, not in the standby buffer. The active/standby switchover is performed, and the message is read from the persistent storage area, and the read message is delivered to the message consumer. If it is in a normal state, the active/standby switchover is continued with the standby buffer.

Preferably, the main buffer is further configured to: after reading a message from the persistent storage area, determine whether there is a next read message in the backup buffer, and if yes, the preparation The current state of the buffer is marked as normal, and if it does not exist, the current state of the standby buffer is maintained.

Preferably, the persistent storage area is located in a distributed file system external to the server.

Preferably, the main buffer and the backup buffer are isomorphic buffers.

Preferably, the message producer and message consumer are two processes running on the server or two threads running in a process on the server.

A method for implementing message communication, which first establishes a first buffer and a second buffer in a memory of a server where a message producer and a message consumer are located, and the second buffer includes a main buffer and a backup buffer. The methods include:

The first buffer buffers the message pushed by the message producer, and stores the message pushed by the message producer into the persistent storage area to store the message persistently in the persistent storage area;

The first buffer sends a message that has been stored to the persistent storage area to the second buffer;

The backup buffer caches the message sent by the first buffer;

The main buffer passes the message in the main buffer to the message consumer, and performs the active/standby switchover with the standby buffer after all the messages located in the main buffer are delivered to the message consumer.

Preferably, the method further includes:

Before buffering the message sent by the first buffer, the backup buffer checks whether there is a memory overflow. If there is a memory overflow, discards the message sent by the first buffer and marks its current state as an abnormal state;

Before performing the active/standby switchover with the standby buffer, the main buffer checks whether the current state of the standby buffer is normal. If it is in an abnormal state, it does not perform the active/standby switchover with the standby buffer, but reads from the persistent storage area. Take the message, In order to pass the read message to the message consumer;

The message sent by the buffer buffer in the first buffer is:

If there is no memory overflow, the backup buffer caches the message sent by the first buffer;

The active/standby switch between the primary buffer and the standby buffer is as follows:

If it is in the normal state, the primary buffer and the standby buffer are switched between active and standby.

Preferably, the method further includes:

After reading the message from the persistent storage area, the main buffer determines whether there is a next read message in the standby buffer, and if so, marks the current state of the standby buffer as a normal state if it does not exist. , maintaining the current state of the standby buffer.

Preferably, the main buffer and the backup buffer are isomorphic buffers.

As can be seen from the above embodiments, the advantages of the present application over the prior art are:

A buffer for buffering messages is set in the memory of the server where the message producer and the message consumer are located. Under normal circumstances, the message transmission is completed through the memory access, which ensures the real-time transmission of the message in the message queue system. Moreover, through the dual structure design of the primary and secondary buffers, it is possible to transmit the message from the main buffer to the message consumer while storing the message pushed by the message producer in the backup buffer, thereby further improving the message. Real-time transmission in the message queue system. In addition, the message pushed by the message producer is stored in the persistent storage area to ensure the reliability of the message.

In addition, when there is a memory overflow problem in the standby buffer in the message queue system, that is, in an abnormal situation, the lost message in the memory can also be retrieved from the persistent storage area, thereby avoiding the memory in the message queue system. A message that a memory overflow has caused an overflow to be delivered to the message consumer.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative labor for those skilled in the art.

FIG. 1 schematically illustrates an exemplary application scenario in which embodiments of the present invention may be implemented;

FIG. 2 is a schematic structural diagram of a message queuing system according to an embodiment of the present application; FIG.

FIG. 3 is a schematic structural diagram of another message queuing system according to an embodiment of the present application; FIG.

FIG. 4 is a schematic structural diagram of another message queuing system according to an embodiment of the present application; FIG.

FIG. 5 is a schematic diagram showing the operation of a main buffer read message according to an embodiment of the present application; FIG.

FIG. 6 schematically illustrates a flowchart of a method of implementing message communication in accordance with an embodiment of the present application; FIG.

FIG. 7 schematically illustrates a flow chart of another method of implementing message communication in accordance with an embodiment of the present application; FIG.

FIG. 8 schematically illustrates a flow chart of another method of implementing message communication in accordance with an embodiment of the present application.

detailed description

Referring to FIG. 1, FIG. 1 schematically illustrates an exemplary application scenario in which embodiments of the present invention may be implemented. There is a message producer 10 and a message consumer 20 on the server. When the message producer 10 communicates with the message consumer 20, the message producer 10 pushes the message 21 to the message queue system 30, the message queue. System 30 is responsible for communicating message 21 to message consumer 20. Additionally, to ensure the reliability of the message 21, the message queue system 30 also needs to store the message 21 pushed by the message producer 10 into a persistent storage medium, such as a local disk on the server. Among them, "delivery" can be expressed as "push" and "pull" modes. The "push" mode is initiated by the message queue system 30 to push the message 21 to the message consumer 20, "pull" mode. It is the message consumer 20 actively pulling the message 21 from the message queue system 30 by polling. The message producer 10 and the message consumer 20 can be two processes on the server or two threads in the same process on the server. Those skilled in the art will appreciate that the schematic diagram shown in Figure 1 is merely one example in which embodiments of the present invention may be implemented. The scope of application of embodiments of the invention is not limited by any aspect of the framework. For example, there are multiple message producers and multiple message consumers on the server, rather than just one message producer and one message consumer as shown in FIG.

The above described objects, features and advantages of the present application will become more apparent and understood.

Device embodiment

The embodiment of the present application also provides a message queuing system. Please refer to FIG. 2. FIG. 2 is a schematic structural diagram of a message queuing system according to an embodiment of the present application. The message queue system 20 includes a first buffer 21, a second buffer 22, and a persistent storage area 23. The first buffer 21 and the second buffer 22 are located at the server where the message producer and the message consumer are located. In the memory, the second buffer 22 includes a main buffer 221 and a backup Buffer 222. The internal structure and connection relationship will be further described below in conjunction with the working principle of the device.

The first buffer 21 is configured to cache the message pushed by the message producer, store the message pushed by the message producer in the persistent storage area 23, and send the message stored in the persistent storage area 23 to the second buffer. twenty two.

After the message produced by the message producer reaches the first buffer 21, it is firstly stored in the persistent storage area 23 to ensure the reliability of the message. If the storage fails, the persistent storage area 23 informs the first buffer 21 that the message storage has failed, so that the first buffer 21 re-stores the message in the persistent storage area 23 until the storage is successful, if the storage is successful, the persistent storage The area 23 informs the first buffer 21 that the message storage was successful.

The persistent storage area 23 is configured to persistently store the message sent by the first buffer 21 by using the persistent storage medium.

In an optional embodiment of the present application, the persistent storage area 23 is located in a disk inside the server where the message producer and the message consumer are located, as shown in FIG. 2, or in a database external to the server.

In another alternative embodiment of the present application, the persistent storage area 23 is located in a distributed file system external to the server, as shown in FIG. Arranging the persistent storage area 23 in a distributed file system makes it possible to reliably implement the need for persistent storage of mass messages. In addition, in the distributed file system, by using the LogFile append method to record messages, it is also possible to avoid the delay problem caused by the remote operation of the distributed file system by the first buffer 21.

The buffer 222 is configured to buffer the message sent by the first buffer 21.

The main buffer 221 is configured to deliver the message located in the main buffer 221 to the message consumer, and after the message located in the main buffer 221 is all delivered to the message consumer, perform the active/standby switch with the backup buffer 222.

That is, the spare buffer 222 is dedicated to buffering messages sent by the first buffer, while the main buffer 221 is dedicated to delivering messages that are located to itself to the message consumer. Once the main buffer 221 passes all of its own messages, it exchanges the identity with the standby buffer, and converts the active/standby relationship, that is, the original main buffer becomes the standby buffer, and starts to buffer the message from the first buffer 21. And the original standby buffer becomes the main buffer, passing its cached message to the message consumer. In the above normal scenario, the entire message queue system does not need to read the message backed up in the persistent storage area 23, and all the logic is in three buffers (ie, the first buffer 21, the main buffer 221, and the spare buffer). Completed in zone 222).

It can be understood that when a buffer for buffering messages is set in the memory of the server, the dual structure design of the primary and secondary buffers can be implemented while storing the message pushed by the message producer in the backup buffer. Messages are passed from the main buffer to the message consumer, which better guarantees the real-time nature of the message transmission in the message queue system.

In an optional implementation manner of the present application, the main buffer 221 and the backup buffer 222 may be isomorphic buffers, that is, the storage mediums of the two buffers and the storage space are the same size.

Considering that the message consumer may not be able to receive messages from the message queue system at times, or can only receive messages at a very slow rate, and the message producer produces the message very quickly, in this case the main buffer 221 The progress of delivering the message does not match the progress of the buffer buffer 222 to cache the message. The former is faster than the latter. Therefore, before the main buffer 221 passes all of its own messages, the buffer 222 is buffered by the first buffer. The area 21 forwards too many messages and a memory overflow occurs. When a memory overflow occurs, the backup buffer 222 actively discards the overflowed message, and the overflowed message cannot be normally delivered to the message consumer. Therefore, in order to solve the problem that the overflowed message cannot be normally delivered to the message consumer due to the memory overflow of the backup buffer 222, in another optional implementation manner of the present application, as shown in FIG.

The buffer 222 is further configured to check whether there is a memory overflow before buffering the message sent by the first buffer 21. If there is a memory overflow, discard the message sent by the first buffer 21 and mark its current status as In an abnormal state, if there is no memory overflow, the message sent by the first buffer 21 is continuously buffered.

The main buffer 221 is further configured to check whether the current state of the backup buffer 222 is in a normal state before performing the active/standby switchover with the standby buffer 222. Instead, the message is read from the persistent storage area 23, and the read message is delivered to the message consumer. If it is in the normal state, the active/standby switchover with the standby buffer continues.

That is to say, when the memory buffer 222 overflows, it is marked as abnormal, indicating that the message buffered by the current backup buffer 222 is incomplete. For some reason, the buffer 222 has discarded some messages. Therefore, in this abnormal scenario, the main buffer 221 does not perform the active/standby switchover with the standby buffer 222, but directly accesses the persistent storage area 23 (eg, the underlying distributed file system), and the persistent storage area is to be persistent. The message saved in 23 is read and then passed to the message consumer. The main buffer 221 can read a corresponding number of messages from the persistent storage area 23 according to the size of its own storage space, and then deliver the read messages one by one to the message consumer.

This manner can continue until the main buffer 221 finds that the progress of transmitting the message itself can match the progress of the buffer buffer 222 buffering the message, and the main buffer 221 no longer needs to read the message from the persistent storage area 23. Instead, it returns to the normal scenario and delivers the message through the active/standby switchover mode. Thus, in another alternative embodiment of the present application:

The main buffer 221 is further configured to: after reading the message from the persistent storage area 23, determine whether there is a next read message in the backup buffer 222. If yes, the progress of the main buffer 221 to deliver the message may be And preparation The buffer 222 caches the progress of the message, and the main buffer 221 marks the current state of the backup buffer 222 as a normal state. If not, the current state of the standby buffer 222 is maintained.

For example, as shown in FIG. 5, the backup buffer 222 caches the message with the identifier 30-50, and the message identified as 1-29 is discarded by the backup buffer 222 due to the memory overflow. The main buffer 221 is firstly taken from the bottom layer. The message is read in the distributed file system. It is assumed that the main buffer 221 can only cache 20 messages at a time, and the main buffer 221 reads the messages identified as 1-20 from the distributed file system, and transmits them to the messages one by one. Message consumer. After the reading is completed, the main buffer 221 determines whether the next read message (ie, the message identified as 21) is in the backup buffer 222. After the judgment, it is found that the next read message with the identifier 21 is not present. In the spare buffer 222, the main buffer 221 continues to maintain the current state flag of the spare buffer 222, that is, an abnormal state. When the main buffer 221 passes all of its own messages, it immediately checks the current state of the standby buffer 222. Since the current state of the checked standby buffer 222 is still abnormal, it continues from the underlying distributed file system. The message is read, ie the messages identified as 21-40 are read and passed to the message consumer one by one. After reading, the main buffer 221 continues to determine whether the next read message (ie, the message identified as 41) is in the backup buffer 222. After the judgment, the next read message with the identifier 41 is read. In the spare buffer 222, the main buffer 221 marks the current state of the spare buffer 222 as a normal state. After the main buffer 221 transmits its own message again, the current state of the checked buffer 222 is normal, so the active/standby switch can be normally performed with the backup buffer 222.

In an alternative embodiment of the present application, the message producer and the message consumer are two processes running on the server or two threads running in a process on the server.

Method embodiment

Please refer to FIG. 6. FIG. 6 is a flow chart schematically showing a method for implementing message communication according to an embodiment of the present application. For example, the method may be performed by the message queue system 20, and the method may include the following steps, for example:

Step 600: Establish a first buffer and a second buffer in advance in a memory of a server where the message producer and the message consumer are located, and the second buffer includes a main buffer and a backup buffer.

Step 601: The first buffer caches the message pushed by the message producer, and stores the message pushed by the message producer into the persistent storage area to store the message persistently in the persistent storage area.

Step 602: The first buffer sends a message that has been stored to the persistent storage area to the second buffer.

Step 603: The backup buffer caches the message sent by the first buffer.

Step 604: The main buffer passes the message located in the main buffer to the message consumer.

Step 605: The main buffer performs an active/standby switchover with the backup buffer after all the messages located in the main buffer are delivered to the message consumer.

As shown in FIG. 7, in an optional implementation manner of the present application, the method may include the following steps, for example:

Step 603a: The standby buffer checks whether there is a memory overflow. If there is a memory overflow, the process proceeds to step 606. If there is no memory overflow, the process proceeds to step 603.

Step 603: The backup buffer caches the message sent by the first buffer.

Step 605a: After all the messages located in the main buffer are delivered to the message consumer, the main buffer checks whether the current state of the backup buffer is in a normal state. If it is in an abnormal state, the process proceeds to step 607, if it is a normal state. Go to step 605.

Step 605: The primary buffer and the standby buffer perform the active/standby switchover, and the process ends.

Step 606: The backup buffer discards the message sent by the first buffer, and marks its current state as an abnormal state, and ends the process.

Step 607: The main buffer does not perform the active/standby switchover with the standby buffer, and reads the message from the persistent storage area to deliver the read message to the message consumer, and ends the process.

As shown in FIG. 8, in another optional implementation manner of the present application, the method may include the following steps, for example:

Step 603: The backup buffer caches the message sent by the first buffer.

Step 607: The main buffer does not perform the active/standby switchover with the standby buffer, and reads the message from the persistent storage area to deliver the read message to the message consumer.

Step 608: The main buffer determines whether there is a next read message in the spare buffer. If yes, the process proceeds to step 609. If not, the process proceeds to step 610.

Step 609: Mark the current state of the backup buffer as a normal state, and end the process.

Step 610: Maintain the current state of the backup buffer and end the process.

In another optional embodiment of the present application, the persistent storage area is located in a distributed file system external to the server.

In another optional implementation manner of the present application, the primary buffer and the secondary buffer are isomorphic buffers.

In another optional embodiment of the present application, the message producer and the message consumer are two processes running on the server or two threads running in a process on the server.

A buffer for buffering messages is set in the memory of the server where the message producer and the message consumer are located. Under normal circumstances, the message transmission is completed through the memory access, which ensures the real-time transmission of the message in the message queue system. Moreover, through the dual structure design of the primary and secondary buffers, it is possible to store the message pushed by the message producer in the standby At the same time as the buffer, the message is transmitted from the main buffer to the message consumer, which further improves the real-time transmission of the message in the message queue system. In addition, the message pushed by the message producer is stored in the persistent storage area to ensure the reliability of the message.

A person skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined. Or it can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may be or may be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware and can be implemented in the form of a software functional unit.

It should be noted that those skilled in the art can understand that all or part of the process of implementing the foregoing embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage. In the medium, the program, when executed, may include the flow of an embodiment of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The message queuing system and the method for implementing the message communication provided by the present application are described in detail above. The principles and implementation manners of the present application are described in the specific embodiments. The description of the above embodiments is only used to help understand the present application. Method and its core idea; at the same time, for those of ordinary skill in the art, based on this The concept of the application is subject to change in the specific embodiments and the scope of application. In summary, the content of the specification should not be construed as limiting the application.

Claims

A message queuing system, comprising: a first buffer, a second buffer, and a persistent storage area; wherein the first buffer and the second buffer are located at a message producer and a message consumer In the memory of the server, the second buffer includes a main buffer and a backup buffer;

The first buffer is configured to cache a message pushed by a message producer, store a message pushed by the message producer into the persistent storage area, and send a message that has been stored to the persistent storage area to the Said second buffer;

The persistent storage area is configured to persistently store the message sent by the first buffer by using a persistent storage medium;

The backup buffer is configured to buffer a message sent by the first buffer.

The main buffer is configured to deliver a message in the main buffer to a message consumer, and after the message located in the main buffer is all delivered to the message consumer, perform primary and backup with the backup buffer. Switch.
The system of claim 1 wherein:

The buffer is further configured to check whether there is a memory overflow before the message sent by the first buffer is cached. If there is a memory overflow, discard the message sent by the first buffer, and The status flag is in an abnormal state, and if there is no memory overflow, the message sent by the first buffer is continuously buffered;

The main buffer is further configured to check whether the current state of the backup buffer is in a normal state before performing the active/standby switchover with the standby buffer, and if not in the normal state, not in the standby buffer. The active/standby switchover is performed, and the message is read from the persistent storage area, and the read message is delivered to the message consumer. If it is in a normal state, the active/standby switchover is continued with the standby buffer.
The system according to claim 2, wherein said main buffer is further configured to determine whether there is a next read in said spare buffer after reading a message from said persistent storage area The message, if present, marks the current state of the spare buffer as a normal state, and if not, maintains the current state of the standby buffer.
A system according to any one of claims 1 to 3, wherein the persistent storage area is located in a distributed file system external to the server.
The system according to any one of claims 1 to 4, wherein the main buffer and the spare buffer are isomorphic buffers.
A system according to any one of claims 1 to 4, wherein said message producer and message consumer are two processes running on said server or a process running on said server Two threads in .
A method for implementing message communication, characterized in that a first buffer and a second buffer are established in advance in a memory of a server where a message producer and a message consumer are located, and the second buffer includes a main buffer and a backup Buffer, the method includes:

The first buffer buffers the message pushed by the message producer, and stores the message pushed by the message producer into the persistent storage area to store the message persistently in the persistent storage area;

The first buffer sends a message that has been stored to the persistent storage area to the second buffer;

The backup buffer caches the message sent by the first buffer;

The main buffer passes the message in the main buffer to the message consumer, and performs the active/standby switchover with the standby buffer after all the messages located in the main buffer are delivered to the message consumer.
The method of claim 7, wherein the method further comprises:

Before buffering the message sent by the first buffer, the backup buffer checks whether there is a memory overflow. If there is a memory overflow, discards the message sent by the first buffer and marks its current state as an abnormal state;

Before performing the active/standby switchover with the standby buffer, the main buffer checks whether the current state of the standby buffer is normal. If it is in an abnormal state, it does not perform the active/standby switchover with the standby buffer, but reads from the persistent storage area. Take a message to pass the read message to the message consumer;

The message sent by the buffer buffer in the first buffer is:

If there is no memory overflow, the backup buffer caches the message sent by the first buffer;

The active/standby switch between the primary buffer and the standby buffer is as follows:

If it is in the normal state, the primary buffer and the standby buffer are switched between active and standby.
The method of claim 8 further comprising:

After reading the message from the persistent storage area, the main buffer determines whether there is a next read message in the standby buffer, and if so, marks the current state of the standby buffer as a normal state if it does not exist. , maintaining the current state of the standby buffer.
A method according to any one of claims 7 to 9, wherein the persistent storage area is located in a distributed file system external to the server.
The method according to any one of claims 7 to 10, wherein the main buffer and the spare buffer are isomorphic buffers.
The method according to any one of claims 7 to 10, wherein the message producer and the message consumer are two processes running on the server or one running on the server Two threads in the process.