CN105243006A

CN105243006A - Flow layer setting method and apparatus based on flow experiment and flow experiment implementing method and apparatus

Info

Publication number: CN105243006A
Application number: CN201510641992.0A
Authority: CN
Inventors: 汤万万; 张芹芹; 张钋; 侯俊琦
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Baidu Online Network Technology Beijing Co Ltd; Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2015-09-30
Filing date: 2015-09-30
Publication date: 2016-01-13
Anticipated expiration: 2035-09-30
Also published as: CN105243006B

Abstract

The disclosure provides a flow layer setting method and apparatus based on a flow experiment and a flow experiment implementing method and apparatus. The flow layer setting method comprises: acquiring an identifier of a network access user; according to a network access user number equalization principle in each user unit, setting the identifier of the network access user in one user unit on a predetermined flow layer; and according to a corresponding relationship between the user units of any one flow layer and the user units of other n flow layers, respectively setting the identifier of the network access user in one user unit of each of other flow layers, wherein each flow layer comprises n sampling units, each sampling unit comprises n user units, the corresponding relationship is set on the basis that n user units in any one sampling unit of any one flow layer are respectively set into n sampling units of any one of other flow layers, and n is an integer greater than 1. According to the technical scheme provided by the disclosure, accuracy of the flow experiment is effectively improved.

Description

Flow layer based on flux experiment is arranged and the implementation method of flux experiment and device

Technical field

The disclosure relates to computer technology, especially relates to a kind of flow layer method to set up based on flux experiment, the flow layer setting device based on flux experiment, the implementation method of flux experiment and the implement device of flux experiment.

Background technology

The page in internet often exists upgrades demand, as often related to renewal of the page when version is changed in website, for another example for search engine, when forming Search Results, card pattern and determining represents the displayed page generation respective change that can cause Search Results when the contents such as the bottom strategy of result change, as certain Search Results represent that position certain card that there occurs in change, the page there occurs change or certain Search Results represent pattern due to optimised and there occurs change etc.

Under normal circumstances, before new page full flow is reached the standard grade, often carry out flux experiment (as AB low discharge experiment etc.).The release experiment page (i.e. new page) of flux experiment and first low discharge (i.e. partial discharge), then, according to the operation of each network calling party to the experiment page or the contrast page (as usual the page), the experiment page and the contrast page are assessed.Decision-making party can determine whether reached the standard grade by new page full flow according to the assessment result of flux experiment.

Usually multiple flow layer is provided with, to choose sampling unit to determine the network calling party showing the experiment page and the network calling party showing the contrast page from each flow layer in flux experiment.The set-up mode of flow layer is generally: carry out random Harsh computing to the cookie of all-network calling party, and all-network calling party to be arranged in a flow layer in corresponding sampling unit according to Hash operation result, because needs arrange multiple flow layer, therefore, the cookie of reply all-network calling party repeats repeatedly random Harsh computing, to form all flow layers; The quantity of the sampling unit that each flow layer comprises is identical; Each sampling unit makes it comprise network calling party quantity due to the reason of Hash operation is substantially identical.

Usually multiple flux experiment can be there is simultaneously and the situation of depositing in network, when multiple contents such as in Geju City page change simultaneously, often need to assess respectively each change, thus need to arrange experimental group and control group respectively for each change; Sampling unit due to the network calling party place in each flow layer is determined by the result of Hash operation, therefore, when choosing sampling unit to arrange multiple experimental group and control group, there will be the phenomenon that a network calling party occurs in multiple groups simultaneously, but, if experimental group and different from the accounting that there is such network calling party in its corresponding control group, then can produce harmful effect to the assessment result of flux experiment.In addition, existing flux experiment is when carrying out user behavior analysis according to each network calling party to the operation of testing the page or the contrast page, often need to filter the daily record of all-network calling party, to obtain the daily record of the network calling party in experimental group and control group; This daily record filter operation undoubtedly can time of at substantial and resource, thus can affect the efficiency of flux experiment and realize cost etc.

Summary of the invention

The object of this invention is to provide implementation method and the device of the setting of a kind of flow layer based on flux experiment and flux experiment.

According to the present invention first aspect, provide a kind of flow layer method to set up based on flux experiment, and described method mainly comprises the following steps: the mark obtaining network calling party; According to the network calling party equal number principle in each Subscriber Unit, the mark of described network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer; According to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, the mark of described network calling party is arranged at respectively in a Subscriber Unit of other each flow layers; Wherein, each flow layer includes n sampling unit, each sampling unit includes n Subscriber Unit, and described corresponding relation point to be located in n sampling unit of other arbitrary flow layers based on n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer to arrange, n be greater than 1 integer.

According to the present invention second aspect, provide a kind of implementation method of flux experiment, and described method mainly comprises the following steps: from n+1 flow layer, choose a flow layer; The sampling unit that K+h different is selected, wherein K the corresponding experimental group of sampling unit, h the corresponding control group of sampling unit from the described flow layer chosen; Network access request according to the network calling party belonging to experimental group or control group sends the page for flux experiment to described network calling party; Wherein, n sampling unit is included in each flow layer, each sampling unit includes n Subscriber Unit, and n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer point is located in n sampling unit of other arbitrary flow layers, n be greater than 1 integer, K and h is the integer being more than or equal to 1, and K+h<=n.

According to third aspect of the present invention, provide a kind of flow layer setting device based on flux experiment, and described device mainly comprises: user ID acquisition device, for obtaining the device of the mark of network calling party; First setting device, for being arranged at the device in a Subscriber Unit of predetermined amount of flow layer by the mark of described network calling party according to the network calling party equal number principle in each Subscriber Unit; Second setting device, for according to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, is arranged at the device in a Subscriber Unit of other each flow layers respectively by the mark of described network calling party; Wherein, each flow layer includes n sampling unit, each sampling unit includes n Subscriber Unit, and described corresponding relation point to be located in n sampling unit of other arbitrary flow layers based on n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer to arrange, n be greater than 1 integer.

According to the present invention the 4th aspect, provide a kind of implement device of flux experiment, and described device mainly comprises: the first selecting device, for choosing the device of a flow layer from n+1 flow layer; Second selecting device, for selecting the device of K+h different sampling unit in the flow layer chosen from this, wherein, K the corresponding experimental group of sampling unit, in addition K sampling unit correspondence control group; Page dispensing device, for sending device for the page of flux experiment according to the network access request of the network calling party belonging to experimental group or control group to described network calling party; Wherein, include n sampling unit in each flow layer, each sampling unit includes n Subscriber Unit, and n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer point is located in n sampling unit of other arbitrary flow layers, n be greater than 1 integer, K be more than or equal to 1 integer.

Compared with prior art, the present invention has the following advantages: in n+1 provided by the invention flow layer, each flow layer includes n sampling unit, and each sampling unit includes n Subscriber Unit, like this, utilizing the corresponding relation pre-set that n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer can be made to be divided is located in n sampling unit of other arbitrary flow layers, thus in each sampling unit that n Subscriber Unit in any one sampling unit in each flow layer can be evenly distributed in other n flow layer, and then the present invention can make between different flow layer completely orthogonal, carrying out in flux experiment process, owing to being choose sampling unit to arrange multiple experimental group and control group from completely orthogonal flow layer, therefore, even if exist a network calling party appear at simultaneously different flow experiment experimental group/control group in phenomenon, but the experimental group of testing due to arbitrary particular flow rate and with exist in its corresponding control group such to derive from certain experiment of testing of other flow layer/accounting of the network calling party of control group is identical, therefore, the harmful effect that the present invention can avoid the experiment of different flow layer to produce experiment effect assessment each other, seen from the above description, technical scheme provided by the invention improves the accuracy of flux experiment.

Accompanying drawing explanation

By reading the detailed description done non-limiting example done with reference to the following drawings, other features, objects and advantages of the present invention will become more obvious:

Fig. 1 is the flow layer method to set up process flow diagram based on flux experiment of the embodiment of the present invention one;

Fig. 2 is the schematic diagram of any two flow layers of the embodiment of the present invention one;

Fig. 3 is the schematic diagram of five flow layers of the embodiment of the present invention one;

Fig. 4 is the implementation method process flow diagram of the flux experiment of the embodiment of the present invention two;

Fig. 5 is the flow layer setting device schematic diagram based on flux experiment of the embodiment of the present invention three;

Fig. 6 is the implement device schematic diagram of the flux experiment of the embodiment of the present invention four.

In accompanying drawing, same or analogous Reference numeral represents same or analogous parts.

Embodiment

Before in further detail exemplary embodiment being discussed, it should be mentioned that some exemplary embodiments are described as the process or method described as process flow diagram.Although operations is described as the process of order by process flow diagram, many operations wherein can be implemented concurrently, concomitantly or simultaneously.In addition, the execution sequence of operations can be rearranged.Described process can be terminated when its operations are completed, but can also have the additional step do not comprised in the accompanying drawings.Described process can correspond to method, function, code, subroutine, subroutine etc.

Within a context alleged " computer equipment ", also referred to as " computer ", refer to the intelligent electronic device that can be performed the predetermined process such as numerical evaluation and/or logical calculated process by operation preset program or instruction, it can comprise processor and storer, the survival instruction that prestores in memory is performed to perform predetermined process process by processor, or perform predetermined process process by the hardware such as ASIC, FPGA, DSP, or combined by said two devices and realize.Computer equipment includes but not limited to server, PC, notebook computer, panel computer, smart mobile phone etc.

Described computer equipment comprises subscriber equipment and the network equipment.Wherein, described subscriber equipment includes but not limited to computer, smart mobile phone, PAD, PDA etc.; The described network equipment includes but not limited to the server group that single network server, multiple webserver form or the cloud be made up of a large amount of computing machine or the webserver based on cloud computing (CloudComputing), wherein, cloud computing is the one of Distributed Calculation, the super virtual machine be made up of a group loosely-coupled computing machine collection.Wherein, described computer equipment isolated operation can realize the present invention, also accessible network by realizing the present invention with the interactive operation of other computer equipments in network.Wherein, the network residing for described computer equipment includes but not limited to internet, wide area network, Metropolitan Area Network (MAN), LAN (Local Area Network), VPN etc.

It should be noted that; described subscriber equipment, the network equipment and network etc. are only citing; other computer equipments that are existing or that may occur from now on or network, as being applicable to the present invention, within also should being included in scope, and are contained in this with way of reference.

Method (some of them are illustrated by process flow diagram) discussed below is implemented by hardware, software, firmware, middleware, microcode, hardware description language or its combination in any.When implementing by software, firmware, middleware or microcode, program code or code segment in order to implement necessary task can be stored in machine or computer-readable medium (such as storage medium).(one or more) processor can implement necessary task.

Concrete structure disclosed herein and function detail are only representational, and are the objects for describing exemplary embodiment of the present invention.But the present invention can carry out specific implementation by many replacement forms, and should not be construed as only being limited to the embodiments set forth herein.

Should be understood that, although may have been used term " first ", " second " etc. here to describe unit, these unit should not limit by these terms.These terms are used to be only used to a unit and another unit to distinguish.For example, when not deviating from the scope of exemplary embodiment, first module can be called as second unit, and second unit can be called as first module similarly.Here used term "and/or" comprise one of them or more any and all combinations of listed associated item.

Should be understood that, when a unit is called as " connection " or " coupling " to another unit, it can directly connect or be coupled to another unit described, or can there is temporary location.On the other hand, " when being directly connected " or " directly coupled " to another unit, then there is not temporary location when a unit is called as.Should explain in a comparable manner the relation be used between description unit other words (such as " and be in ... between " compared to " and be directly in ... between ", " with ... contiguous " compared to " and with ... be directly close to " etc.).

Here used term is only used to describe specific embodiment and be not intended to limit exemplary embodiment.Unless context refers else clearly, otherwise singulative used here " ", " one " are also intended to comprise plural number.It is to be further understood that, the existence of the feature that term used here " comprises " and/or " comprising " specifies to state, integer, step, operation, unit and/or assembly, and do not get rid of and there is or add other features one or more, integer, step, operation, unit, assembly and/or its combination.

Also it should be mentioned that and to replace in implementation at some, the function/action mentioned can according to being different from occurring in sequence of indicating in accompanying drawing.For example, depend on involved function/action, in fact the two width figure in succession illustrated can perform simultaneously or sometimes can perform according to contrary order substantially.

Below in conjunction with accompanying drawing, the present invention is described in further detail.

Embodiment one, flow layer method to set up based on flux experiment.

Fig. 1 is the process flow diagram of the flow layer method to set up based on flux experiment of the present embodiment, and the method shown in Fig. 1 mainly comprises step S100, step S110 and step S120.Method described in the present embodiment is normally performed in computer equipment, and preferably, the method described in the present embodiment can perform in desk-top computer, mobile computer, server or cloud equipment.Below each step in Fig. 1 is described respectively.

The mark of S100, acquisition network calling party.

Concrete, the mark of the network calling party in the present embodiment can go out a network calling party by unique identification, is namely man-to-man relation between network calling party and mark.The mark of network calling party can be specially the cookie etc. of network calling party.Certainly, the mark of network calling party also can be registered user's title etc. of network calling party, the concrete manifestation form of the present embodiment not mark of limiting network calling party.

Exemplarily, the present embodiment when detecting that network calling party accesses predetermined website first, can obtain the mark of network calling party, and performing follow-up step S110 and step S120; As when need for certain Search Results represent position adjust, the method of the present embodiment starts, from current time, detect the network calling party of access websites first, and obtain the cookie of the network calling party of the access websites first detected.

S110, according to the network calling party equal number principle in each Subscriber Unit, the mark of network calling party to be arranged in a Subscriber Unit of predetermined amount of flow layer.

Concrete, be previously provided with in the present embodiment n+1 (n be greater than 1 integer) individual flow layer, include n sampling unit in each flow layer, and each sampling unit includes n Subscriber Unit; Thus include n × n Subscriber Unit in any one flow layer in the present embodiment.

Predetermined amount of flow layer in the present embodiment can be any one the flow layer in all flow layers, and under normal conditions, the predetermined amount of flow layer in the present embodiment is the first flow layer (as the 0th flow layer) in all flow layers.

The network calling party quantity that the object of the network calling party equal number principle in each Subscriber Unit in the present embodiment is to make each Subscriber Unit of predetermined amount of flow layer to comprise is substantially identical, and then the network calling party data that the present embodiment can make each Subscriber Unit of all flow layers comprise are substantially identical.

Exemplarily, the present embodiment can utilize Hash operation to realize the network calling party equal number principle in above-mentioned each Subscriber Unit; A concrete example, when often getting the mark of a network calling party, mark for this network calling party carries out Hash operation, then, the mark of network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer by the result according to Hash operation, as carried out delivery to the result of Hash operation, the numerical value obtained according to delivery determines the Subscriber Unit belonging to this network calling party, and the mark of this network calling party is arranged in this Subscriber Unit.

Certainly, the present embodiment also can utilize other modes to realize the network calling party equal number principle in above-mentioned each Subscriber Unit, as the mark of the network calling party got is distributed in corresponding Subscriber Unit by number order according to Subscriber Unit each in predetermined amount of flow layer in turn.In addition, the mark of network calling party to be arranged in a Subscriber Unit of predetermined amount of flow layer in the present embodiment can for arranging the corresponding relation between the mark of network calling party and the mark of Subscriber Unit, and the mark of Subscriber Unit can be the numbering etc. of Subscriber Unit.The concrete manifestation form that the present embodiment does not limit the network calling party equal number principle in each Subscriber Unit and the specific implementation etc. the mark of network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer.

S120, according to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, the mark of network calling party to be arranged at respectively in a Subscriber Unit of other each flow layers.

Concrete, be provided with n+1 flow layer in the present embodiment, each flow layer is made up of n sampling unit, and each sampling unit is made up of n Subscriber Unit, and usually can be provided with multiple network calling party in each Subscriber Unit.Sampling unit in the present embodiment chooses the least unit of the network calling party participating in flux experiment, Subscriber Unit in the present embodiment is the least unit of composition sampling unit, simultaneously, Subscriber Unit is also the least unit forming flow layer, and the network calling party in the present embodiment is the least unit of composition Subscriber Unit.

N Subscriber Unit in any one sampling unit in each flow layer in the present embodiment can be evenly distributed in each sampling unit in other n flow layer, namely for any one sampling unit, its all Subscriber Units comprised are broken up in each sampling unit of other each flow layers uniformly, like this, include all Subscriber Units in each flow layer, and there will not be the situation that same Subscriber Unit repeats in a flow layer.

Exemplarily, the example that of the present embodiment is concrete: as shown in Figure 2: be provided with 10000 flow layers, 9999 sampling units (numeral sampling unit in Fig. 2) are provided with in each flow layer, and when including 9999 Subscriber Units (Subscriber Unit in Fig. 2 in not shown sampling unit) in each sampling unit, in the 1st sampling unit in any one flow layer, 9999 Subscriber Units are evenly distributed in 9999 sampling units in other any one flow layers, in the 2nd sampling unit in any one flow layer, 9999 Subscriber Units are evenly distributed in 9999 sampling units in other any one flow layers, by that analogy, in the 9999th sampling unit in any one flow layer, 9999 Subscriber Units are evenly distributed in 9999 sampling units in other any one flow layers.

From the description of above-mentioned object lesson, the all-network calling party being arranged in same Subscriber Unit in the present embodiment is as a whole with the Subscriber Unit at its place and appear at some sampling units of different flow layers.

The present embodiment n Subscriber Unit achieved by the corresponding relation between the Subscriber Unit in arbitrary flow layer of pre-setting and the Subscriber Unit in other n flow layer in any one sampling unit in each flow layer above-mentioned can be evenly distributed in this purpose in each sampling unit in other n flow layer.

Exemplarily, be set with n+1 flow layer (namely the 0th flow layer, the 1st flow layer ... and the n-th flow layer), each flow layer comprise n sampling unit (namely the 1st sampling unit, the 2nd sampling unit ... and n-th sampling unit) and each sampling unit comprises n Subscriber Unit when, the Subscriber Unit in arbitrary flow layer that the present embodiment pre-sets and the corresponding relation between the Subscriber Unit in other n flow layer can be described to the form of following corresponding relation A, corresponding relation B and corresponding relation C:

The 1st Subscriber Unit in each sampling unit in n corresponding 1st flow layer to the n-th flow layer of Subscriber Unit difference in 1st sampling unit of corresponding relation A, the 0th flow layer.

The concrete example of of this corresponding relation A is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation A can be expressed as following formula (1):

S_{n \times i}^{j} = S_{i}^{0};

Formula (1)

In formula (1), i=0,1,2 ... n-1, and j=1,2,3 ... n, and n be greater than 1 integer;

Above-mentioned formula (1) can be described as following form:

In the 1st Subscriber Unit in the 1st sampling unit of the 1st Subscriber Unit in 1st sampling unit of the 0th flow layer respectively in corresponding 1st flow layer, the 1st sampling unit in the 2nd flow layer the 1st Subscriber Unit ... and n-th the 1st Subscriber Unit in the 1st sampling unit in flow layer;

In the 1st Subscriber Unit in the 2nd sampling unit of the 2nd Subscriber Unit in 1st sampling unit of the 0th flow layer respectively in corresponding 1st flow layer, the 2nd sampling unit in the 2nd flow layer the 1st Subscriber Unit ... and n-th the 1st Subscriber Unit in the 2nd sampling unit in flow layer;

……

In the 1st Subscriber Unit in nth user unit in 1st sampling unit of the 0th flow layer the n-th sampling unit respectively in corresponding 1st flow layer, the n-th sampling unit in the 2nd flow layer the 1st Subscriber Unit ... and n-th the 1st Subscriber Unit in the n-th sampling unit in flow layer.

The 2nd Subscriber Unit in the 1st sampling unit in corresponding 1st flow layer to the n-th flow layer of n × (n-1) in 2nd sampling unit to the n-th sampling unit of corresponding relation B, the 0th flow layer individual Subscriber Unit difference is to nth user unit.

The concrete example of of this corresponding relation B is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation B can be expressed as following formula (2):

S_{i}^{j} = S_{n \times i + (j - 1)}^{0}

Formula (2)

In formula (2), i=1,2 ... n-1, and j=1,2,3 ... n, and n be greater than 1 integer;

Above-mentioned formula (2) can be described as following form:

The 2nd Subscriber Unit in the 1st sampling unit in the corresponding 1st flow layer of the 1st Subscriber Unit in 2nd sampling unit of the 0th flow layer;

The 3rd Subscriber Unit in the 1st sampling unit in the corresponding 1st flow layer of the 1st Subscriber Unit in 3rd sampling unit of the 0th flow layer;

.......

Nth user unit in the 1st sampling unit in the corresponding 1st flow layer of the 1st Subscriber Unit in n-th sampling unit of the 0th flow layer;

……

The 2nd Subscriber Unit in the 1st sampling unit in the corresponding 2nd flow layer of the 2nd Subscriber Unit in 2nd sampling unit of the 0th flow layer;

The 3rd Subscriber Unit in the 1st sampling unit in the corresponding 2nd flow layer of the 2nd Subscriber Unit in 3rd sampling unit of the 0th flow layer;

.......

Nth user unit in the 1st sampling unit in the corresponding 2nd flow layer of the 2nd Subscriber Unit in n-th sampling unit of the 0th flow layer;

……

The 2nd Subscriber Unit in the 1st sampling unit in the corresponding n-th flow layer of nth user unit in 2nd sampling unit of the 0th flow layer;

The 3rd Subscriber Unit in the 1st sampling unit in the corresponding n-th flow layer of nth user unit in 3rd sampling unit of the 0th flow layer;

.......

Nth user unit in the 1st sampling unit in the corresponding n-th flow layer of nth user unit in n-th sampling unit of the 0th flow layer.

The 2nd Subscriber Unit in 1st sampling unit of corresponding relation C, the 1st flow layer to the n-th flow layer is to the 2nd Subscriber Unit in the 2nd sampling unit to the n-th sampling unit in nth user unit respectively corresponding 1st flow layer to the n-th flow layer to nth user unit.

Above-mentioned corresponding relation C also can be described to the corresponding relation between the 2nd Subscriber Unit in the 0th flow layer to the 2nd Subscriber Unit to nth user unit in the 2nd sampling unit to the n-th sampling unit in nth user unit and the 1st flow layer to the n-th flow layer.That is, the corresponding relation between the 2nd Subscriber Unit in the 0th flow layer that in fact embodies of above-mentioned corresponding relation C to the 2nd Subscriber Unit to nth user unit in the 2nd sampling unit to the n-th sampling unit in nth user unit and the 1st flow layer to the n-th flow layer.

The concrete example of of above-mentioned corresponding relation C is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation C can be expressed as following formula (3):

i f (S_{n \times (i - 1) + k}^{j} + k) \leq S_{n \times k + n - 1}^{0}

S_{n \times i + k}^{j} = S_{n \times (i - 1) + k}^{j} + k

else

S_{n \times i + k}^{j} = S_{n \times k}^{0} + (S_{n \times (i - 1) + k}^{j} + k) - S_{n \times k + n - 1}^{0} - 1

Formula (3)

In above-mentioned formula (3), k=1,2 ... n-1; I=1,2 ... n-1; J=1,2,3 ... n, and n be greater than 1 integer.

Exemplarily, below in conjunction with Fig. 3, to utilize above-mentioned corresponding relation A, corresponding relation B and corresponding relation C in n the sampling unit making n Subscriber Unit in any one sampling unit in each flow layer be evenly distributed in other arbitrary flow layers more specifically example be described.

In Fig. 3, when n=5, be provided with 6 flow layers in the present embodiment, each flow layer is made up of 5 sampling units, and each sampling unit is made up of 5 Subscriber Units, thus each flow layer includes 25 Subscriber Units.

Setting the 0th flow layer (" the 0th layer " namely in Fig. 3) is predetermined amount of flow layer, for the Subscriber Unit in the 0th flow layer, according to order number consecutively from left to right, i.e. digital 1-25 in the 1st row of Fig. 3 represents that the 1st Subscriber Unit in the 0th flow layer is to the 25th Subscriber Unit.

After setting the in the 0th flow layer the 1st Subscriber Unit to the 25th Subscriber Unit, each Subscriber Unit in the 0th flow layer can be determined at the 1st flow layer (" the 1st layer " namely in Fig. 3) to the distribution in the 5th flow layer (" the 5th layer " namely in Fig. 3), the distribution situation of the numeral for representing Subscriber Unit in " the 1st layer ", " the 2nd layer ", " the 3rd layer " and " the 4th layer " in concrete Fig. 3 according to above-mentioned corresponding relation A, corresponding relation B and corresponding relation C.

It should be noted that, above-mentioned corresponding relation A, corresponding relation B, corresponding relation C and Fig. 3 is only a concrete example, the present embodiment can adopt other concrete corresponding relations in n the sampling unit making n Subscriber Unit in any one sampling unit in each flow layer and be evenly distributed in other arbitrary flow layers completely, such as, the Subscriber Unit being numbered " 1 " in the 0th flow layer in Fig. 3 also can be located at the position (as being arranged in the position being numbered the Subscriber Unit of " 6 " or " 11 " or " 16 " or " 21 " of the 1st flow layer of Fig. 3) of other Subscriber Units except first Subscriber Unit in the 1st sampling unit in the 1st flow layer, again such as, the present embodiment can using the 5th flow layer as predetermined amount of flow layer, and for the Subscriber Unit in the 5th flow layer according to order number consecutively from left to right, then, each Subscriber Unit in the 5th flow layer is determined at the 0th flow layer to the distribution in the 4th flow layer according to the corresponding relation preset.The present embodiment does not limit the concrete manifestation form of the corresponding relation between the Subscriber Unit of predetermined amount of flow layer and the Subscriber Unit of other n flow layer, as long as n Subscriber Unit in this corresponding relation can ensure in each flow layer any one sampling unit is evenly distributed in n sampling unit in other arbitrary flow layers.

The present embodiment in the Subscriber Unit mark of network calling party being arranged at predetermined amount of flow layer after, the position be identified in any one flow layer of this network calling party can be determined according to the corresponding relation pre-set, thus the mark of this network calling party is arranged in the corresponding Subscriber Unit of other each flow layers respectively.

From the description of above-described embodiment one, n+1 the flow layer pre-set in the present invention can be evenly distributed in each sampling unit in other n flow layer due to n Subscriber Unit in any one sampling unit in each flow layer, therefore, be all completely orthogonal between any two flow layers; Thus when choosing sampling unit and arranging experimental group and control group, even if exist a network calling party appear at simultaneously different flow experiment experimental group/control group in phenomenon, but be identical due to experimental group with the accounting that there is such network calling party in its corresponding control group, thus the present embodiment can avoid the different harmful effect to the assessment result of flux experiment of the accounting of such network calling party.

The implementation method of embodiment two, flux experiment.

Fig. 2 is the process flow diagram of the implementation method of the flux experiment of the present embodiment, and the method shown in Fig. 2 mainly comprises step S200, step S210 and step S220.Method described in the present embodiment is normally performed in computer equipment, and preferably, the method described in the present embodiment can perform in desk-top computer, mobile computer, server or cloud equipment.Below each step in Fig. 2 is described respectively.

S200, from n+1 flow layer, choose a flow layer.

Concrete, be previously provided with in the present embodiment n+1 (n be greater than 1 integer) individual flow layer, include n sampling unit in each flow layer, and each sampling unit includes n Subscriber Unit; Thus include n × n Subscriber Unit in any one flow layer in the present embodiment.The network calling party data that each Subscriber Unit of all flow layers in the present embodiment comprises are substantially identical.

The present embodiment can use various ways to choose a flow layer from n+1 flow layer, and as produced the random number between a 0-n at random, this random number is the number of plies of the flow layer be selected; For another example from all flow layers, a flow layer is chosen according to polling mode order.The present embodiment does not limit the specific implementation choosing a flow layer from n+1 flow layer.

S210, the sampling unit selecting K+h different from the flow layer chosen, wherein K the corresponding experimental group of sampling unit, and h sampling unit correspondence control group.

Concrete, the present embodiment can use various ways from a flow layer of selected taking-up, select the individual different sampling unit of K+h, and as produced K+h random number between 1-n at random, this K+h random number is the serial number of the sampling unit be selected; For another example from all sampling units a flow layer of selected taking-up, K+h sampling unit is chosen according to the serial number of sampling unit.K+h in the present embodiment is less than or equal to n, and the value of K with h can be identical, also can not be identical.Under normal circumstances, because flux experiment needs experimental group equal with the sample size of control group, therefore, the value of K with h is normally identical, but have in the application scenarios of particular demands at some, experimental group in flux experiment and the sample size of control group can be unequal, and now, the value of K with h is not identical.The present embodiment does not limit the specific implementation selecting the individual different sampling unit of K+h from the flow layer chosen.

The present embodiment is determining the corresponding experimental group of K sampling unit in the individual different sampling unit of K+h, and during h sampling unit correspondence control group, usually can distribute different sampling marks respectively for experimental group and control group; The sampling mark of experimental group and the sampling mark of control group all can indicate the concrete flux experiment that network calling party participates in, and the flux experiment namely can determining that network calling party participates according to mark of sampling is the flux experiment for the change particular content in the page.

The present embodiment can after distributing sampling mark for experimental group and control group, set up sampling mark, the mark of Subscriber Unit belonging to network calling party and the corresponding relation between mark (cookie as network calling party) this three of network calling party, thus can determine according to this corresponding relation the flux experiment for the change particular content in the page that Subscriber Unit belonging to network calling party and this Subscriber Unit participate in.

S220, to send for the page of flux experiment to this network calling party according to the network access request of the network calling party belonging to experimental group or control group.

Concrete, it is the network access request initiated for pre-determined link (link corresponding to the contrast page namely in flux experiment) in the network access request of network calling party, and this network calling party is when belonging to experimental group in this flux experiment or control group, this network calling party hits the flux experiment corresponding to it, and namely network calling party triggers the flux experiment corresponding to it, the present embodiment can according to all sampling marks be identified at corresponding to Subscriber Unit belonging to the mark of searching this network calling party in above-mentioned corresponding relation and this Subscriber Unit of this network calling party, then, all sampling marks corresponding to Subscriber Unit belonging to the mark of network calling party determine the page that send to this network calling party, the experiment page arranged as sent the particular content based on certain in the page or multiple change to this network calling party or the contrast page etc. corresponding with this experiment page, and send the corresponding page to this network calling party.

It should be noted that, Subscriber Unit belonging to the mark of network calling party identifies there being multiple sampling, and multiple sampling mark be for the multiple change in a page for different flow experiment set by sampling mark when, the present embodiment can be the experiment page of the content change included in each flux experiment to the experiment page that the network calling party belonging to experimental group issues, and can be the contrast page of the content change part included in other flux experiment to the contrast page that the network calling party belonging to control group issues.

After having issued the corresponding page to network calling party, the present embodiment can get the log information of this network calling party based on this page, as got the fusion log information etc. of record network calling party based on the complete behavior sequence of this page; The present embodiment can when receiving log information, according to above-mentioned corresponding relation determine this network calling party corresponding sampling mark, and arrange in the log information of this network calling party this belonging to network calling party the sampling corresponding to Subscriber Unit mark.Because a Subscriber Unit may take part in the multiple flux experiment for the change of webpage different content simultaneously, thus a Subscriber Unit corresponding multiple sampling may identify (as SID), in the case, corresponding multiple sampling mark can be set respectively in all log informations belonging to each network calling party under this Subscriber Unit.

Exemplarily, the log information of the network calling party belonging to same Subscriber Unit can be stored in same daily record storage block by the present embodiment, thus in the change needed for the particular content of webpage to flux experiment in network calling party in corresponding experimental group and control group when carrying out user behavior analysis, first can determine the sampling mark of experimental group and the sampling mark of control group, then, the all Subscriber Units involved by the sampling mark of corresponding experimental group and the sampling mark of corresponding control group are determined according to the above-mentioned corresponding relation pre-set, afterwards, the daily record storage block that each Subscriber Unit is corresponding respectively can be determined, thus the sampling of the sampling mark experimentally organized and control group mark is filtered the log information in corresponding daily record storage block, the log information belonging to the network calling party of experimental group corresponding to sampling mark of corresponding experimental group and the log information belonging to the network calling party of control group corresponding to sampling mark of corresponding control group can be got fast, finally make the present embodiment can carry out user behavior analysis according to the log information of the network calling party belonging to experimental group with the log information of the network calling party belonging to control group.The specific implementation of the present embodiment not limited subscriber behavioural analysis.

Seen from the above description, the present embodiment only needs to carry out filtering to the log information in the daily record storage block corresponding to corresponding Subscriber Unit to obtain required log information, avoid the phenomenon filtering to obtain required log information to the log information of all-network calling party, thus the present embodiment not only can save a large amount of computational resources, but also the evaluation time of flux experiment can be shortened to a great extent, and then improve the efficiency of flux experiment, and reduce flux experiment realize cost.The concrete statistics of of the present embodiment is: from a certain flow layer, extracting the sampling unit of 1% as experimental group, and in the sampling unit of another extraction 1% flux experiment as a control group, when daily record filtration treatment is carried out on basis based on existing daily record storage mode, the daily record amount of filtering every day is greatly about 4.5T (4608G) left and right; And when carrying out daily record filtration treatment on the basis based on daily record storage mode of the present invention, the daily record amount that every day filters is greatly about about 46G, the daily record of the present embodiment filter that the computational resource consumed is only that the computational resource consumed is filtered in daily record of the prior art 1%, daily record filter needed for time also significantly shortened thereupon.

Embodiment three, flow layer setting device based on flux experiment.

The flow layer setting device based on flux experiment in the present embodiment is arranged in computer equipment usually, preferably, the flow layer setting device based on flux experiment described in the present embodiment can be arranged in desk-top computer, mobile computer, server or cloud equipment.The primary structure of the flow layer setting device based on flux experiment of the present embodiment as shown in Figure 5.

Referring to specific embodiment, the flow layer setting device based on flux experiment is described.

In Fig. 5, the flow layer setting device based on flux experiment of the present embodiment mainly comprises: for obtaining the device (following abbreviation " user ID acquisition device 300 ") of the mark of network calling party, for according to the network calling party equal number principle in each Subscriber Unit the mark of network calling party being arranged at device (following abbreviation " the first setting device 310 ") in a Subscriber Unit of predetermined amount of flow layer and for according to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, the mark of described network calling party is arranged at respectively the device (following abbreviation " the second setting device 320 ") in a Subscriber Unit of other each flow layers.

User ID acquisition device 300 is mainly used in the mark obtaining network calling party.

Concrete, the mark of the network calling party that user ID acquisition device 300 obtains can go out a network calling party by unique identification, is namely man-to-man relation between network calling party and mark.The mark of the network calling party that user ID acquisition device 300 obtains can be specially the cookie etc. of network calling party.Certainly, the mark of the network calling party that user ID acquisition device 300 obtains also can be registered user's title etc. of network calling party, and the present embodiment not limited subscriber identifies the concrete manifestation form of the mark of the network calling party that acquisition device 300 obtains.

Exemplarily, user ID acquisition device 300 when detecting that network calling party accesses predetermined website first, can obtain the mark of network calling party, thus making the first setting device 310 and the second setting device 320 perform corresponding operation; As when need for certain Search Results represent position adjust, user ID acquisition device 300 detects the network calling party of access websites first from current time, and obtains the cookie of the network calling party of the access websites first detected.

First setting device 310 is mainly used in the mark of network calling party being arranged in a Subscriber Unit of predetermined amount of flow layer according to the network calling party equal number principle in each Subscriber Unit.

Exemplarily, first setting device 310 can comprise: for carrying out device (the following abbreviation " Hash operation device " of Hash operation to the mark of network calling party, not shown) and the device (following abbreviation " the 3rd setting device ", not shown) that for the result of the Hash operation according to Hash operation device the mark of network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer.

Hash operation device is mainly used in carrying out Hash operation to the mark of network calling party.

3rd setting device is mainly used in the mark of described network calling party being arranged in a Subscriber Unit of predetermined amount of flow layer according to the result of Hash operation.

Concrete, be previously provided with in the present embodiment n+1 (n be greater than 1 integer) individual flow layer, include n sampling unit in each flow layer, and each sampling unit includes n Subscriber Unit; Thus include n × n Subscriber Unit in any one flow layer in the present embodiment.Predetermined amount of flow layer in the present embodiment can be any one the flow layer in all flow layers, and under normal conditions, the predetermined amount of flow layer in the present embodiment is the first flow layer (as the 0th flow layer) in all flow layers.

The network calling party quantity that the object of the network calling party equal number principle in each Subscriber Unit that the first setting device 310 uses is to make each Subscriber Unit of predetermined amount of flow layer to comprise is substantially identical, and then the network calling party data that the first setting device 310 can make each Subscriber Unit of all flow layers comprise are substantially identical.

Exemplarily, the first setting device 310 (as Hash operation device) can utilize Hash operation to realize the network calling party equal number principle in above-mentioned each Subscriber Unit, a concrete example, when user ID acquisition device 300 often gets the mark of a network calling party, first setting device 310 (as Hash operation device) carries out Hash operation for the mark of this network calling party, then, the mark of network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer according to the result of Hash operation by the first setting device 310 (as the 3rd setting device), as the first setting device 310 (as the 3rd setting device) carries out delivery to the result of Hash operation, first setting device 310 (as the 3rd setting device) according to delivery the numerical value that obtains determine belonging to this network calling party Subscriber Unit, and the mark of this network calling party is arranged in this Subscriber Unit.

Certainly, first setting device 310 also can utilize other modes to realize the network calling party equal number principle in above-mentioned each Subscriber Unit, as the mark of the network calling party got is distributed in corresponding Subscriber Unit by the first setting device 310 number order according to Subscriber Unit each in predetermined amount of flow layer in turn.In addition, it can be the corresponding relation that the first setting device 310 arranges between the mark of network calling party and the mark of Subscriber Unit that the mark of network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer by the first setting device 310, and the mark of Subscriber Unit can be the numbering etc. of Subscriber Unit.The present embodiment does not limit the concrete manifestation form of the network calling party equal number principle in each Subscriber Unit that the first setting device 310 uses and the first setting device 310 and the mark of network calling party is arranged at specific implementation etc. in a Subscriber Unit of predetermined amount of flow layer.

Second setting device 320 is mainly used in, according to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, the mark of network calling party being arranged at respectively in a Subscriber Unit of other each flow layers.

Concrete; the present embodiment be provided with n+1 flow layer based in the flow layer setting device of flux experiment; each flow layer is made up of n sampling unit; each sampling unit is made up of n Subscriber Unit, usually can be provided with multiple network calling party in each Subscriber Unit.Sampling unit in the present embodiment chooses the least unit of the network calling party participating in flux experiment, Subscriber Unit in the present embodiment is the least unit of composition sampling unit, simultaneously, Subscriber Unit is also the least unit forming flow layer, and the network calling party in the present embodiment is the least unit of composition Subscriber Unit.

N Subscriber Unit in any one sampling unit in each flow layer in the present embodiment can be evenly distributed in each sampling unit in other n flow layer, namely for any one sampling unit, its all Subscriber Units comprised are broken up in each sampling unit of other each flow layers uniformly, like this, include all Subscriber Units in each flow layer, and there will not be the situation that same Subscriber Unit repeats in a flow layer.The object lesson of flow layer refers to the description in above-described embodiment one, is not repeated.

Second setting device 320 n the Subscriber Unit achieved by the corresponding relation between the Subscriber Unit in arbitrary flow layer of pre-setting and the Subscriber Unit in other n flow layer in any one sampling unit in each flow layer above-mentioned can be distributed in this purpose in each sampling unit in any one flow layer uniformly respectively.

Exemplarily, be set with n+1 flow layer (i.e. the 0th flow layer, 1st flow layer, and the n-th flow layer), each flow layer comprises n sampling unit (i.e. the 1st sampling unit, 2nd sampling unit, and n-th sampling unit), and each sampling unit is when comprising n Subscriber Unit, Subscriber Unit in the arbitrary flow layer pre-set in second setting device 320 and the corresponding relation between the Subscriber Unit in other n flow layer can be described to following corresponding relation A, the form of corresponding relation B and corresponding relation C:

The concrete example of of this corresponding relation A is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation A can be expressed as the form of above-mentioned formula (1).The specific descriptions of formula (1) refer to the record in above-described embodiment one, are not repeated.

The concrete example of of this corresponding relation B is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation B can be expressed as the form of above-mentioned formula (2).The specific descriptions of formula (2) refer to the record in above-described embodiment one, are not repeated.

Above-mentioned corresponding relation C also can be described to the corresponding relation between the 2nd Subscriber Unit in the 0th flow layer to the 2nd Subscriber Unit to nth user unit in the 2nd sampling unit to the n-th sampling unit in nth user unit and the 1st flow layer to the n-th flow layer.That is, the corresponding relation between 2, the ground Subscriber Unit in the 0th flow layer that in fact embodies of above-mentioned corresponding relation C to the 2nd Subscriber Unit to nth user unit in the 2nd sampling unit to the n-th sampling unit in nth user unit and the 1st flow layer to the n-th flow layer.

The concrete example of of above-mentioned corresponding relation C is: be labeled as successively by all Subscriber Units in the 0th flow layer: (representing the 1st Subscriber Unit in the 0th flow layer), (representing the 2nd Subscriber Unit in the 0th flow layer) ..., (represent n-th in the 0th flow layer ²individual Subscriber Unit, last Subscriber Unit namely in the 0th flow layer) when, corresponding relation C can be expressed as the form of above-mentioned formula (3).The specific descriptions of formula (3) refer to the record in above-described embodiment one, are not repeated.

Second setting device 320 in the Subscriber Unit mark of network calling party being arranged at predetermined amount of flow layer after, can determine the position be identified in any one flow layer of this network calling party according to the corresponding relation pre-set, thus the mark of this network calling party is arranged in the corresponding Subscriber Unit of other each flow layers by the second setting device 320 respectively.

The implement device of embodiment four, flux experiment.

The implement device of the flux experiment in the present embodiment is arranged in computer equipment usually, and preferably, the implement device of the flux experiment described in the present embodiment can be arranged in desk-top computer, mobile computer, server or cloud equipment.The primary structure of the implement device of the flux experiment of the present embodiment as shown in Figure 6.

Be described referring to the implement device of specific embodiment to flux experiment.

In Fig. 6, the implement device of the flux experiment of the present embodiment mainly comprises: for choosing the device (hereinafter referred to as " the first selecting device 400 ") of a flow layer from n+1 flow layer, for selecting the device of the individual different sampling unit of K+h from the described flow layer chosen, wherein K the corresponding experimental group of sampling unit, the corresponding control group (hereinafter referred to as " the second selecting device 410 ") of h sampling unit and for sending device (hereinafter referred to as " page dispensing device 420 ") for the page of flux experiment according to the network access request of the network calling party belonging to experimental group or control group to described network calling party in addition.

First selecting device 400 is mainly used in choosing a flow layer from n+1 flow layer.

First selecting device 400 can use various ways to realize chooses a flow layer from n+1 flow layer, and as the first selecting device 400 produces the random number between a 0-n at random, this random number is the number of plies of the flow layer be selected; For another example the first selecting device 400 chooses a flow layer according to polling mode order from all flow layers.The present embodiment does not limit the specific implementation that the first selecting device 400 chooses a flow layer from n+1 flow layer.

Second selecting device 410 is mainly used in selecting the individual different sampling unit of K+h the flow layer chosen from the first selecting device 400, wherein K the corresponding experimental group of sampling unit, in addition h the corresponding control group of sampling unit.

Concrete, second selecting device 410 can use various ways from a flow layer of selected taking-up, select the individual different sampling unit of K+h, as the second selecting device 410 produces K+h random number between 1-n at random, this K+h random number is the serial number of the sampling unit be selected; For another example the second selecting device 410 chooses K+h sampling unit according to the serial number of sampling unit from all sampling units a flow layer of selected taking-up.K+h in the present embodiment such as to be less than at the n, and the value of K with h can be identical, also can not be identical.Under normal circumstances, because flux experiment needs experimental group equal with the sample size of control group, therefore, the value of K with h is normally identical, but have in the application scenarios of particular demands at some, experimental group in flux experiment and the sample size of control group can be unequal, and now, the value of K with h is not identical.The present embodiment does not limit the specific implementation that the second selecting device 410 selects the individual different sampling unit of K+h from the flow layer chosen.

Exemplarily, the implement device of the flux experiment of the present embodiment can also optionally comprise: for distributing the device of different sampling marks respectively for experimental group and control group (hereinafter referred to as " sampling mark distributor ", not shown) and device (hereinafter referred to as " corresponding relation apparatus for establishing ", not shown) for setting up the corresponding relation between sampling mark, the mark of Subscriber Unit belonging to network calling party and the mark of network calling party.

Sampling mark distributor is mainly used in as experimental group and control group distribute different sampling marks respectively.

Corresponding relation apparatus for establishing is mainly used in setting up sampling mark, the mark of Subscriber Unit belonging to network calling party and the corresponding relation between the mark of network calling party.

Concrete, the corresponding experimental group of K sampling unit in the individual different sampling unit of K+h is determined at the second selecting device 410, and during other h sampling unit correspondence control group, sampling mark distributor can identify for experimental group and control group distribute different sampling respectively usually; The sampling mark of experimental group and the sampling mark of control group all can indicate the concrete flux experiment that network calling party participates in, and the flux experiment namely can determining that network calling party participates according to mark of sampling is the flux experiment for the change particular content in the page.

Corresponding relation apparatus for establishing can be after experimental group and control group distribute sampling mark at sampling mark distributor, set up sampling mark, the mark of Subscriber Unit belonging to network calling party and the corresponding relation between mark (cookie as network calling party) this three of network calling party, thus can determine according to this corresponding relation the flux experiment for the change particular content in the page that Subscriber Unit belonging to network calling party and this Subscriber Unit participate in.

Page dispensing device 420 is mainly used in sending for the page of flux experiment to network calling party according to the network access request of the network calling party belonging to experimental group or control group.

Concrete, it is the network access request initiated for pre-determined link (link corresponding to the contrast page namely in flux experiment) in the network access request of network calling party, and this network calling party is when belonging to experimental group in this flux experiment or control group, this network calling party hits the flux experiment corresponding to it, and namely network calling party triggers the flux experiment corresponding to it, page dispensing device 420 can according to all sampling marks be identified at corresponding to Subscriber Unit belonging to the mark of searching this network calling party in above-mentioned corresponding relation and this Subscriber Unit of this network calling party, then, all sampling marks corresponding to the Subscriber Unit of page dispensing device 420 belonging to the mark of network calling party determine the page that send to this network calling party, the experiment page arranged as sent the particular content based on certain in the page or multiple change to this network calling party or the contrast page etc. corresponding with this experiment page, page dispensing device 420 sends the corresponding page to this network calling party.

It should be noted that, Subscriber Unit belonging to the mark of network calling party identifies there being multiple sampling, and multiple sampling mark be for the multiple change in a page for different flow experiment set by sampling mark when, page dispensing device 420 can be the experiment page of the content change included in each flux experiment to the experiment page that the network calling party belonging to experimental group issues, and can be the contrast page of the content change part included in other flux experiment to the contrast page that the network calling party belonging to control group issues.

Exemplarily, the implement device of the flux experiment of the present embodiment can also optionally comprise: for according to the device of sampling mark corresponding to above-mentioned corresponding relation determination network calling party (hereinafter referred to as " sampling mark determining device ", not shown), for arranging the device of the sampling mark of above-mentioned correspondence (hereinafter referred to as " sampling mark setting device " in the log information of network calling party, not shown) and for the log information of the network calling party belonging to same Subscriber Unit being stored in device in same daily record storage block (hereinafter referred to as " log information memory storage ", not shown).

Sampling mark determining device is mainly used in the sampling mark corresponding according to above-mentioned corresponding relation determination network calling party.

Sampling mark setting device is mainly used in the sampling mark arranging above-mentioned correspondence in the log information of network calling party.

Log information memory storage is mainly used in the log information of the network calling party belonging to same Subscriber Unit to be stored in same daily record storage block.

Concrete, after page dispensing device 420 has issued the corresponding page to network calling party, the present embodiment can get the log information of this network calling party based on this page, as received the fusion log information etc. of record network calling party based on the complete behavior sequence of this page; The present embodiment can when receiving log information, determine that sampling corresponding to this network calling party identifies by sampling mark determining device according to above-mentioned corresponding relation, and the sampling corresponding to the Subscriber Unit arranging belonging to this network calling party in the log information of this network calling party by sampling mark setting device identifies.Because a Subscriber Unit may take part in the multiple flux experiment for the change of webpage different content simultaneously, thus a Subscriber Unit corresponding multiple sampling may identify (as SID), in the case, sampling mark setting device can arrange corresponding multiple sampling mark respectively in all log informations belonging to each network calling party under this Subscriber Unit.The log information including sampling mark belonging to the network calling party of same Subscriber Unit can be stored in same daily record storage block by log information memory storage.

The implement device of the flux experiment of the present embodiment can also optionally comprise: for determine the sampling mark of the experimental group needing to carry out user behavior analysis and control group according to above-mentioned corresponding relation sampling mark corresponding to the device of each Subscriber Unit (hereinafter referred to as " Subscriber Unit determining device ", not shown), for obtain from daily record storage block corresponding to each Subscriber Unit belong to the log information needing the log information of the network calling party of the experimental group of carrying out user behavior analysis and belong to the network calling party needing the control group carrying out user behavior analysis device (hereinafter referred to as " log information acquisition device ", not shown) and for needing the log information of the network calling party of the experimental group of carrying out user behavior analysis and belong to need the log information of the network calling party of the control group carrying out user behavior analysis to carry out the device of user behavior analysis (hereinafter referred to as " user behavior analysis device " according to belonging to, not shown)

Subscriber Unit determining device is mainly used in each Subscriber Unit corresponding to sampling mark determining to need the sampling mark of the experimental group of carrying out user behavior analysis and control group according to above-mentioned corresponding relation.

Log information acquisition device is mainly used in obtaining from daily record storage block corresponding to each Subscriber Unit belonging to the log information needing the log information of the network calling party of the experimental group of carrying out user behavior analysis He belong to the network calling party needing the control group carrying out user behavior analysis.

User behavior analysis device is mainly used in needing the log information of the network calling party of the experimental group of carrying out user behavior analysis and belong to need the log information of the network calling party of the control group carrying out user behavior analysis to carry out user behavior analysis according to belonging to.

Concrete, in the change needed for the particular content of webpage to flux experiment in network calling party in corresponding experimental group and control group when carrying out user behavior analysis, Subscriber Unit determining device first can determine the sampling mark of experimental group and the sampling mark of control group, then, Subscriber Unit determining device determines all Subscriber Units involved by the sampling mark of corresponding experimental group and the sampling mark of corresponding control group according to the above-mentioned corresponding relation pre-set, afterwards, log information acquisition device can determine the daily record storage block that each Subscriber Unit is corresponding respectively, thus the sampling mark of the sampling mark experimentally organized of log information acquisition device and control group is filtered the log information in corresponding daily record storage block, log information acquisition device can get the log information belonging to the network calling party of experimental group corresponding to sampling mark of corresponding experimental group and the log information belonging to the network calling party of control group corresponding to sampling mark of corresponding control group fast, behavioural analysis device in final utilization family can carry out user behavior analysis according to the log information of the network calling party belonging to experimental group with the log information of the network calling party belonging to control group.The specific implementation of the user behavior analysis of the present embodiment not performed by limited subscriber behavioural analysis device.

It should be noted that the present invention can be implemented in the assembly of software and/or software restraint, such as, each device of the present invention can adopt special IC (ASIC) or any other similar hardware device to realize.In one embodiment, software program of the present invention can perform to realize step mentioned above or function by processor.Similarly, software program of the present invention (comprising relevant data structure) can be stored in computer readable recording medium storing program for performing, such as, and RAM storer, magnetic or CD-ROM driver or flexible plastic disc and similar devices.In addition, steps more of the present invention or function can adopt hardware to realize, such as, as coordinating with processor thus performing the circuit of each step or function.

To those skilled in the art, obviously, the invention is not restricted to the details of above-mentioned one exemplary embodiment, and when not deviating from spirit of the present invention or essential characteristic, the present invention can be realized in other specific forms.Therefore, no matter from the viewpoint of which, all should regard embodiment as exemplary, and be nonrestrictive, scope of the present invention is limited by claims instead of above-mentioned explanation, therefore, all changes be intended in the implication of the equivalency by dropping on claim and scope are included in the present invention.Any Reference numeral in claim should be considered as the claim involved by limiting.In addition, obviously " comprising " one word do not get rid of other unit or step, odd number does not get rid of plural number.Multiple unit of stating in system claims or device also can be realized by software or hardware by a unit or device.The word such as first and second is used for representing title, and does not represent any particular order.

Although show and describe exemplary embodiment especially above, it will be appreciated by those skilled in the art that when not deviating from the spirit and scope of claims, can change to some extent in its form and details.Here sought protection is set forth in the dependent claims.

Claims

1., based on a flow layer method to set up for flux experiment, wherein, the method comprises the following steps:

Obtain the mark of network calling party;

According to the network calling party equal number principle in each Subscriber Unit, the mark of described network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer;

According to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, the mark of described network calling party is arranged at respectively in a Subscriber Unit of other each flow layers;

Wherein, each flow layer includes n sampling unit, each sampling unit includes n Subscriber Unit, and described corresponding relation point to be located in n sampling unit of other arbitrary flow layers based on n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer to arrange, n be greater than 1 integer.

2. method according to claim 1, wherein, the step of the mark of described acquisition network calling party comprises:

Obtain the cookie of network calling party.

3. method according to claim 1, wherein, describedly to comprise the step that the mark of described network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer according to the network calling party equal number principle in each Subscriber Unit:

Hash operation is carried out to the mark of described network calling party;

The mark of described network calling party is arranged in a Subscriber Unit of predetermined amount of flow layer by the result according to described Hash operation.

4. the method according to claim 1 or 2 or 3, wherein, described corresponding relation comprises:

The 1st Subscriber Unit in each sampling unit in n corresponding 1st flow layer to the n-th flow layer of Subscriber Unit difference in 1st sampling unit of the 0th flow layer.

5. the method according to claim 1 or 2 or 3, wherein, described corresponding relation comprises:

The 2nd Subscriber Unit in the 1st sampling unit in corresponding 1st flow layer to the n-th flow layer of n × (n-1) in 2nd sampling unit to the n-th sampling unit of the 0th flow layer individual Subscriber Unit difference is to nth user unit.

6. the method according to claim 1 or 2 or 3, wherein, described corresponding relation comprises:

The 2nd Subscriber Unit in 1st sampling unit of the 1st flow layer to the n-th flow layer is to the 2nd Subscriber Unit in the 2nd sampling unit to the n-th sampling unit in nth user unit respectively corresponding 1st flow layer to the n-th flow layer to nth user unit.

7. an implementation method for flux experiment, wherein, the method comprises the following steps:

A flow layer is chosen from n+1 flow layer;

The sampling unit that K+h different is selected, wherein K the corresponding experimental group of sampling unit, in addition h the corresponding control group of sampling unit from the described flow layer chosen;

Network access request according to the network calling party belonging to experimental group or control group sends the page for flux experiment to described network calling party;

Wherein, n sampling unit is included in each flow layer, each sampling unit includes n Subscriber Unit, and n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer point is located in n sampling unit of other arbitrary flow layers, n be greater than 1 integer, K and h is the integer being more than or equal to 1, and K+h is less than or equal to n.

8. method according to claim 7, wherein, described method also comprises:

For described experimental group and control group distribute different sampling marks respectively;

Set up sampling mark, the mark of Subscriber Unit belonging to network calling party and the corresponding relation between the mark of network calling party.

9. method according to claim 8, wherein, described method also comprises:

The sampling mark corresponding according to described corresponding relation determination network calling party;

The sampling mark of described correspondence is set in the log information of network calling party;

The log information of the network calling party belonging to same Subscriber Unit is stored in same daily record storage block.

10. method according to claim 9, wherein, described method also comprises:

The sampling mark of the experimental group needing to carry out user behavior analysis and each Subscriber Unit corresponding to sampling mark of control group is determined according to described corresponding relation;

Obtain from the daily record storage block that described each Subscriber Unit is corresponding and belong to the log information needing the log information of the network calling party of the experimental group of carrying out user behavior analysis He belong to the network calling party needing the control group carrying out user behavior analysis;

Need the log information of the network calling party of the experimental group of carrying out user behavior analysis and belong to need the log information of the network calling party of the control group carrying out user behavior analysis to carry out user behavior analysis according to described belonging to.

11. 1 kinds, based on the flow layer setting device of flux experiment, wherein, comprising:

User ID acquisition device, for obtaining the device of the mark of network calling party;

First setting device, for being arranged at the device in a Subscriber Unit of predetermined amount of flow layer by the mark of described network calling party according to the network calling party equal number principle in each Subscriber Unit;

Second setting device, for according to the corresponding relation between the Subscriber Unit of arbitrary flow layer and the Subscriber Unit of other n flow layer, is arranged at the device in a Subscriber Unit of other each flow layers respectively by the mark of described network calling party;

12. devices according to claim 11, wherein, described user ID acquisition device is specifically for obtaining the cookie of network calling party.

13. devices according to claim 11, wherein, described first setting device comprises:

Hash operation device, for carrying out the device of Hash operation to the mark of described network calling party;

3rd setting device, for being arranged at the device in a Subscriber Unit of predetermined amount of flow layer by the mark of described network calling party according to the result of described Hash operation.

14. devices according to claim 11 or 12 or 13, wherein, described corresponding relation comprises:

15. devices according to claim 11 or 12 or 13, wherein, described corresponding relation comprises:

16. devices according to claim 11 or 12 or 13, wherein, described corresponding relation comprises:

The implement device of 17. 1 kinds of flux experiment, wherein, comprising:

First selecting device, for choosing the device of a flow layer from n+1 flow layer;

Second selecting device, for selecting the device of the individual different sampling unit of K+h from the described flow layer chosen, wherein K the corresponding experimental group of sampling unit, in addition h the corresponding control group of sampling unit;

Page dispensing device, for sending device for the page of flux experiment according to the network access request of the network calling party belonging to experimental group or control group to described network calling party;

Wherein, n sampling unit is included in each flow layer, each sampling unit includes n Subscriber Unit, and n Subscriber Unit in arbitrary sampling unit of arbitrary flow layer point is located in n sampling unit of other arbitrary flow layers, n be greater than 1 integer, K and h be more than or equal to 1 integer, and K+h is less than or equal to n.

18. devices according to claim 17, wherein, the implement device of described flux experiment also comprises:

Sampling mark distributor, for distributing the device of different sampling marks respectively for described experimental group and control group;

Corresponding relation apparatus for establishing, for setting up the device of the corresponding relation between sampling mark, the mark of Subscriber Unit belonging to network calling party and the mark of network calling party.

19. devices according to claim 18, wherein, the implement device of described flux experiment also comprises:

Sampling mark determining device, for the device that the sampling corresponding according to described corresponding relation determination network calling party identifies;

Sampling mark setting device, for arranging the device of the sampling mark of described correspondence in the log information of network calling party;

Log information memory storage, for being stored in the device in same daily record storage block by the log information of the network calling party belonging to same Subscriber Unit.

20. devices according to claim 19, wherein, the implement device of described flux experiment also comprises:

Subscriber Unit determining device, for determine the sampling mark of the experimental group needing to carry out user behavior analysis and control group according to described corresponding relation sampling mark corresponding to the device of each Subscriber Unit;

Log information acquisition device, for obtaining the device belonging to the log information needing the log information of the network calling party of the experimental group of carrying out user behavior analysis and belong to the network calling party needing the control group carrying out user behavior analysis from daily record storage block corresponding to described each Subscriber Unit;

User behavior analysis device, for belonging to the log information of the network calling party needing the experimental group of carrying out user behavior analysis and belonging to the device needing the log information of the network calling party of the control group carrying out user behavior analysis to carry out user behavior analysis described in basis.