CA2212019A1 - Method for classifying test subjects in knowledge and functionality states - Google Patents
Method for classifying test subjects in knowledge and functionality statesInfo
- Publication number
- CA2212019A1 CA2212019A1 CA002212019A CA2212019A CA2212019A1 CA 2212019 A1 CA2212019 A1 CA 2212019A1 CA 002212019 A CA002212019 A CA 002212019A CA 2212019 A CA2212019 A CA 2212019A CA 2212019 A1 CA2212019 A1 CA 2212019A1
- Authority
- CA
- Canada
- Prior art keywords
- test
- test item
- item
- items
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
- G01R31/28—Testing of electronic circuits, e.g. by signal tracer
- G01R31/317—Testing of digital circuits
- G01R31/3181—Functional testing
- G01R31/3183—Generation of test inputs, e.g. test vectors, patterns or sequences
- G01R31/318371—Methodologies therefor, e.g. algorithms, procedures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0203—Market surveys; Market polls
Abstract
A method for classifying a test subject in one of a plurality of states in a domain, a domain being a set of facts, a quality measure, or a combination of the two. The set of facts for a knowledge domain is any set of facts while the set of facts for a functionality domain is a set of facts relating to the functionality of a test subject. A state is characterized by a subset of facts, a value for a quality measure, or a combination of a subset of facts and a value for a quality measure. A first state is higher than or equal to a second state and a second state is lower than or equal to a first state if (1) the subset of facts or the quality measure value associated with the first state respectively includes the subset of facts or is greater than or equal to the quality measure value associated with the second state or (2) the subset of facts and the quality measure value associated with the first state respectively includes the subset of facts and is greater than or equal to the quality measure value associated with the second state. Decision-theoretic rules are specified for selecting the test items to be administered to a test subject, for determining when it is appropriate to stop administering test items, and for determining the classification of the test subject. A test subject is classified in the highest state of which he has the knowledge or functionality.
Claims (57)
1. A method for classifying a test subject in one of a plurality of states in a domain, a domain being a set of facts, a quality measure having a range of values, or a combination of a set of facts and a quality measure, the set of facts for a knowledge domain being any set of facts, the set of facts for a functionality domain being a set of facts relating to the functionality of a test subject, a state being characterized by a subset of facts, a value in the range of values for a quality measure, or a combination of a subset of facts and a value for a quality measure, a first state being higher than or equal to a second state and a second state being lower than or equal to a first state if (1) the subset of facts or the quality measure value associated with the first state respectively includes the subset of facts or is greater than or equal to the quality measure value associated with the second state or (2) the subset of facts and the quality measure value associated with the first state respectively includes the subset of facts and is greater than or equal to the quality measure value associated with the second state, a test subject being classified in the highest state of which he has the knowledge or functionality, the method comprising the steps:
specifying a domain comprising a plurality of states and determining the higher-lower-neither relationships for each state, the higher-lower-neither relationships for a state being a specification of which states are higher, which states are lower, and which states are neither higher or lower, the plurality of states including a first, second, and third fact state characterized by subsets of facts wherein (1) the first and second fact states are higher than the third fact state and the first fact state is neither higher nor lower than the second fact state or (2) the first fact state is higher than the second and third fact states and the second fact state is neither higher nor lower than the third fact state;
specifying a test item pool comprising a plurality of test items;
specifying an initial state probability set (SPS) for the test subject to be classified, each member of the initial SPS being an initial estimate of the probability density value that the test subject is associated with a particular state in the domain;
specifying a class conditional density fi(x¦s) for each test item i in the test item pool for each state s in the domain, a class conditional density being a specification of the probability of a test subject in state s providing a response x to the test item i, each item partitioning the domain of states into a plurality of partitions according to the class conditional densities associated with the item, a partition being a subset of states for which the class conditional densities are the same or the union of such subsets.
specifying a domain comprising a plurality of states and determining the higher-lower-neither relationships for each state, the higher-lower-neither relationships for a state being a specification of which states are higher, which states are lower, and which states are neither higher or lower, the plurality of states including a first, second, and third fact state characterized by subsets of facts wherein (1) the first and second fact states are higher than the third fact state and the first fact state is neither higher nor lower than the second fact state or (2) the first fact state is higher than the second and third fact states and the second fact state is neither higher nor lower than the third fact state;
specifying a test item pool comprising a plurality of test items;
specifying an initial state probability set (SPS) for the test subject to be classified, each member of the initial SPS being an initial estimate of the probability density value that the test subject is associated with a particular state in the domain;
specifying a class conditional density fi(x¦s) for each test item i in the test item pool for each state s in the domain, a class conditional density being a specification of the probability of a test subject in state s providing a response x to the test item i, each item partitioning the domain of states into a plurality of partitions according to the class conditional densities associated with the item, a partition being a subset of states for which the class conditional densities are the same or the union of such subsets.
2. The method of claim 1 further comprising the steps:
utilizing a test item selection process selected from the group consisting of (1) administering one of a sequence of test items from the test item pool to the test subject and updating the SPS after receiving a response to the administered test item and (2) specifying a strategy tree which identifies the next test item to be administered, a strategy tree being a plurality of paths, each path beginning with the first test item to be administered, continuing through a sequence alternating between a particular response to the last test item and the specification of the next test item, and ending with a particular response to the final test item in the path, the classification of the test subject being specified for each path of the strategy tree using a decision rule.
utilizing a test item selection process selected from the group consisting of (1) administering one of a sequence of test items from the test item pool to the test subject and updating the SPS after receiving a response to the administered test item and (2) specifying a strategy tree which identifies the next test item to be administered, a strategy tree being a plurality of paths, each path beginning with the first test item to be administered, continuing through a sequence alternating between a particular response to the last test item and the specification of the next test item, and ending with a particular response to the final test item in the path, the classification of the test subject being specified for each path of the strategy tree using a decision rule.
3. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the largest value of a weighted uncertainty measure, the uncertainty measure being a measure of the uncertainty as to which of the test item's partitions that the test subject is in, the uncertainty measure being smallest when all but one of the partition probabilities are near 0, a partition probability being the probability of the test subject being in the partition.
4. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the smallest value of a weighted SPS
uncertainty measure given the hypothetical administration of i(n+1), an SPS uncertainty measure being a minimum when all but one of the SPS probability density values are near 0.
uncertainty measure given the hypothetical administration of i(n+1), an SPS uncertainty measure being a minimum when all but one of the SPS probability density values are near 0.
5. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the smallest value of a weighted SPS
uncertainty measure given the hypothetical administration of the sequence of test items i(n+1), i(n+2), .
.., i(n+k), k being an integer, an SPS uncertainty measure being a minimum when all but one of the SPS
probability density values are near 0.
uncertainty measure given the hypothetical administration of the sequence of test items i(n+1), i(n+2), .
.., i(n+k), k being an integer, an SPS uncertainty measure being a minimum when all but one of the SPS
probability density values are near 0.
6. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the largest value of a weighted distance measure between the SPS after a hypothetical administration of an (n+1)'th test item and the SPS after the actual administration of the n'th test item, the distance measure being a measure of the differences in the two SPSs.
7. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the largest value of a weighted distance measure between the SPS after a hypothetical administration of the sequence of test items i(n+1), i(n+2), ..., i(n+k), k being an integer, and the SPS after the actual administration of the n'th test item, the distance measure being a measure of the differences in the two SPSs.
8. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, the test item i(n+1) being the test item corresponding to the largest value of a weighted discrepancy measure summed over all pairs of states, a discrepancy measure for a test item given two states being a measure of the distance between the class conditional densities for the item and the two states.
9. The method of claim 8 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, the test item i(n+1) being selected using a two-valued function .PHI., the function .PHI. being a function of (1) a test item and (2) a first state and a second state, .PHI. having a first value if the test item separates the first and second states, .PHI. having a second value if the test item does not separate the first and second states..
10. The method of claim 9 wherein .PHI. has a first value for a plurality of the test items for a specified first state and a specified second state, the test item i(n+1) being selected randomly from the plurality of test items.
11. The method of claim 2 wherein test item i'(n+1) is tentatively selected as the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process, test item i'(n+1) being selected using a predetermined selection rule from test items in the test item pool that have not already been administered, a random decision being made either to use test item i'(n+1) as the (n+1)'th administered test item i(n+1) or to select another test item from the test item pool.
12. The method of claim 11 wherein the test items are ordered according to a goodness criterion associated with the predetermined selection rule, the test item i'(n+1) being the best test item, a plurality of the next-in-order test items being denoted as the better test items, one of the better test items being selected as the (n+1)'th administered test item i(n+1) if the decision is made to select a test item other than test item i'(n+1).
13. The method of claim 12 wherein the selection of one of the better test items is randomly made, the random selection being biased in accordance with the order of the better test items.
14. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, the test item i(r) being the test item that would be selected using selection rule r, the index r denoting any one of a plurality of selection rules, the test item i(n+1) being a random selection from the test items i(r).
15. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, the test item i(n+1) being the test item that maximizes a weighted relative ranking measure based on a plurality of item selection rules, a weighted relative ranking measure being a weighted function of the relative rankings of attractiveness for each test item with respect to a plurality of item selection rules.
16. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, the test item i(n+1) being the test item corresponding to the largest value of the sum of .pi.n(j).pi.n(k)djk(i) over all states j and k in the domain, .pi.n(j) denoting the members of the updated SPS after the test subject has responded to the n'th administered test item, djk(i) denoting a measure of the degree of discrimination between states j and k provided by test item i as measured by a discrepancy measure on the corresponding class conditional densities.
17. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the smallest value of a weighted loss function for k=1, a loss function being a function of (1) the state in the domain, (2) a classification decision action that specifies a state, and (3) the number k of test items to be administered beginning with i(n+1).
18. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item that results in the smallest value of a weighted loss function given the hypothetical administration of the sequence of test items i(n+1), i(n+2), . . ., i(n+k), k being an integer, a loss function being a function of (1) the state in the domain, (2) a classification decision action that specifies a state, and (3) the number k of test items to be administered beginning with i(n+1).
19. The method of claim 2 wherein the (n+1)'th administered test item i(n+1) in the step of utilizing a test item selection process is selected from test items in the test item pool that have not already been administered, i(n+1) being the test item for which a weighted loss in administering the next k test items is the smallest, the loss in administering k test items being defined by a loss function consisting of two additive components, the first component being a measure of the loss associated with the classification of the test subject after administering the k test items, the loss associated with an incorrect classification being higher than the loss associated with a correct classification, the second component being the cost of administering the k test items.
20. The method of claim 19 wherein the first component of the loss function is (1) a constant A1(s) if the test subject would be classified correctly after administering k additional test items and (2) a constant A2(s) if the test subject would be classified correctly after administering k additional test items, the constants A1(s) and A2(s) having a possible dependence on the state s, the second component of the loss function being the sum of the individual costs of administering the k additional test items.
21. The method of claim 2 wherein the respective responses to test items i(n+1), i(n+2), . . .
, i(n+k) are x(n+1), x(n+2), ..., x(n+k), k being an integer, and the SPS updating rule is a function of the class conditional densities evaluated at x(n+1), x(n+2) . . ., x(n+k) and a given SPS with the SPS
probability density value for a state being nondecreasing in the class conditional density value for fixed SPS and fixed class conditional density values for all other states.
, i(n+k) are x(n+1), x(n+2), ..., x(n+k), k being an integer, and the SPS updating rule is a function of the class conditional densities evaluated at x(n+1), x(n+2) . . ., x(n+k) and a given SPS with the SPS
probability density value for a state being nondecreasing in the class conditional density value for fixed SPS and fixed class conditional density values for all other states.
22. The method of claim 2 wherein in the step of utilizing a test item selection process a stopping rule is applied after each selection of a test item, an additional test item being selected only if the stopping rule so specifies, the test subject being classified as to a state in the domain in accordance with a decision rule if the stopping rule requires that the selection of test items be terminated.
23. The method of claim 22 wherein the domain is a combination model and the stopping rule is to stop selecting test items after selecting the n'th test item in if the marginal posterior value for a state in the discrete component of the domain is greater than a first predetermined value and the posterior variance of the continuous parameter in the domain is less than a second predetermined value.
24. The method of claim 22 wherein the stopping rule is to stop selecting test items if a weighted uncertainty measure with respect to the SPS after selection of the n'th test item in is less than a predetermined value.
25. The method of claim 22 wherein the stopping rule is to stop selecting test items if a weighted distance measure between the initial SPS and the SPS after selection of the n'th test item in exceeds a predetermined value.
26. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if a weighted loss function is less than a predetermined value.
27. The method of claim 22 wherein the stopping rule is to stop selecting test items if the largest value of the SPS after selecting of the n'th test item in exceeds a threshold.
28. The method of claim 22 wherein the stopping rule is to stop selecting test items if a predetermined number of test items have been selected.
29. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if, given the hypothetical selection of the sequence of one or more test items i(n+1), i(n+2) . . ., i(n+k) where k is an integer, a weighted loss function increases in value.
30. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if, given the hypothetical selection of the sequence of one or more test items i(n+1) i(n+2), ..., i(n+k) where k is an integer, a weighted uncertainty measure decreases by less than a predetermined value, the weighted uncertainty measure being with respect to the SPS after the selection of item in and after the hypothetical selection of the sequence of test items i(n+1) i(n+2), . . . , i(n+k).
31. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if, given the hypothetical selection of the sequence of one or more test items i(n+1) i(n+2), . . ., i(n+k) where k is an integer, a weighted distance measure increases by less than a predetermined value, the weighted distance measure being with respect to the SPS after the selection of item in and after the hypothetical selection of the sequence of test items i(n+1), i(n+2), . . ., i(n+k).
32. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if a specified criterion is satisfied, the specified criterion being defined in terms of one or more first conditions, one or more second conditions, one or more third conditions, one or more fourth conditions, one or more fifth conditions, one or more sixth conditions, or combinations thereof, a first condition being that a weighted loss function is less than a first predetermined value, a second condition being that a weighted loss function increases in value given the hypothetical selection of one or more test items, a third condition being that a weighted uncertainty measure is less than a second predetermined value, a fourth condition being that a weighted uncertainty measure decreases by less than a third predetermined value given the hypothetical selection of one or more test items, a fifth condition being that a weighted distance measure is larger than a fourth predetermined value, and a sixth condition being that a weighted distance measure increases by less than a predetermined value given the hypothetical selection of one or more test items.
33. The method of claim 22 wherein the decision rule is to select the state associated with the highest value in the SPS.
34. The method of claim 22 wherein the decision rule is to select the state associated with the smallest value for a weighted loss function.
35. The method of claim 2 further comprising the step:
determining for a test item in the test item pool the weighted frequency and/or the probability of being selected;
removing a test item from the test item pool if the weighted frequency and/or the probability of being selected is less than a predetermined value.
determining for a test item in the test item pool the weighted frequency and/or the probability of being selected;
removing a test item from the test item pool if the weighted frequency and/or the probability of being selected is less than a predetermined value.
36. The method of claim 22 wherein the stopping rule is to stop selecting test items after selection of the n'th test item in if, given the hypothetical administration of the sequence of one or more test items i(n+1) i(n+2) ..., i(n+k) where k is an integer, if a specified criterion is satisfied, the specified criterion being defined in terms of one or more first conditions, one or more second conditions, one or more third conditions, or combinations thereof, a first condition being that a weighted loss function increases in value, a second condition being that a weighted uncertainty measure decreases by less than a specified predetermined value, and a third condition being that a weighted distance measure increases by less than a specified predetermined value, the weighted uncertainty measure and the weighted distance measure being with respect to the SPS after the selection of item in and after the hypothetical selection of the sequence of test items i(n+1) i(n+2), ..., i(n+k).
37. The method of claim 2 wherein one or more test items at the ends of a specified strategy tree are removed if the weighted loss in administering test items for the resulting strategy tree is less than the weighted loss for the specified strategy tree, the weighted loss being obtained by weighting a loss function over all paths in the strategy tree and all test subject states, the loss function being a measure of the loss associated with administering the test items in a path of the strategy tree.
38. The method of claim 37 wherein the loss function is a function of (1) the state of the domain, (2) a classification decision action that specifies a state, and (3) the number of items administered.
39. The method of claim 37 wherein the loss function consists of two additive components, the first component being a measure of the loss associated with the classification of the test subject after administering the k test items, the loss associated with an incorrect classification being higher than the loss associated with a correct classification, the second component being the cost of administering the k test items.
40. The method of claim 39 wherein the first component of the loss function is (1) a constant A1(s) if the test subject would be classified correctly after administering k additional test items and (2) a constant A2(s) if the test subject would be classified incorrectly after administering k additional test items, the constants A1(s) and A2(s) having a possible dependence on the state s, the second component of the loss function being the sum of the individual costs of administering the k additional test items.
41. The method of claim 2 wherein the decision rule is to select the state associated with the highest value in the SPS.
42. The method of claim 2 wherein the decision rule is to select the state associated with the smallest value for a weighted loss function.
43. The method of claim 2 further comprising the step:
administering test items to a test subject in accordance with the strategy tree;
classifying the test subject in accordance with the path followed by the test subject through the strategy tree.
administering test items to a test subject in accordance with the strategy tree;
classifying the test subject in accordance with the path followed by the test subject through the strategy tree.
44. The method of claim 1 wherein the domain specifying step comprises the steps:
specifying at least one initial model of the states in the domain;
administering the test items in the test item pool to a plurality of test subjects, classifying each test subject as to state for each initial model;
identifying superfluous states in each model and eliminating them from the model;
identifying missing states in each model and adding them to the model.
specifying at least one initial model of the states in the domain;
administering the test items in the test item pool to a plurality of test subjects, classifying each test subject as to state for each initial model;
identifying superfluous states in each model and eliminating them from the model;
identifying missing states in each model and adding them to the model.
45. The method of claim 1 wherein the test-item-pool-specifying step comprises the steps:
specifying at least one initial model of the states in the domain;
administering the test items in the test item pool to a plurality of test subjects.
determining for a test item in the test item pool the weighted frequency and/or probability that the test item is administered;
removing the test item if the weighted frequency and/or probability is less than a predetermined value.
specifying at least one initial model of the states in the domain;
administering the test items in the test item pool to a plurality of test subjects.
determining for a test item in the test item pool the weighted frequency and/or probability that the test item is administered;
removing the test item if the weighted frequency and/or probability is less than a predetermined value.
46. The method of claim 1 wherein the domain specifying step comprises the steps:
specifying at least one initial model of the states in the domain;
administering a sequence of test items from the test item pool to each of one or more test subjects, the corresponding sequence of responses obtained for each of the test subjects being called a test subject response pattern;
determining the ideal response pattern for a test subject in each of one or more domain states for each administered sequence of test items using the class conditional densities associated with each test item, an ideal response being a value or a set of values;
identifying those ideal response patterns that do not satisfy a specified criterion with respect to each test subject response pattern, the specified criterion being specified in terms of one or more distance measures, a distance measure being a measure of the differences between a test subject response pattern and an ideal response pattern.
specifying at least one initial model of the states in the domain;
administering a sequence of test items from the test item pool to each of one or more test subjects, the corresponding sequence of responses obtained for each of the test subjects being called a test subject response pattern;
determining the ideal response pattern for a test subject in each of one or more domain states for each administered sequence of test items using the class conditional densities associated with each test item, an ideal response being a value or a set of values;
identifying those ideal response patterns that do not satisfy a specified criterion with respect to each test subject response pattern, the specified criterion being specified in terms of one or more distance measures, a distance measure being a measure of the differences between a test subject response pattern and an ideal response pattern.
47. The method of claim 46 wherein a state is removed from the initial model if its corresponding ideal response pattern does not satisfy the specified criterion with respect to a specified number of test subject patterns.
48. The method of claim 46 wherein one or more states is added to the initial model if a specified number of ideal response patterns do not satisfy the specified criterion with respect to one or more test subject response patterns.
49. The method of claim 1 wherein the test-item-pool specifying step comprises the steps:
determining the intersections of the partitions of states by the test items in the test item pool.
determining the intersections of the partitions of states by the test items in the test item pool.
50. The method of claim 49 further comprising the steps:
adding new types of test items to the test item pool to reduce multi-state intersections to single-state intersections.
adding new types of test items to the test item pool to reduce multi-state intersections to single-state intersections.
51. The method of claim 1 wherein the domain-specifying step comprises the steps:
determining the intersections of the partitions of states by the test items in the test item pool;
replacing the original domain model with a new domain model with new states, the new states being the intersections of the partitions of the original states by the test items, the higher-lower-neither relationships of the new states being derived from the higher-lower-neither relationships of the original states.
determining the intersections of the partitions of states by the test items in the test item pool;
replacing the original domain model with a new domain model with new states, the new states being the intersections of the partitions of the original states by the test items, the higher-lower-neither relationships of the new states being derived from the higher-lower-neither relationships of the original states.
52. The method of claim 1 wherein the class-conditional-density-specifying step comprises the steps;
(a) specifying prior distribution functions for the parameters of each test item and state, the class conditional densities for the test items being determinable from the item parameter distribution functions;
(b) administering a sequence of test items from the test item pool to each of a plurality of test subjects and recording the sequence of responses;
(c) updating the SPS for each response in the sequence of responses using the initial SPS and the test item class conditional densities;
(d) determining the test subject's classification;
(e) updating the distribution functions for the parameters of each test item and state utilizing each test subject's classification;
(f) repeating steps (c), (d), (e), and (f) until the process converges.
(a) specifying prior distribution functions for the parameters of each test item and state, the class conditional densities for the test items being determinable from the item parameter distribution functions;
(b) administering a sequence of test items from the test item pool to each of a plurality of test subjects and recording the sequence of responses;
(c) updating the SPS for each response in the sequence of responses using the initial SPS and the test item class conditional densities;
(d) determining the test subject's classification;
(e) updating the distribution functions for the parameters of each test item and state utilizing each test subject's classification;
(f) repeating steps (c), (d), (e), and (f) until the process converges.
53. The method of claim 1 wherein the test-item-pool-specifying step comprises the steps:
determining the sharpness of a test item from the test item pool, sharpness being a measure of the capability of a test item to discriminate between test subjects in different states, sharpness being measured by use of one or more discrepancy measures;
removing the test item from the pool if its sharpness does not satisfy a predetermined criterion.
determining the sharpness of a test item from the test item pool, sharpness being a measure of the capability of a test item to discriminate between test subjects in different states, sharpness being measured by use of one or more discrepancy measures;
removing the test item from the pool if its sharpness does not satisfy a predetermined criterion.
54. The method of claim 1 wherein the class-conditional-density-specifying step comprises the steps:
identifying test items having questionable class conditional densities, a questionable class conditional density being indicated by a sharpness criterion not being satisfied;
changing the class conditional probability density of one or more test items to achieve greater sharpness.
identifying test items having questionable class conditional densities, a questionable class conditional density being indicated by a sharpness criterion not being satisfied;
changing the class conditional probability density of one or more test items to achieve greater sharpness.
55. The method of claim 54 wherein the class-conditional-density-specifying step further comprises the steps:
removing from the test item pool any test items for which the class conditional densities were changed and the sharpness criterion was not satisfied.
removing from the test item pool any test items for which the class conditional densities were changed and the sharpness criterion was not satisfied.
56. The method of claim 1 further comprising the step:
specifying a remediation program for each state in the domain, a remediation program for state X being a compilation of facts associated with one or more other states in the domain and a procedure for teaching the facts in the compilation to a test subject, the compilation not including facts associated with state X.
specifying a remediation program for each state in the domain, a remediation program for state X being a compilation of facts associated with one or more other states in the domain and a procedure for teaching the facts in the compilation to a test subject, the compilation not including facts associated with state X.
57. The method of claim 1 comprising the steps:
(a) administering one of a sequence of test items from the test item pool to the test subject;
(b) updating the SPS after receiving a response to the administered test item;
(c) applying a stopping rule, steps (a) and (b) being repeated if the stopping rule so specifies;
(d) classifying the test subject as to a state in the domain in accordance with a classification rule if the stopping rule so specifies;
(e) directing the test subject to a remediation program.
(a) administering one of a sequence of test items from the test item pool to the test subject;
(b) updating the SPS after receiving a response to the administered test item;
(c) applying a stopping rule, steps (a) and (b) being repeated if the stopping rule so specifies;
(d) classifying the test subject as to a state in the domain in accordance with a classification rule if the stopping rule so specifies;
(e) directing the test subject to a remediation program.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/712,110 | 1996-09-13 | ||
US08/712,110 US5855011A (en) | 1996-09-13 | 1996-09-13 | Method for classifying test subjects in knowledge and functionality states |
Publications (2)
Publication Number | Publication Date |
---|---|
CA2212019A1 true CA2212019A1 (en) | 1998-03-13 |
CA2212019C CA2212019C (en) | 2011-09-13 |
Family
ID=24860795
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CA2212019A Expired - Fee Related CA2212019C (en) | 1996-09-13 | 1997-07-31 | Method for classifying test subjects in knowledge and functionality states |
Country Status (3)
Country | Link |
---|---|
US (3) | US5855011A (en) |
JP (1) | JPH10153947A (en) |
CA (1) | CA2212019C (en) |
Families Citing this family (159)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5875440A (en) * | 1997-04-29 | 1999-02-23 | Teleran Technologies, L.P. | Hierarchically arranged knowledge domains |
GB9803368D0 (en) * | 1998-02-17 | 1998-04-15 | Cambridge Consultants | Measurement system |
JPH11306238A (en) * | 1998-03-30 | 1999-11-05 | Internatl Business Mach Corp <Ibm> | Probability integration system |
US6405188B1 (en) * | 1998-07-31 | 2002-06-11 | Genuity Inc. | Information retrieval system |
US6542880B2 (en) * | 1998-12-22 | 2003-04-01 | Indeliq, Inc. | System, method and article of manufacture for a goal based system utilizing a table based architecture |
US6782374B2 (en) * | 1998-12-22 | 2004-08-24 | Accenture Global Services Gmbh | System, method and article of manufacturing for a runtime program analysis tool for a simulation engine |
US7194444B1 (en) * | 1999-02-08 | 2007-03-20 | Indeliq, Inc. | Goal based flow of a control presentation system |
US6658412B1 (en) * | 1999-06-30 | 2003-12-02 | Educational Testing Service | Computer-based method and system for linking records in data files |
US8202097B1 (en) * | 1999-09-01 | 2012-06-19 | Educational Testing Service | Computer based test item generation |
US6633857B1 (en) * | 1999-09-04 | 2003-10-14 | Microsoft Corporation | Relevance vector machine |
US7072846B1 (en) * | 1999-11-16 | 2006-07-04 | Emergent Music Llc | Clusters for rapid artist-audience matching |
US8150724B1 (en) | 1999-11-16 | 2012-04-03 | Emergent Discovery Llc | System for eliciting accurate judgement of entertainment items |
GB9927371D0 (en) * | 1999-11-20 | 2000-01-19 | Ncr Int Inc | Processing database entries to provide predictions of future data values |
US6879944B1 (en) * | 2000-03-07 | 2005-04-12 | Microsoft Corporation | Variational relevance vector machine |
US6419496B1 (en) * | 2000-03-28 | 2002-07-16 | William Vaughan, Jr. | Learning method |
US6917943B2 (en) * | 2000-05-12 | 2005-07-12 | Limit Point Systems, Inc. | Sheaf data model |
US20040172401A1 (en) * | 2000-06-15 | 2004-09-02 | Peace Terrence B. | Significance testing and confidence interval construction based on user-specified distributions |
US6895398B2 (en) * | 2000-07-18 | 2005-05-17 | Inferscape, Inc. | Decision engine and method and applications thereof |
US20070027672A1 (en) * | 2000-07-31 | 2007-02-01 | Michel Decary | Computer method and apparatus for extracting data from web pages |
US6618717B1 (en) * | 2000-07-31 | 2003-09-09 | Eliyon Technologies Corporation | Computer method and apparatus for determining content owner of a website |
JP3883795B2 (en) * | 2000-08-24 | 2007-02-21 | 富士通株式会社 | Attendance class selection device, attendance class selection method, and storage medium |
AUPR009000A0 (en) * | 2000-09-13 | 2000-10-05 | Guignard, Paul Dr | Intelligent courseware developement and delivery environment |
JP3954295B2 (en) * | 2000-11-02 | 2007-08-08 | 独立行政法人科学技術振興機構 | IDENTIFICATION / RESPONSE MEASUREMENT METHOD, COMPUTER-READABLE RECORDING MEDIUM CONTAINING IDENTIFICATION / REACTION MEASUREMENT PROGRAM |
WO2002065232A2 (en) * | 2000-11-10 | 2002-08-22 | The Procter & Gamble Company | Method and system for facilitating assessments |
US6965852B2 (en) * | 2000-12-15 | 2005-11-15 | International Business Machines Corporation | Pseudo random test pattern generation using Markov chains |
DE10291392B4 (en) * | 2001-02-15 | 2007-08-16 | Metalife Ag | Method, system and data carrier for generating correlations and / or interactions and / or knowledge from a plurality of searched data sets |
US6688889B2 (en) | 2001-03-08 | 2004-02-10 | Boostmyscore.Com | Computerized test preparation system employing individually tailored diagnostics and remediation |
AU2002252645A1 (en) * | 2001-04-11 | 2002-10-28 | Fair Isaac And Company, Inc. | Model-based and data-driven analytic support for strategy development |
US6789047B1 (en) | 2001-04-17 | 2004-09-07 | Unext.Com Llc | Method and system for evaluating the performance of an instructor of an electronic course |
US6823323B2 (en) * | 2001-04-26 | 2004-11-23 | Hewlett-Packard Development Company, L.P. | Automatic classification method and apparatus |
US7286793B1 (en) | 2001-05-07 | 2007-10-23 | Miele Frank R | Method and apparatus for evaluating educational performance |
US20030033085A1 (en) * | 2001-05-10 | 2003-02-13 | Sanchez Humberto A. | Mechanism for ensuring defect-free objects via object class tests |
US6917926B2 (en) * | 2001-06-15 | 2005-07-12 | Medical Scientists, Inc. | Machine learning method |
US6790045B1 (en) * | 2001-06-18 | 2004-09-14 | Unext.Com Llc | Method and system for analyzing student performance in an electronic course |
US10347145B1 (en) | 2001-10-05 | 2019-07-09 | Vision Works Ip Corporation | Method and apparatus for periodically questioning a user using a computer system or other device to facilitate memorization and learning of information |
EP1316907A2 (en) * | 2001-11-30 | 2003-06-04 | Koninklijke Philips Electronics N.V. | System for teaching the use of the functions of an apparatus, apparatus and method used in such a system |
US20040018479A1 (en) * | 2001-12-21 | 2004-01-29 | Pritchard David E. | Computer implemented tutoring system |
US20030177047A1 (en) * | 2002-02-04 | 2003-09-18 | Buckley Michael E. | Method and system for decision oriented systems engineering |
US7292681B2 (en) * | 2002-02-15 | 2007-11-06 | Apple Corporate Technologies, Inc. | Technique and an apparatus for producing postcards having an audio message for playback by recipient |
US6877989B2 (en) * | 2002-02-15 | 2005-04-12 | Psychological Dataccorporation | Computer program for generating educational and psychological test items |
US7130776B2 (en) * | 2002-03-25 | 2006-10-31 | Lockheed Martin Corporation | Method and computer program product for producing a pattern recognition training set |
SE0200991D0 (en) * | 2002-03-28 | 2002-03-28 | Adaptlogic Ab | Method for adapting information |
US8128414B1 (en) | 2002-08-20 | 2012-03-06 | Ctb/Mcgraw-Hill | System and method for the development of instructional and testing materials |
US8523575B2 (en) * | 2002-09-02 | 2013-09-03 | Nextthinksoft Pty Ltd. | Recalling items of information |
AU2002951608A0 (en) * | 2002-09-23 | 2002-10-10 | Lewis Cadman Consulting Pty Ltd | A method of delivering a test to a candidate |
WO2004053292A2 (en) | 2002-11-13 | 2004-06-24 | Educational Testing Service | Systems and methods for testing over a distributed network |
US7985074B2 (en) * | 2002-12-31 | 2011-07-26 | Chicago Science Group, L.L.C. | Method and apparatus for improving math skills |
US20040143481A1 (en) * | 2003-01-21 | 2004-07-22 | Li Bernard A. | Online business method for surveying customer accessory package preferences |
CA2516160A1 (en) * | 2003-02-14 | 2004-09-02 | Ctb/Mcgraw-Hill | System and method for creating, assessing, modifying, and using a learning map |
US20040220892A1 (en) * | 2003-04-29 | 2004-11-04 | Ira Cohen | Learning bayesian network classifiers using labeled and unlabeled data |
US7296030B2 (en) * | 2003-07-17 | 2007-11-13 | At&T Corp. | Method and apparatus for windowing in entropy encoding |
US20050053904A1 (en) * | 2003-08-13 | 2005-03-10 | Jennifer Shephard | System and method for on-site cognitive efficacy assessment |
US7240039B2 (en) * | 2003-10-29 | 2007-07-03 | Hewlett-Packard Development Company, L.P. | System and method for combining valuations of multiple evaluators |
US7526465B1 (en) * | 2004-03-18 | 2009-04-28 | Sandia Corporation | Human-machine interactions |
US7418458B2 (en) * | 2004-04-06 | 2008-08-26 | Educational Testing Service | Method for estimating examinee attribute parameters in a cognitive diagnosis model |
US7980855B1 (en) | 2004-05-21 | 2011-07-19 | Ctb/Mcgraw-Hill | Student reporting systems and methods |
US7927105B2 (en) * | 2004-09-02 | 2011-04-19 | International Business Machines Incorporated | Method and system for creating and delivering prescriptive learning |
US6970110B1 (en) | 2005-01-08 | 2005-11-29 | Collins Dennis G | Probability centrifuge algorithm with minimum laterally adiabatically-reduced Fisher information calculation |
US7539977B1 (en) * | 2005-01-21 | 2009-05-26 | Xilinx, Inc. | Automatic bug isolation in computer programming languages |
US20060188862A1 (en) * | 2005-02-18 | 2006-08-24 | Harcourt Assessment, Inc. | Electronic assessment summary and remedial action plan creation system and associated methods |
US20060246411A1 (en) * | 2005-04-27 | 2006-11-02 | Yang Steven P | Learning apparatus and method |
US8764455B1 (en) * | 2005-05-09 | 2014-07-01 | Altis Avante Corp. | Comprehension instruction system and method |
US8170466B2 (en) * | 2005-05-27 | 2012-05-01 | Ctb/Mcgraw-Hill | System and method for automated assessment of constrained constructed responses |
US20070009871A1 (en) * | 2005-05-28 | 2007-01-11 | Ctb/Mcgraw-Hill | System and method for improved cumulative assessment |
US20070031801A1 (en) * | 2005-06-16 | 2007-02-08 | Ctb Mcgraw Hill | Patterned response system and method |
US20080254437A1 (en) * | 2005-07-15 | 2008-10-16 | Neil T Heffernan | Global Computer Network Tutoring System |
US20070111182A1 (en) * | 2005-10-26 | 2007-05-17 | International Business Machines Corporation | Method and system for distributing answers |
US8676680B2 (en) | 2006-02-03 | 2014-03-18 | Zillow, Inc. | Automatically determining a current value for a home |
US8515839B2 (en) * | 2006-02-03 | 2013-08-20 | Zillow, Inc. | Automatically determining a current value for a real estate property, such as a home, that is tailored to input from a human user, such as its owner |
US8005712B2 (en) * | 2006-04-06 | 2011-08-23 | Educational Testing Service | System and method for large scale survey analysis |
US20070259325A1 (en) * | 2006-04-22 | 2007-11-08 | Clapper Rock L | Internet-based method and system for human-human question and answer sessions in delayed and real time |
US20080038705A1 (en) * | 2006-07-14 | 2008-02-14 | Kerns Daniel R | System and method for assessing student progress and delivering appropriate content |
US10347148B2 (en) * | 2006-07-14 | 2019-07-09 | Dreambox Learning, Inc. | System and method for adapting lessons to student needs |
US8234116B2 (en) * | 2006-08-22 | 2012-07-31 | Microsoft Corporation | Calculating cost measures between HMM acoustic models |
US8639176B2 (en) * | 2006-09-07 | 2014-01-28 | Educational Testing System | Mixture general diagnostic model |
US20080077458A1 (en) | 2006-09-19 | 2008-03-27 | Andersen Timothy J | Collecting and representing home attributes |
US20080208646A1 (en) * | 2007-02-28 | 2008-08-28 | Thompson Ralph E | Method for increasing productivity and safety in the mining and heavy construction industries |
WO2008121323A1 (en) * | 2007-03-28 | 2008-10-09 | Worcester Polytechnic Institute | Global computer network self-tutoring system |
US20090075246A1 (en) * | 2007-09-18 | 2009-03-19 | The Learning Chameleon, Inc. | System and method for quantifying student's scientific problem solving efficiency and effectiveness |
WO2009058344A1 (en) * | 2007-10-31 | 2009-05-07 | Worcester Polytechnic Institute | Computer method and system for increasing the quality of student learning |
US8140421B1 (en) | 2008-01-09 | 2012-03-20 | Zillow, Inc. | Automatically determining a current value for a home |
US8639177B2 (en) * | 2008-05-08 | 2014-01-28 | Microsoft Corporation | Learning assessment and programmatic remediation |
US20090325140A1 (en) * | 2008-06-30 | 2009-12-31 | Lou Gray | Method and system to adapt computer-based instruction based on heuristics |
US8020125B1 (en) * | 2008-09-10 | 2011-09-13 | Cadence Design Systems, Inc. | System, methods and apparatus for generation of simulation stimulus |
US11392918B1 (en) * | 2008-11-17 | 2022-07-19 | Charles Schwab & Co, Inc. | System and method for assisting individuals in assessing and improving their behavior regarding financial and other-than-financial planning based on their personal circumstances and assisting with implementing such plans |
US20100190143A1 (en) * | 2009-01-28 | 2010-07-29 | Time To Know Ltd. | Adaptive teaching and learning utilizing smart digital learning objects |
US20100190142A1 (en) * | 2009-01-28 | 2010-07-29 | Time To Know Ltd. | Device, system, and method of automatic assessment of pedagogic parameters |
WO2010139042A1 (en) * | 2009-06-02 | 2010-12-09 | Kim Desruisseaux | Learning environment with user defined content |
US8540518B2 (en) * | 2009-10-27 | 2013-09-24 | Honeywell International Inc. | Training system and method based on cognitive models |
US8655827B2 (en) * | 2009-10-29 | 2014-02-18 | Hewlett-Packard Development Company, L.P. | Questionnaire generation |
JP2011138197A (en) * | 2009-12-25 | 2011-07-14 | Sony Corp | Information processing apparatus, method of evaluating degree of association, and program |
US8639649B2 (en) * | 2010-03-23 | 2014-01-28 | Microsoft Corporation | Probabilistic inference in differentially private systems |
US20110244953A1 (en) * | 2010-03-30 | 2011-10-06 | Smart Technologies Ulc | Participant response system for the team selection and method therefor |
EP2426931A1 (en) * | 2010-09-06 | 2012-03-07 | Advanced Digital Broadcast S.A. | A method and a system for determining a video frame type |
US10380653B1 (en) | 2010-09-16 | 2019-08-13 | Trulia, Llc | Valuation system |
US8761658B2 (en) | 2011-01-31 | 2014-06-24 | FastTrack Technologies Inc. | System and method for a computerized learning system |
US9836455B2 (en) * | 2011-02-23 | 2017-12-05 | New York University | Apparatus, method and computer-accessible medium for explaining classifications of documents |
US10198735B1 (en) | 2011-03-09 | 2019-02-05 | Zillow, Inc. | Automatically determining market rental rate index for properties |
US10460406B1 (en) | 2011-03-09 | 2019-10-29 | Zillow, Inc. | Automatically determining market rental rates for properties |
EP2705440A4 (en) * | 2011-05-06 | 2014-12-31 | Opower Inc | Method and system for selecting similar consumers |
US8909127B2 (en) | 2011-09-27 | 2014-12-09 | Educational Testing Service | Computer-implemented systems and methods for carrying out non-centralized assessments |
US9348049B2 (en) * | 2012-01-05 | 2016-05-24 | Cgg Services Sa | Simultaneous joint estimation of the P-P and P-S residual statics |
US9058354B2 (en) * | 2012-01-26 | 2015-06-16 | University Of Rochester | Integrated multi-criteria decision support framework |
US20130245998A1 (en) * | 2012-03-13 | 2013-09-19 | Filippo Balestrieri | Selecting entities in a sampling process |
GB201206728D0 (en) * | 2012-04-16 | 2012-05-30 | Shl Group Ltd | testing system |
US20150086962A1 (en) * | 2012-04-23 | 2015-03-26 | Universiteit Antwerpen | Methods and Systems for Testing and Correcting |
US10796346B2 (en) | 2012-06-27 | 2020-10-06 | Opower, Inc. | Method and system for unusual usage reporting |
US20150099254A1 (en) * | 2012-07-26 | 2015-04-09 | Sony Corporation | Information processing device, information processing method, and system |
US20140038161A1 (en) * | 2012-07-31 | 2014-02-06 | Apollo Group, Inc. | Multi-layered cognitive tutor |
US9547316B2 (en) | 2012-09-07 | 2017-01-17 | Opower, Inc. | Thermostat classification method and system |
US20140095109A1 (en) * | 2012-09-28 | 2014-04-03 | Nokia Corporation | Method and apparatus for determining the emotional response of individuals within a group |
US10453355B2 (en) | 2012-09-28 | 2019-10-22 | Nokia Technologies Oy | Method and apparatus for determining the attentional focus of individuals within a group |
US9633401B2 (en) | 2012-10-15 | 2017-04-25 | Opower, Inc. | Method to identify heating and cooling system power-demand |
US8755737B1 (en) | 2012-12-24 | 2014-06-17 | Pearson Education, Inc. | Fractal-based decision engine for intervention |
US10067516B2 (en) | 2013-01-22 | 2018-09-04 | Opower, Inc. | Method and system to control thermostat using biofeedback |
US20140335498A1 (en) * | 2013-05-08 | 2014-11-13 | Apollo Group, Inc. | Generating, assigning, and evaluating different versions of a test |
US10719797B2 (en) | 2013-05-10 | 2020-07-21 | Opower, Inc. | Method of tracking and reporting energy performance for businesses |
US10001792B1 (en) | 2013-06-12 | 2018-06-19 | Opower, Inc. | System and method for determining occupancy schedule for controlling a thermostat |
US10467924B2 (en) * | 2013-09-20 | 2019-11-05 | Western Michigan University Research Foundation | Behavioral intelligence framework, content management system, and tool for constructing same |
US10754884B1 (en) | 2013-11-12 | 2020-08-25 | Zillow, Inc. | Flexible real estate search |
US10885238B1 (en) | 2014-01-09 | 2021-01-05 | Opower, Inc. | Predicting future indoor air temperature for building |
US10031534B1 (en) | 2014-02-07 | 2018-07-24 | Opower, Inc. | Providing set point comparison |
US9852484B1 (en) | 2014-02-07 | 2017-12-26 | Opower, Inc. | Providing demand response participation |
US9947045B1 (en) | 2014-02-07 | 2018-04-17 | Opower, Inc. | Selecting participants in a resource conservation program |
US10037014B2 (en) | 2014-02-07 | 2018-07-31 | Opower, Inc. | Behavioral demand response dispatch |
US10984489B1 (en) | 2014-02-13 | 2021-04-20 | Zillow, Inc. | Estimating the value of a property in a manner sensitive to nearby value-affecting geographic features |
US20150269567A1 (en) * | 2014-03-19 | 2015-09-24 | Mastercard International Incorporated | Methods and systems for improving payment card acceptance quality |
US9835352B2 (en) | 2014-03-19 | 2017-12-05 | Opower, Inc. | Method for saving energy efficient setpoints |
US9727063B1 (en) | 2014-04-01 | 2017-08-08 | Opower, Inc. | Thermostat set point identification |
US10019739B1 (en) | 2014-04-25 | 2018-07-10 | Opower, Inc. | Energy usage alerts for a climate control device |
US10108973B2 (en) | 2014-04-25 | 2018-10-23 | Opower, Inc. | Providing an energy target for high energy users |
US10171603B2 (en) | 2014-05-12 | 2019-01-01 | Opower, Inc. | User segmentation to provide motivation to perform a resource saving tip |
US10235662B2 (en) | 2014-07-01 | 2019-03-19 | Opower, Inc. | Unusual usage alerts |
US10024564B2 (en) | 2014-07-15 | 2018-07-17 | Opower, Inc. | Thermostat eco-mode |
US10467249B2 (en) | 2014-08-07 | 2019-11-05 | Opower, Inc. | Users campaign for peaking energy usage |
US10410130B1 (en) | 2014-08-07 | 2019-09-10 | Opower, Inc. | Inferring residential home characteristics based on energy data |
US10572889B2 (en) | 2014-08-07 | 2020-02-25 | Opower, Inc. | Advanced notification to enable usage reduction |
US9576245B2 (en) | 2014-08-22 | 2017-02-21 | O Power, Inc. | Identifying electric vehicle owners |
US11093982B1 (en) | 2014-10-02 | 2021-08-17 | Zillow, Inc. | Determine regional rate of return on home improvements |
US10033184B2 (en) | 2014-11-13 | 2018-07-24 | Opower, Inc. | Demand response device configured to provide comparative consumption information relating to proximate users or consumers |
US11093950B2 (en) | 2015-02-02 | 2021-08-17 | Opower, Inc. | Customer activity score |
US10198483B2 (en) | 2015-02-02 | 2019-02-05 | Opower, Inc. | Classification engine for identifying business hours |
US10074097B2 (en) | 2015-02-03 | 2018-09-11 | Opower, Inc. | Classification engine for classifying businesses based on power consumption |
US10371861B2 (en) | 2015-02-13 | 2019-08-06 | Opower, Inc. | Notification techniques for reducing energy usage |
US10643232B1 (en) | 2015-03-18 | 2020-05-05 | Zillow, Inc. | Allocating electronic advertising opportunities |
US10817789B2 (en) | 2015-06-09 | 2020-10-27 | Opower, Inc. | Determination of optimal energy storage methods at electric customer service points |
US9958360B2 (en) | 2015-08-05 | 2018-05-01 | Opower, Inc. | Energy audit device |
US10559044B2 (en) | 2015-11-20 | 2020-02-11 | Opower, Inc. | Identification of peak days |
US10789549B1 (en) | 2016-02-25 | 2020-09-29 | Zillow, Inc. | Enforcing, with respect to changes in one or more distinguished independent variable values, monotonicity in the predictions produced by a statistical model |
US10650008B2 (en) | 2016-08-26 | 2020-05-12 | International Business Machines Corporation | Parallel scoring of an ensemble model |
US11861747B1 (en) | 2017-09-07 | 2024-01-02 | MFTB Holdco, Inc. | Time on market and likelihood of sale prediction |
US11158203B2 (en) * | 2018-02-14 | 2021-10-26 | International Business Machines Corporation | Phased word expansion for vocabulary learning |
US11449762B2 (en) | 2018-02-20 | 2022-09-20 | Pearson Education, Inc. | Real time development of auto scoring essay models for custom created prompts |
US11741849B2 (en) | 2018-02-20 | 2023-08-29 | Pearson Education, Inc. | Systems and methods for interface-based machine learning model output customization |
CN109612173A (en) * | 2018-11-15 | 2019-04-12 | 南京航空航天大学 | A kind of assessment of fault and diagnostic method of vapor cycle refrigeration system |
US11941999B2 (en) * | 2019-02-21 | 2024-03-26 | Instructure, Inc. | Techniques for diagnostic assessment |
US11861748B1 (en) | 2019-06-28 | 2024-01-02 | MFTB Holdco, Inc. | Valuation of homes using geographic regions of varying granularity |
CN112446809B (en) * | 2020-11-25 | 2022-08-12 | 四川大学 | Mental health comprehensive self-adaptive evaluation method and system |
KR102412817B1 (en) * | 2020-12-18 | 2022-06-23 | 연세대학교 산학협력단 | X filling method and apparatus |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4847784A (en) * | 1987-07-13 | 1989-07-11 | Teknowledge, Inc. | Knowledge based tutor |
US5193144A (en) * | 1988-12-14 | 1993-03-09 | Shimano, Inc. | Fuzzy system |
CA2050888C (en) * | 1989-04-14 | 1998-06-16 | Tsutomu Ishida | Method of and apparatus for evaluating membership functions or rules in fuzzy reasoning system |
US5002491A (en) * | 1989-04-28 | 1991-03-26 | Comtek | Electronic classroom system enabling interactive self-paced learning |
JPH0535308A (en) * | 1991-07-25 | 1993-02-12 | Mitsubishi Electric Corp | Device and method for identifying fuzzy membership function |
US5245698A (en) * | 1991-08-14 | 1993-09-14 | Omron Corporation | Apparatus for and method of correcting membership functions |
US5259766A (en) * | 1991-12-13 | 1993-11-09 | Educational Testing Service | Method and system for interactive computer science testing, anaylsis and feedback |
US5267865A (en) * | 1992-02-11 | 1993-12-07 | John R. Lee | Interactive computer aided natural learning method and apparatus |
US5692906A (en) * | 1992-04-01 | 1997-12-02 | Corder; Paul R. | Method of diagnosing and remediating a deficiency in communications skills |
US5437554A (en) * | 1993-02-05 | 1995-08-01 | National Computer Systems, Inc. | System for providing performance feedback to test resolvers |
US5574828A (en) * | 1994-04-28 | 1996-11-12 | Tmrc | Expert system for generating guideline-based information tools |
US5644686A (en) * | 1994-04-29 | 1997-07-01 | International Business Machines Corporation | Expert system and method employing hierarchical knowledge base, and interactive multimedia/hypermedia applications |
US5704018A (en) * | 1994-05-09 | 1997-12-30 | Microsoft Corporation | Generating improved belief networks |
US5749736A (en) * | 1995-03-22 | 1998-05-12 | Taras Development | Method and system for computerized learning, response, and evaluation |
US5875431A (en) * | 1996-03-15 | 1999-02-23 | Heckman; Frank | Legal strategic analysis planning and evaluation control system and method |
US5727950A (en) * | 1996-05-22 | 1998-03-17 | Netsage Corporation | Agent based instruction system and method |
US5980447A (en) * | 1996-11-27 | 1999-11-09 | Phase Ii R & D -Dependency & Codependency Recovery Program Inc. | System for implementing dependency recovery process |
US5966541A (en) * | 1997-12-04 | 1999-10-12 | Incert Software Corporation | Test protection, and repair through binary-code augmentation |
-
1996
- 1996-09-13 US US08/712,110 patent/US5855011A/en not_active Expired - Lifetime
-
1997
- 1997-07-31 CA CA2212019A patent/CA2212019C/en not_active Expired - Fee Related
- 1997-09-04 JP JP9239571A patent/JPH10153947A/en active Pending
-
1998
- 1998-11-16 US US09/193,061 patent/US6301571B1/en not_active Expired - Lifetime
- 1998-11-16 US US09/192,815 patent/US6260033B1/en not_active Expired - Fee Related
Also Published As
Publication number | Publication date |
---|---|
US5855011A (en) | 1998-12-29 |
US6260033B1 (en) | 2001-07-10 |
US6301571B1 (en) | 2001-10-09 |
JPH10153947A (en) | 1998-06-09 |
CA2212019C (en) | 2011-09-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2212019A1 (en) | Method for classifying test subjects in knowledge and functionality states | |
Hall et al. | Feature subset selection: a correlation based filter approach | |
Dechter et al. | Network-based heuristics for constraint-satisfaction problems | |
Fraiman et al. | Selection of variables for cluster analysis and classification rules | |
Debuse et al. | Feature subset selection within a simulated annealing data mining algorithm | |
US7460717B2 (en) | Method and system for fuzzy clustering of images | |
Vanderschraaf | Knowledge, equilibrium and convention | |
Deogun et al. | Feature selection and effective classifiers | |
EP1661066A1 (en) | Hierarchical determination of feature relevancy | |
Hargrove et al. | A fractal landscape realizer for generating synthetic maps | |
Janikow | Exemplar learning in fuzzy decision trees | |
US5802507A (en) | Method for constructing a neural device for classification of objects | |
Michalski et al. | Learning patterns in noisy data: The AQ approach | |
CN115775026A (en) | Federated learning method based on organization similarity | |
US6505181B1 (en) | Recognition system | |
Gordon et al. | Terrain-based genetic algorithm (tbga) modeling parameter space as terrain | |
Cervantes et al. | A comparison between the Pittsburgh and Michigan approaches for the binary PSO algorithm | |
Siemann et al. | Transitive responding in pigeons: Influences of stimulus frequency and reinforcement history | |
Chang et al. | A boundary hunting radial basis function classifier which allocates centers constructively | |
CN115952418A (en) | Method and device for optimizing machine learning model based on model hyper-parameters | |
Reynolds et al. | Diversity as a necessity for sustainability in cultural systems: Collective problem-solving in cultural algorithms | |
KR100727555B1 (en) | Creating method for decision tree using time-weighted entropy and recording medium thereof | |
DE102020208041A1 (en) | Device and method for filling a knowledge graph using strategic data splits | |
US20030023575A1 (en) | System and method of automatic object classification by tournament strategy | |
Sun | Adaptive Optimizer for Automated Hyperparameter Optimization Problem |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
EEER | Examination request | ||
FZDC | Discontinued application reinstated | ||
MKLA | Lapsed |
Effective date: 20150731 |
|
MKLA | Lapsed |
Effective date: 20150731 |