CA2107412A1 - Collaborative and quality control scoring system - Google Patents

Collaborative and quality control scoring system

Info

Publication number
CA2107412A1
CA2107412A1 CA002107412A CA2107412A CA2107412A1 CA 2107412 A1 CA2107412 A1 CA 2107412A1 CA 002107412 A CA002107412 A CA 002107412A CA 2107412 A CA2107412 A CA 2107412A CA 2107412 A1 CA2107412 A1 CA 2107412A1
Authority
CA
Canada
Prior art keywords
test
resolver
score
resolvers
collaborative
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002107412A
Other languages
French (fr)
Other versions
CA2107412C (en
Inventor
Jay V. Clark
William L. Bramlett, Jr.
Judith Moyer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NCS Pearson Inc
Original Assignee
National Computer Systems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=21763965&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=CA2107412(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by National Computer Systems Inc filed Critical National Computer Systems Inc
Publication of CA2107412A1 publication Critical patent/CA2107412A1/en
Application granted granted Critical
Publication of CA2107412C publication Critical patent/CA2107412C/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/06Electrically-operated teaching apparatus or devices working with questions and answers of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers
    • G09B7/066Electrically-operated teaching apparatus or devices working with questions and answers of the multiple-choice answer-type, i.e. where a given question is provided with a series of answers and a choice has to be made from the answers with answer indicating cards, blocks
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S706/00Data processing: artificial intelligence
    • Y10S706/902Application using ai with detail of the ai system
    • Y10S706/927Education or instruction

Abstract

A system for managing resolvers of test answers to maintain consistent and efficient scoring of test answers. Collaborative scoring allows two test resolvers to score the same test answer and resolves potential discrepancies in the scores provided by the resolvers. Quality items monitor a test resolver's scoring performance against a known standard. The system transmits a quality-control test item, having a known score, to a test resolver. After receiving a score from the resolver, the system compares the known score with the resolver's score and can generate a quality profile for the resolver.
CA002107412A 1993-02-05 1993-09-30 Collaborative and quality control scoring system Expired - Lifetime CA2107412C (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US08/014,176 US5437554A (en) 1993-02-05 1993-02-05 System for providing performance feedback to test resolvers
US08/014,176 1993-02-05

Publications (2)

Publication Number Publication Date
CA2107412A1 true CA2107412A1 (en) 1994-08-06
CA2107412C CA2107412C (en) 2004-01-20

Family

ID=21763965

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002107412A Expired - Lifetime CA2107412C (en) 1993-02-05 1993-09-30 Collaborative and quality control scoring system

Country Status (5)

Country Link
US (21) US5437554A (en)
CA (1) CA2107412C (en)
DE (1) DE4403497B4 (en)
GB (1) GB2275121B (en)
HK (3) HK1002467A1 (en)

Families Citing this family (161)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5987149A (en) * 1992-07-08 1999-11-16 Uniscore Incorporated Method for scoring and control of scoring open-ended assessments using scorers in diverse locations
US5672060A (en) * 1992-07-08 1997-09-30 Meadowbrook Industries, Ltd. Apparatus and method for scoring nonobjective assessment materials through the application and use of captured images
US5437554A (en) 1993-02-05 1995-08-01 National Computer Systems, Inc. System for providing performance feedback to test resolvers
US5772213A (en) * 1995-06-07 1998-06-30 Mcglew; John James Game based on data base of characters of different geographic regions
US5618043A (en) * 1995-06-07 1997-04-08 Mcglew; John J. Game based on data base of characters of different geographic regions
US6041311A (en) * 1995-06-30 2000-03-21 Microsoft Corporation Method and apparatus for item recommendation using automated collaborative filtering
US5905865A (en) 1995-10-30 1999-05-18 Web Pager, Inc. Apparatus and method of automatically accessing on-line services in response to broadcast of on-line addresses
US7483670B2 (en) * 1996-05-09 2009-01-27 Walker Digital, Llc Method and apparatus for educational testing
US6014134A (en) * 1996-08-23 2000-01-11 U S West, Inc. Network-based intelligent tutoring system
US5855011A (en) * 1996-09-13 1998-12-29 Tatsuoka; Curtis M. Method for classifying test subjects in knowledge and functionality states
US5909589A (en) * 1996-11-12 1999-06-01 Lance T. Parker Internet based training
IL119746A (en) * 1996-12-03 2000-06-01 Ergolight Ltd Computerized apparatus and methods for identifying usability problems of a computerized system
AU5133398A (en) 1996-12-03 1998-06-29 Ergolight Ltd. Computerized apparatus and methods for identifying usability problems of a computerized system
US6295439B1 (en) 1997-03-21 2001-09-25 Educational Testing Service Methods and systems for presentation and evaluation of constructed responses assessed by human evaluators
US5991595A (en) * 1997-03-21 1999-11-23 Educational Testing Service Computerized system for scoring constructed responses and methods for training, monitoring, and evaluating human rater's scoring of constructed responses
WO1998043222A1 (en) * 1997-03-21 1998-10-01 Educational Testing Service System and method for evaluating raters of scoring constructed responses to test questions
GB2337844B (en) * 1997-03-21 2001-07-11 Educational Testing Service System and method for on-line essay evaluation
US6259890B1 (en) * 1997-03-27 2001-07-10 Educational Testing Service System and method for computer based test creation
US6250930B1 (en) * 1997-05-30 2001-06-26 Picante Communications Corporation Multi-functional communication and aggregation platform
US6120299A (en) * 1997-06-06 2000-09-19 Educational Testing Service System and method for interactive scoring of standardized test responses
US6175841B1 (en) * 1997-07-17 2001-01-16 Bookette Software Company Computerized systems for producing on-line instructional materials
US5978836A (en) 1997-07-28 1999-11-02 Solectron Corporation Workflow systems and methods
US7546346B2 (en) 1997-07-28 2009-06-09 Juniper Networks, Inc. Workflow systems and methods for project management and information management
US6311040B1 (en) * 1997-07-31 2001-10-30 The Psychological Corporation System and method for scoring test answer sheets having open-ended questions
US6173154B1 (en) 1997-07-31 2001-01-09 The Psychological Corporation System and method for imaging test answer sheets having open-ended questions
US6347943B1 (en) 1997-10-20 2002-02-19 Vuepoint Corporation Method and system for creating an individualized course of instruction for each user
US6267601B1 (en) 1997-12-05 2001-07-31 The Psychological Corporation Computerized system and method for teaching and assessing the holistic scoring of open-ended questions
US5957699A (en) * 1997-12-22 1999-09-28 Scientific Learning Corporation Remote computer-assisted professionally supervised teaching system
US6293801B1 (en) 1998-01-23 2001-09-25 Scientific Learning Corp. Adaptive motivation for computer-assisted training system
US6120298A (en) * 1998-01-23 2000-09-19 Scientific Learning Corp. Uniform motivation for multiple computer-assisted training systems
US6113645A (en) * 1998-04-22 2000-09-05 Scientific Learning Corp. Simulated play of interactive multimedia applications for error detection
US6067638A (en) * 1998-04-22 2000-05-23 Scientific Learning Corp. Simulated play of interactive multimedia applications for error detection
US6042384A (en) * 1998-06-30 2000-03-28 Bookette Software Company Computerized systems for optically scanning and electronically scoring and reporting test results
US6471521B1 (en) 1998-07-31 2002-10-29 Athenium, L.L.C. System for implementing collaborative training and online learning over a computer network and related techniques
US6604131B1 (en) * 1999-04-22 2003-08-05 Net Shepherd, Inc. Method and system for distributing a work process over an information network
US6288753B1 (en) 1999-07-07 2001-09-11 Corrugated Services Corp. System and method for live interactive distance learning
US6654748B1 (en) 1999-12-07 2003-11-25 Rwd Technologies, Inc. Dynamic application browser and database for use therewith
US6452679B1 (en) 1999-12-29 2002-09-17 Kimberly-Clark Worldwide, Inc. Method and apparatus for controlling the manufacturing quality of a moving web
US7526440B2 (en) * 2000-06-12 2009-04-28 Walker Digital, Llc Method, computer product, and apparatus for facilitating the provision of opinions to a shopper from a panel of peers
US20020049738A1 (en) * 2000-08-03 2002-04-25 Epstein Bruce A. Information collaboration and reliability assessment
US7573616B2 (en) * 2000-08-11 2009-08-11 Ctb/Mcgraw-Hill Llc Enhanced data capture from imaged documents
JP3580497B2 (en) * 2000-09-22 2004-10-20 インターナショナル・ビジネス・マシーンズ・コーポレーション Company diagnosis system, company diagnosis method, company diagnosis server, and recording medium
US6547568B1 (en) * 2000-10-12 2003-04-15 Kiyokazu Yamano Education intermediary system and method
US6461166B1 (en) * 2000-10-17 2002-10-08 Dennis Ray Berman Learning system with learner-constructed response based testing methodology
US6704741B1 (en) 2000-11-02 2004-03-09 The Psychological Corporation Test item creation and manipulation system and method
US7409700B1 (en) * 2000-11-03 2008-08-05 The Walt Disney Company System and method for enhanced broadcasting and interactive
US6551107B1 (en) 2000-11-03 2003-04-22 Cardioconcepts, Inc. Systems and methods for web-based learning
JP2004529406A (en) 2000-11-10 2004-09-24 アフィノバ, インコーポレイテッド Method and apparatus for dynamic real-time market segmentation
US6988895B1 (en) 2001-01-12 2006-01-24 Ncs Pearson, Inc. Electronic test item display as an image with overlay controls
GB2388699A (en) * 2001-01-23 2003-11-19 Educational Testing Service Methods for automated essay analysis
US20060014129A1 (en) * 2001-02-09 2006-01-19 Grow.Net, Inc. System and method for processing test reports
US6577846B2 (en) * 2001-02-12 2003-06-10 Ctb-Mcgraw Hill, Llc Methods for range finding of open-ended assessments
US6675133B2 (en) 2001-03-05 2004-01-06 Ncs Pearsons, Inc. Pre-data-collection applications test processing system
US6751351B2 (en) 2001-03-05 2004-06-15 Nsc Pearson, Inc. Test question response verification system
US6961482B2 (en) * 2001-03-05 2005-11-01 Ncs Pearson, Inc. System for archiving electronic images of test question responses
US6810232B2 (en) 2001-03-05 2004-10-26 Ncs Pearson, Inc. Test processing workflow tracking system
US6688889B2 (en) 2001-03-08 2004-02-10 Boostmyscore.Com Computerized test preparation system employing individually tailored diagnostics and remediation
US6928260B2 (en) * 2001-04-10 2005-08-09 Childcare Education Institute, Llc Online education system and method
US7074128B2 (en) 2001-08-03 2006-07-11 Drb Lit Ltd. Method and system for enhancing memorization by using a mnemonic display
US7922494B2 (en) 2001-08-28 2011-04-12 International Business Machines Corporation Method for improved administering of tests using customized user alerts
US20030064354A1 (en) * 2001-09-28 2003-04-03 Lewis Daniel M. System and method for linking content standards, curriculum, instructions and assessment
US7632101B2 (en) 2001-10-05 2009-12-15 Vision Works Ip Corporation Method and apparatus for periodically questioning a user using a computer system or other device to facilitate memorization and learning of information
US10347145B1 (en) 2001-10-05 2019-07-09 Vision Works Ip Corporation Method and apparatus for periodically questioning a user using a computer system or other device to facilitate memorization and learning of information
US20030086116A1 (en) * 2001-11-05 2003-05-08 Hall John M. Method to automatically evaluate a hard copy response and immediately generate commentary based thereon
US7321858B2 (en) 2001-11-30 2008-01-22 United Negro College Fund, Inc. Selection of individuals from a pool of candidates in a competition system
US6690913B2 (en) * 2001-12-20 2004-02-10 Kabushiki Kaisha Toshiba Correction support apparatus, correction support method, correction support program, and correction support system
US7127208B2 (en) * 2002-01-23 2006-10-24 Educational Testing Service Automated annotation
US7162198B2 (en) * 2002-01-23 2007-01-09 Educational Testing Service Consolidated Online Assessment System
US20030180703A1 (en) * 2002-01-28 2003-09-25 Edusoft Student assessment system
US8374540B2 (en) 2002-03-15 2013-02-12 Educational Testing Service Consolidated on-line assessment system
US20030232317A1 (en) * 2002-04-22 2003-12-18 Patz Richard J. Method of presenting an assessment
US20030207246A1 (en) * 2002-05-01 2003-11-06 Scott Moulthrop Assessment and monitoring system and method for scoring holistic questions
US8892895B1 (en) 2002-05-07 2014-11-18 Data Recognition Corporation Integrated system for electronic tracking and control of documents
US6772081B1 (en) * 2002-05-21 2004-08-03 Data Recognition Corporation Priority system and method for processing standardized tests
US7088949B2 (en) * 2002-06-24 2006-08-08 Educational Testing Service Automated essay scoring
US7149468B2 (en) * 2002-07-25 2006-12-12 The Mcgraw-Hill Companies, Inc. Methods for improving certainty of test-taker performance determinations for assessments with open-ended items
US8128414B1 (en) 2002-08-20 2012-03-06 Ctb/Mcgraw-Hill System and method for the development of instructional and testing materials
US20040064472A1 (en) * 2002-09-27 2004-04-01 Oetringer Eugen H. Method and system for information management
US7062536B2 (en) * 2002-10-10 2006-06-13 International Business Machines Corporation Grouping electronic reply messages
DE10258953A1 (en) * 2002-12-16 2004-07-22 S&C Polymer Silicon- und Composite-Spezialitäten GmbH Dispenser for fluid substances
US8385811B1 (en) 2003-02-11 2013-02-26 Data Recognition Corporation System and method for processing forms using color
US20040202987A1 (en) * 2003-02-14 2004-10-14 Scheuring Sylvia Tidwell System and method for creating, assessing, modifying, and using a learning map
US6993818B2 (en) * 2003-03-17 2006-02-07 Memx, Inc. Multi-fixture assembly of cutting tools
US20040223648A1 (en) * 2003-05-05 2004-11-11 Keith Hoene Determining differences between documents
US7357640B2 (en) * 2003-07-02 2008-04-15 Drb Lit Ltd. Lock-In Training system
US6876098B1 (en) * 2003-09-25 2005-04-05 The United States Of America As Represented By The Administrator Of The Environmental Protection Agency Methods of operating a series hybrid vehicle
TWM249950U (en) * 2004-01-02 2004-11-11 Cvc Technologies Inc Cap tightening machine capable of controlling tightening torque
US20050196742A1 (en) * 2004-03-04 2005-09-08 Harber Jonathan D. System and method for data analysis and presentation
US7364432B2 (en) * 2004-03-31 2008-04-29 Drb Lit Ltd. Methods of selecting Lock-In Training courses and sessions
US8155578B2 (en) * 2004-05-14 2012-04-10 Educational Testing Service Method and system for generating and processing an assessment examination
US7980855B1 (en) 2004-05-21 2011-07-19 Ctb/Mcgraw-Hill Student reporting systems and methods
US7657220B2 (en) * 2004-05-21 2010-02-02 Ordinate Corporation Adaptive scoring of responses to constructed response questions
US20060004621A1 (en) * 2004-06-30 2006-01-05 Malek Kamal M Real-time selection of survey candidates
US20060003306A1 (en) * 2004-07-02 2006-01-05 Mcginley Michael P Unified web-based system for the delivery, scoring, and reporting of on-line and paper-based assessments
US7606421B2 (en) * 2004-12-08 2009-10-20 Ctb/Mcgraw-Hill Llc Data extraction from temporal image data
JP4872214B2 (en) * 2005-01-19 2012-02-08 富士ゼロックス株式会社 Automatic scoring device
US7330716B2 (en) 2005-01-21 2008-02-12 Responsive Innovations, Llc Wireless communication system
US8041347B2 (en) 2005-01-21 2011-10-18 Responsive Innovations, Llc Transmitter controlled communication links
US8202098B2 (en) 2005-02-28 2012-06-19 Educational Testing Service Method of model scaling for an automated essay scoring system
US20060252023A1 (en) * 2005-05-03 2006-11-09 Lexmark International, Inc. Methods for automatically identifying user selected answers on a test sheet
US7791756B2 (en) * 2005-05-03 2010-09-07 Lexmark International, Inc. Methods for identifying marks using a digital master document and scanned image enhancement
US20070042335A1 (en) * 2005-05-11 2007-02-22 Ctb Mcgraw-Hill System and method for assessment or survey response collection using a remote, digitally recording user input device
US8170466B2 (en) * 2005-05-27 2012-05-01 Ctb/Mcgraw-Hill System and method for automated assessment of constrained constructed responses
US20060286537A1 (en) * 2005-05-31 2006-12-21 Mandella George V System and method for improving performance using practice tests
US20070031801A1 (en) * 2005-06-16 2007-02-08 Ctb Mcgraw Hill Patterned response system and method
US7761843B2 (en) * 2005-07-01 2010-07-20 Ronald Ortscheid Computer source code generator
CN101300613A (en) * 2005-07-20 2008-11-05 奥迪纳特公司 Spoken language proficiency assessment by computer
US20070048723A1 (en) * 2005-08-19 2007-03-01 Caveon, Llc Securely administering computerized tests over a network
US20070111182A1 (en) * 2005-10-26 2007-05-17 International Business Machines Corporation Method and system for distributing answers
US20070218450A1 (en) * 2006-03-02 2007-09-20 Vantage Technologies Knowledge Assessment, L.L.C. System for obtaining and integrating essay scoring from multiple sources
JP2007239511A (en) * 2006-03-06 2007-09-20 Denso Corp Drive control device for vehicle
US8608477B2 (en) * 2006-04-06 2013-12-17 Vantage Technologies Knowledge Assessment, L.L.C. Selective writing assessment with tutoring
US20070292835A1 (en) * 2006-06-06 2007-12-20 Clement Edwin Hartman Method for reporting student relevant data
US20080108298A1 (en) * 2006-11-07 2008-05-08 Selen Mats A Certified two way source initiated transfer
US7831195B2 (en) * 2006-12-11 2010-11-09 Sharp Laboratories Of America, Inc. Integrated paper and computer-based testing administration system
US20080227080A1 (en) * 2007-03-14 2008-09-18 Gekl Technologies, Inc. Student assessment system
US20080227075A1 (en) 2007-03-15 2008-09-18 Ctb/Mcgraw-Hill, Llc Method and system for redundant data capture from scanned documents
US20080280280A1 (en) * 2007-05-11 2008-11-13 Aplia, Inc. Method of capturing workflow
US20090015875A1 (en) * 2007-06-20 2009-01-15 Ctb/Mcgraw-Hill Companies, Inc. Image manipulation of digitized images of documents
US9311581B2 (en) * 2007-10-19 2016-04-12 Daniel W. Shinn System and method for tracking the fulfillment status of requirements for completing an objective
US8738659B1 (en) * 2007-10-22 2014-05-27 Data Recognition Corporation Method and apparatus for managing priority in standardized test and survey imaging
US8488220B1 (en) 2007-10-22 2013-07-16 Data Recognition Corporation Method and apparatus for calibrating imaging equipment
US8649601B1 (en) 2007-10-22 2014-02-11 Data Recognition Corporation Method and apparatus for verifying answer document images
US9195875B1 (en) 2007-10-22 2015-11-24 Data Recognition Corporation Method and apparatus for defining fields in standardized test imaging
US8526055B1 (en) 2007-10-22 2013-09-03 Data Recognition Corporation Standardized test and survey imaging system
US8526766B2 (en) 2007-10-31 2013-09-03 Ctb/Mcgraw-Hill Llc Use of composite bitmapped images in conjunction with display of captured data
US9477809B2 (en) * 2007-12-27 2016-10-25 James G. Marx Systems and methods for workflow processing
US20090198504A1 (en) * 2008-02-05 2009-08-06 Medavante, Inc. Rater resource allocation systems and methods
US8376755B2 (en) * 2008-05-09 2013-02-19 Location Inc. Group Corporation System for the normalization of school performance statistics
GB0809159D0 (en) * 2008-05-20 2008-06-25 Chalkface Project Ltd Education tool
GB2462612A (en) * 2008-08-12 2010-02-17 Tag Learning Ltd A method of facilitating assessment of a coursework answer
US20100042422A1 (en) * 2008-08-15 2010-02-18 Adam Summers System and method for computing and displaying a score with an associated visual quality indicator
US20100257832A1 (en) * 2009-04-14 2010-10-14 Smith Robert L Mower Blade
US8782063B2 (en) 2009-10-20 2014-07-15 Universal Research Solutions, Llc Generation and data management of a medical study using instruments in an integrated media and medical system
US8429547B2 (en) 2009-10-20 2013-04-23 Universal Research Solutions, Llc Generation and data management of a medical study using instruments in an integrated media and medical system
EP2362180B1 (en) * 2010-02-05 2016-01-27 Poly-clip System GmbH & Co. KG Method for measuring a storage frame
US8774704B2 (en) * 2010-03-10 2014-07-08 Michael & Susan Dell Foundation Virtual reader for processing applications
US8200675B2 (en) * 2010-03-10 2012-06-12 Michael & Susan Dell Foundation Virtual reader for scoring applications
US20110269110A1 (en) * 2010-05-03 2011-11-03 Mcclellan Catherine Computer-Implemented Systems and Methods for Distributing Constructed Responses to Scorers
US8521077B2 (en) * 2010-07-21 2013-08-27 Xerox Corporation System and method for detecting unauthorized collaboration on educational assessments
US20120053967A1 (en) * 2010-09-01 2012-03-01 American Express Travel Related Services Company, Inc. System and Method for Account Reconciliation
US9679256B2 (en) 2010-10-06 2017-06-13 The Chancellor, Masters And Scholars Of The University Of Cambridge Automated assessment of examination scripts
US8761658B2 (en) 2011-01-31 2014-06-24 FastTrack Technologies Inc. System and method for a computerized learning system
US9208132B2 (en) 2011-03-08 2015-12-08 The Nielsen Company (Us), Llc System and method for concept development with content aware text editor
US8868446B2 (en) 2011-03-08 2014-10-21 Affinnova, Inc. System and method for concept development
US20120259676A1 (en) 2011-04-07 2012-10-11 Wagner John G Methods and apparatus to model consumer choice sourcing
US20130171594A1 (en) * 2011-12-30 2013-07-04 Pepper Technology LLC Systems and methods for providing training and collaborative activities through a group-based training and evaluation platform
US9311383B1 (en) 2012-01-13 2016-04-12 The Nielsen Company (Us), Llc Optimal solution identification system and method
WO2014104620A1 (en) * 2012-12-24 2014-07-03 주식회사 테라클 Method and apparatus for managing learning contents
US9928383B2 (en) 2014-10-30 2018-03-27 Pearson Education, Inc. Methods and systems for network-based analysis, intervention, and anonymization
US9483954B2 (en) 2013-03-12 2016-11-01 Pearson Education, Inc. Educational network based intervention
US10516691B2 (en) 2013-03-12 2019-12-24 Pearson Education, Inc. Network based intervention
WO2014143729A1 (en) 2013-03-15 2014-09-18 Affinnova, Inc. Method and apparatus for interactive evolutionary optimization of concepts
WO2014152010A1 (en) 2013-03-15 2014-09-25 Affinnova, Inc. Method and apparatus for interactive evolutionary algorithms with respondent directed breeding
WO2014169288A1 (en) 2013-04-12 2014-10-16 Pearson Education, Inc. Evaluation control
US10147108B2 (en) 2015-04-02 2018-12-04 The Nielsen Company (Us), Llc Methods and apparatus to identify affinity between segment attributes and product characteristics
US10129131B2 (en) * 2015-06-30 2018-11-13 Dell Products, Lp System and method for device optimization in a network of devices with embedded electronics
US10471594B2 (en) * 2015-12-01 2019-11-12 Kindred Systems Inc. Systems, devices, and methods for the distribution and collection of multimodal data associated with robots
RU171340U1 (en) * 2016-11-18 2017-05-29 Федеральное государственное бюджетное учреждение "4 Центральный научно-исследовательский институт" Министерства обороны Российской Федерации DEVICE FOR TESTING TRAINERS
US10984196B2 (en) 2018-01-11 2021-04-20 International Business Machines Corporation Distributed system for evaluation and feedback of digital text-based content
US11563576B2 (en) 2019-01-23 2023-01-24 International Business Machines Corporation Distributed anonymous scoring technique

Family Cites Families (109)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US34476A (en) * 1862-02-25 Improvement in hose-couplings
US3405457A (en) * 1965-10-23 1968-10-15 Univ Illinois Versatile display teaching system
US3538626A (en) * 1968-02-16 1970-11-10 Response Systems Corp Student-responder apparatus
US3762072A (en) * 1971-07-16 1973-10-02 Lerali Sa Apparatus for the individual and automatic monitoring of replies given under examination
JPS5147091B2 (en) 1971-09-30 1976-12-13
US3932948A (en) 1973-04-18 1976-01-20 Eastman Kodak Company Audio-visual learning system
US3981087A (en) * 1973-10-11 1976-09-21 Sachs Thomas D Teaching machine
US4205780A (en) * 1977-03-21 1980-06-03 Teknekron, Inc. Document processing system and method
US4151659A (en) * 1978-06-07 1979-05-01 Eric F. Burtis Machine for teaching reading
DE3009211C2 (en) * 1980-03-11 1983-08-18 Volker Prof. Dr. 6905 Schriesheim Hepp Mobile event module
US4358277A (en) * 1981-06-26 1982-11-09 Sunbelt Communications, Ltd. Method and apparatus for conducting an opinion survey
US4518361A (en) * 1982-08-05 1985-05-21 Conway Malcolm J Method and apparatus for effecting and evaluating action upon visual imaging
US4798543A (en) * 1983-03-31 1989-01-17 Bell & Howell Company Interactive training method and system
US4553261A (en) * 1983-05-31 1985-11-12 Horst Froessl Document and data handling and retrieval system
US4478584A (en) * 1983-08-26 1984-10-23 Kathryn Kaney Method for recording uncoded scores or ratings by multiple evaluators on a single automatic machine scannable document
US4705479A (en) 1984-07-09 1987-11-10 Maron Melvin J Process of teaching
US4899276A (en) * 1984-08-14 1990-02-06 International Business Machines Corporation Field-directed screen help technique
DE3529301A1 (en) 1984-08-28 1986-03-27 Jost von Dr. Grüningen Fellenberg Psychotechnical method and device for conducting the method
US4694352A (en) 1984-09-21 1987-09-15 Canon Kabushiki Kaisha Image information processing system
US4789543A (en) * 1984-11-30 1988-12-06 E. I. Du Pont De Nemours And Company Neutral technetium 99-m complexes useful as radiodiagnostic agents
US4715818A (en) 1984-12-04 1987-12-29 Ronald Lewis Shapiro Computer training system
US4648062A (en) * 1985-02-01 1987-03-03 International Business Machines Corporation Method for providing an on line help facility for interactive information handling systems
US4708503A (en) 1985-03-21 1987-11-24 Meadowbrook Industries, Inc. Work carrier for use in computer printing
US4845739A (en) * 1985-07-10 1989-07-04 Fdr Interactive Technologies Telephonic-interface statistical analysis system
US4908759A (en) * 1985-08-29 1990-03-13 Bell Communications Research, Inc. Hierarchical database conversion with conditional write
JPS6275578A (en) 1985-09-30 1987-04-07 株式会社東芝 Marking device
US4671772A (en) * 1985-10-22 1987-06-09 Keilty, Goldsmith & Boone Performance appraisal and training system and method of utilizing same
US4741047A (en) 1986-03-20 1988-04-26 Computer Entry Systems Corporation Information storage, retrieval and display system
US4764120A (en) 1986-05-27 1988-08-16 Mcdonald's Corporation Student response system
US4930077A (en) * 1987-04-06 1990-05-29 Fan David P Information processing expert system for text analysis and predicting public opinion based information available to the public
US4760246A (en) 1987-04-20 1988-07-26 Cognitronics Corporation Mark-reading apparatus for use with answer sheets
US4785472A (en) * 1987-05-11 1988-11-15 The Trustees Of The Stevens Institute Of Technology Remote teaching system
US4867685A (en) * 1987-09-24 1989-09-19 The Trustees Of The College Of Aeronautics Audio visual instructional system
US4996642A (en) * 1987-10-01 1991-02-26 Neonics, Inc. System and method for recommending items
US4895518A (en) * 1987-11-02 1990-01-23 The University Of Michigan Computerized diagnostic reasoning evaluation system
US4878175A (en) * 1987-11-03 1989-10-31 Emtek Health Care Systems Method for generating patient-specific flowsheets by adding/deleting parameters
US5147205A (en) * 1988-01-29 1992-09-15 Gross Theodore D Tachistoscope and method of use thereof for teaching, particularly of reading and spelling
US4937439A (en) 1988-05-13 1990-06-26 National Computer Systems, Inc. Method and system for creating and scanning a customized survey form
US5058185A (en) 1988-06-27 1991-10-15 International Business Machines Corporation Object management and delivery system having multiple object-resolution capability
US5228116A (en) * 1988-07-15 1993-07-13 Aicorp., Inc. Knowledge base management system
US5100329A (en) * 1990-06-22 1992-03-31 Deesen Kenneth C Computer assisted coaching method
US5054096A (en) 1988-10-24 1991-10-01 Empire Blue Cross/Blue Shield Method and apparatus for converting documents into electronic data for transaction processing
FR2638415B1 (en) * 1988-11-02 1991-01-11 Sebarro Mottas Engineering Sa MOTORIZED OR TRACT VEHICLE WITH TWO BEARING STEERING WHEEL
AU4649589A (en) * 1988-11-10 1990-06-12 Response Technologies, Inc. Expert data collection, analysis, and response system and method
US5072383A (en) * 1988-11-19 1991-12-10 Emtek Health Care Systems, Inc. Medical information system with automatic updating of task list in response to entering orders and charting interventions on associated forms
US4958284A (en) * 1988-12-06 1990-09-18 Npd Group, Inc. Open ended question analysis system and method
US5003613A (en) 1988-12-21 1991-03-26 Recognition Equipment Incorporated Document processing system and method
US5105354A (en) * 1989-01-23 1992-04-14 Nippon Kayaku Kabushiki Kaisha Method and apparatus for correlating respiration and heartbeat variability
US5086385A (en) 1989-01-31 1992-02-04 Custom Command Systems Expandable home automation system
US5140650A (en) 1989-02-02 1992-08-18 International Business Machines Corporation Computer-implemented method for automatic extraction of data from printed forms
US5002491A (en) * 1989-04-28 1991-03-26 Comtek Electronic classroom system enabling interactive self-paced learning
JPH031709A (en) 1989-05-30 1991-01-08 Toyo Commun Equip Co Ltd Variable resistance circuit and variable gain amplifier
US4975865A (en) * 1989-05-31 1990-12-04 Mitech Corporation Method and apparatus for real-time control
US4978305A (en) * 1989-06-06 1990-12-18 Educational Testing Service Free response test grading method
US5011413A (en) * 1989-07-19 1991-04-30 Educational Testing Service Machine-interpretable figural response testing
US5211564A (en) 1989-07-19 1993-05-18 Educational Testing Service Computerized figural response testing system and method
US5035625A (en) 1989-07-24 1991-07-30 Munson Electronics, Inc. Computer game teaching method and system
CA1321026C (en) 1989-09-28 1993-08-03 Arny I. Sokoloff Method and apparatus for optically reading pre-printed survey pages
US5059127A (en) * 1989-10-26 1991-10-22 Educational Testing Service Computerized mastery testing system, a computer administered variable length sequential testing system for making pass/fail decisions
US5204812A (en) * 1989-11-02 1993-04-20 International Business Machines Corp. User access of multiple documents based on document relationship classification
US5318450A (en) 1989-11-22 1994-06-07 Gte California Incorporated Multimedia distribution system for instructional materials
US5344132A (en) 1990-01-16 1994-09-06 Digital Image Systems Image based document processing and information management system and apparatus
US5038392A (en) 1990-02-12 1991-08-06 International Business Machines Corporation Method and apparatus for adaptive image processing by recognizing a characterizing indicium in a captured image of a document
US5023435A (en) 1990-03-06 1991-06-11 Deniger David B Response form processing system
US5151948A (en) 1990-03-12 1992-09-29 International Business Machines Corporation System and method for processing documents having amounts recorded thereon
US5119433A (en) 1990-03-12 1992-06-02 International Business Machines Corporation Method and system for locating the amount field on a document
US5176520A (en) 1990-04-17 1993-01-05 Hamilton Eric R Computer assisted instructional delivery system and method
JPH0414788A (en) * 1990-05-07 1992-01-20 Toshiba Lighting & Technol Corp Infrared ray heater and manufacture thereof
USRE34476E (en) 1990-05-14 1993-12-14 Norwood Donald D Hybrid information management system for handwriting and text
US5195033A (en) * 1990-06-08 1993-03-16 Assessment Systems, Inc. Testing system including removable storage means for transfer of test related data and means for issuing a certification upon successful completion of the test
US5204813A (en) * 1990-06-08 1993-04-20 Assessment Systems, Inc. Computer-controlled testing process and device for administering an examination
US5103490A (en) * 1990-06-13 1992-04-07 National Computer Systems, Inc. Method and apparatus for storing and merging multiple optically scanned images
US5134669A (en) * 1990-06-13 1992-07-28 National Computer Systems Image processing system for documentary data
JPH04147288A (en) 1990-10-11 1992-05-20 Brother Ind Ltd Corrected teaching material distribution device
US5180309A (en) * 1990-12-04 1993-01-19 United States Of America As Represented By The Secretary Of The Navy Automated answer evaluation and scoring system and method
JPH04241381A (en) * 1991-01-16 1992-08-28 Brother Ind Ltd Electronic learning machine
JPH04246791A (en) 1991-02-01 1992-09-02 Hitachi Ltd Questionnaire data collecting and analyzing system
US5258855A (en) 1991-03-20 1993-11-02 System X, L. P. Information processing methodology
US5437555A (en) 1991-05-02 1995-08-01 Discourse Technologies, Inc. Remote teaching system
US5634101A (en) 1991-05-31 1997-05-27 R. Alan Blau & Associates, Co. Method and apparatus for obtaining consumer information
US5195813A (en) * 1991-09-03 1993-03-23 Kc Hilites, Inc. Wireless control for auxiliary lighting
US5452379A (en) 1991-09-13 1995-09-19 Meadowbrook Industries, Ltd. Image capture and storage techniques in association with optical mark reading
JP2979772B2 (en) 1991-09-13 1999-11-15 松下電器産業株式会社 Die bonding equipment
US5259766A (en) * 1991-12-13 1993-11-09 Educational Testing Service Method and system for interactive computer science testing, anaylsis and feedback
US5376007A (en) 1991-12-19 1994-12-27 Zirm; Matthias Apparatus and method for teaching and learning microsurgical operating techniques
US5294229A (en) * 1992-01-27 1994-03-15 Jonathan Hartzell Teacher and parent interactive communication system incorporating pocket sized portable audio numeric terminals
US5379213A (en) 1992-01-28 1995-01-03 Fleetwood Furniture Company, Inc. Test scoring system and method
US5267865A (en) 1992-02-11 1993-12-07 John R. Lee Interactive computer aided natural learning method and apparatus
US5418865A (en) 1992-03-20 1995-05-23 Xerox Corporation Mark sensing on a form
US5302132A (en) 1992-04-01 1994-04-12 Corder Paul R Instructional system and method for improving communication skills
US5310349A (en) 1992-04-30 1994-05-10 Jostens Learning Corporation Instructional management system
US5672060A (en) 1992-07-08 1997-09-30 Meadowbrook Industries, Ltd. Apparatus and method for scoring nonobjective assessment materials through the application and use of captured images
US5987149A (en) * 1992-07-08 1999-11-16 Uniscore Incorporated Method for scoring and control of scoring open-ended assessments using scorers in diverse locations
US5999908A (en) 1992-08-06 1999-12-07 Abelow; Daniel H. Customer-based product design module
US5565316A (en) 1992-10-09 1996-10-15 Educational Testing Service System and method for computer based testing
US5416312A (en) * 1992-11-20 1995-05-16 Cherloc Document bearing an image or a text and provided with an indexing frame, and associated document analysis system
WO1994015272A1 (en) 1992-12-22 1994-07-07 Morgan Michael W Pen-based electronic teaching system
US5437554A (en) * 1993-02-05 1995-08-01 National Computer Systems, Inc. System for providing performance feedback to test resolvers
US5433615A (en) 1993-02-05 1995-07-18 National Computer Systems, Inc. Categorized test item reporting system
US5321611A (en) * 1993-02-05 1994-06-14 National Computer Systems, Inc. Multiple test scoring system
US5544255A (en) 1994-08-31 1996-08-06 Peripheral Vision Limited Method and system for the capture, storage, transport and authentication of handwritten signatures
US5848393A (en) * 1995-12-15 1998-12-08 Ncr Corporation "What if . . . " function for simulating operations within a task workflow management system
US5691895A (en) * 1995-12-18 1997-11-25 International Business Machines Corporation Mechanism and architecture for manufacturing control and optimization
US6157921A (en) * 1998-05-01 2000-12-05 Barnhill Technologies, Llc Enhancing knowledge discovery using support vector machines in a distributed network environment
US5991595A (en) 1997-03-21 1999-11-23 Educational Testing Service Computerized system for scoring constructed responses and methods for training, monitoring, and evaluating human rater's scoring of constructed responses
GB2337844B (en) 1997-03-21 2001-07-11 Educational Testing Service System and method for on-line essay evaluation
US6311040B1 (en) 1997-07-31 2001-10-30 The Psychological Corporation System and method for scoring test answer sheets having open-ended questions
US6042384A (en) * 1998-06-30 2000-03-28 Bookette Software Company Computerized systems for optically scanning and electronically scoring and reporting test results
KR20000057105A (en) * 1998-12-28 2000-09-15 오오자와 슈지로 An apparatus for sterilizing a water-soluble lubricant

Also Published As

Publication number Publication date
US5716213A (en) 1998-02-10
US6976847B2 (en) 2005-12-20
US6183260B1 (en) 2001-02-06
US6159018A (en) 2000-12-12
US5752836A (en) 1998-05-19
US5718591A (en) 1998-02-17
US6168440B1 (en) 2001-01-02
US6558166B1 (en) 2003-05-06
US7044743B2 (en) 2006-05-16
US6193521B1 (en) 2001-02-27
GB9402185D0 (en) 1994-03-30
DE4403497A1 (en) 1994-09-01
US5466159A (en) 1995-11-14
US6155839A (en) 2000-12-05
HK1002467A1 (en) 1998-08-28
US6749435B2 (en) 2004-06-15
US20030198933A1 (en) 2003-10-23
US6918772B2 (en) 2005-07-19
US20040063087A1 (en) 2004-04-01
US5437554A (en) 1995-08-01
US20040086841A1 (en) 2004-05-06
DE4403497B4 (en) 2006-11-23
HK1003280A1 (en) 1998-10-23
US5709551A (en) 1998-01-20
US5458493A (en) 1995-10-17
US5690497A (en) 1997-11-25
US5558521A (en) 1996-09-24
HK1003496A1 (en) 1998-10-30
CA2107412C (en) 2004-01-20
US20040202991A1 (en) 2004-10-14
US5735694A (en) 1998-04-07
GB2275121A (en) 1994-08-17
US6183261B1 (en) 2001-02-06
GB2275121B (en) 1997-11-26

Similar Documents

Publication Publication Date Title
CA2107412A1 (en) Collaborative and quality control scoring system
CA2145675A1 (en) Collaborative work system
CA2230230A1 (en) Method and apparatus for providing identification
CA2107413A1 (en) Categorized test item reporting system and method
CA2191197A1 (en) Method and means for the production of hyaluronic acid
CA2118885A1 (en) Process control system
AU4496496A (en) Weldable high-tensile steel excellent in low-temperature toughness
CA2040614A1 (en) Computer-assisted instructional delivery system
CA2123685A1 (en) Medicament for promoting growth of mammalian nerve
CA2187979A1 (en) Process for preparing nicotinamide
CA2074454A1 (en) Medicine based on neutralized sulphur derivatives
ES2099331T3 (en) ESTERS OF PIRIDYLOXY-ACRYLIC ACID.
HU906417D0 (en) Process for producing 3-(1-pyroglutamyl)-1-thiazolidine-4-carboxylic acid and its derivatives
GB9220327D0 (en) Process
AU6665294A (en) Stable emulsifiable gel matrix for forming gel concentrate with agricultural chemical, and aqueous macroemulsion therefrom
IE900058L (en) New substituted pyrido(2,3-d)pyrimidines
WO1994008586A3 (en) Azido derivatives of cyclopentane heptanoic or heptenoic acid
CA2169219A1 (en) Substituted benzenesulfonylureas and -thioureas, processes for their preparation, their use for the production of pharmaceutical preparations, and pharmaceutical preparations containing them
HUT71793A (en) 2,4-diamino-3-hydroxy-carboxylic acid derivatives, pharmaceutical compns. contg. them and process for producing the said compds.
AU7693694A (en) Substituted pyridines, process for producing them and their use as pesticides and fungicides
CA2153903A1 (en) Process for the preparation of 10-desacetoxybaccatin iii
朱功勤 et al. ON THE CONVERGENCE OF PADE-TYPE APPROXIMANTS IN TWO VARIABLES
AU1076595A (en) Intestinal flora improver composition, and apparatus and system for producing the same
Martin Training to raise the standard
WO2000077672A3 (en) Method and system for skills assessment

Legal Events

Date Code Title Description
EEER Examination request