Systems and methods for automated machine learning model training quality control
Systems and methods for automated custom training of a scoring model are disclosed herein. The method include: receiving a plurality of responses received from a plurality of students in response to providing of a prompt; identifying an evaluation model relevant to the provided prompt, which evaluat...
Saved in:
Main Authors | , , , , , , , |
---|---|
Format | Patent |
Language | English |
Published |
16.01.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Systems and methods for automated custom training of a scoring model are disclosed herein. The method include: receiving a plurality of responses received from a plurality of students in response to providing of a prompt; identifying an evaluation model relevant to the provided prompt, which evaluation model can be a machine learning model trained to output a score relevant to at least portions of a response; generating a training indicator that provides a graphical depiction of the degree to which the identified evaluation model is trained; determining a training status of the model; receiving at least one evaluation input when the model is identified as insufficiently trained; updating training of the evaluation model based on the at least one received evaluation input; and controlling the training indicator to reflect the degree to which the evaluation model is trained subsequent to the updating of the training of the evaluation model. |
---|---|
AbstractList | Systems and methods for automated custom training of a scoring model are disclosed herein. The method include: receiving a plurality of responses received from a plurality of students in response to providing of a prompt; identifying an evaluation model relevant to the provided prompt, which evaluation model can be a machine learning model trained to output a score relevant to at least portions of a response; generating a training indicator that provides a graphical depiction of the degree to which the identified evaluation model is trained; determining a training status of the model; receiving at least one evaluation input when the model is identified as insufficiently trained; updating training of the evaluation model based on the at least one received evaluation input; and controlling the training indicator to reflect the degree to which the evaluation model is trained subsequent to the updating of the training of the evaluation model. |
Author | Baikadi, Alok Foltz, Peter Budden, Jill Rosenstein, Mark Habermehl, Kyle Hopkins, Stephen Becker, Lee Hellman, Scott |
Author_xml | – fullname: Habermehl, Kyle – fullname: Hellman, Scott – fullname: Baikadi, Alok – fullname: Foltz, Peter – fullname: Rosenstein, Mark – fullname: Hopkins, Stephen – fullname: Becker, Lee – fullname: Budden, Jill |
BookMark | eNqNiksKwjAQQLPQhb87jAcQrKJ1rShupbouQzK1gWSmJtNFb6-IB3D1eI83NSMWpom5VUNWihmQHUTSVlyGRhJgrxJR6VPRtp4JAmFiz0-I4iiAJvRfffUYvA5ghTVJmJtxgyHT4seZWV7O99N1RZ3UlDu0xKT1oyqKQ7kr1_vjZvvP8wZN-zo2 |
ContentType | Patent |
DBID | EVB |
DatabaseName | esp@cenet |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: EVB name: esp@cenet url: http://worldwide.espacenet.com/singleLineSearch?locale=en_EP sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Medicine Chemistry Sciences Education Physics |
ExternalDocumentID | US11875706B2 |
GroupedDBID | EVB |
ID | FETCH-epo_espacenet_US11875706B23 |
IEDL.DBID | EVB |
IngestDate | Fri Sep 20 10:12:59 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-epo_espacenet_US11875706B23 |
Notes | Application Number: US201916281014 |
OpenAccessLink | https://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20240116&DB=EPODOC&CC=US&NR=11875706B2 |
ParticipantIDs | epo_espacenet_US11875706B2 |
PublicationCentury | 2000 |
PublicationDate | 20240116 |
PublicationDateYYYYMMDD | 2024-01-16 |
PublicationDate_xml | – month: 01 year: 2024 text: 20240116 day: 16 |
PublicationDecade | 2020 |
PublicationYear | 2024 |
RelatedCompanies | Pearson Education, Inc |
RelatedCompanies_xml | – name: Pearson Education, Inc |
Score | 3.5052347 |
Snippet | Systems and methods for automated custom training of a scoring model are disclosed herein. The method include: receiving a plurality of responses received from... |
SourceID | epo |
SourceType | Open Access Repository |
SubjectTerms | ADVERTISING APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND,DEAF OR MUTE CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING CRYPTOGRAPHY DIAGRAMS DISPLAY EDUCATION EDUCATIONAL OR DEMONSTRATION APPLIANCES ELECTRIC DIGITAL DATA PROCESSING GLOBES PHYSICS PLANETARIA SEALS |
Title | Systems and methods for automated machine learning model training quality control |
URI | https://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20240116&DB=EPODOC&locale=&CC=US&NR=11875706B2 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LS8NAEB5Kfd40KlofrCC5BZMm2SSHICRpKEIf2lZ6K9l0I3pIi0kR_72z2631otdZGHYHZmdm95v5AO7aeVZgmUyNwMyZ4WTMNTKa-Qb1XTZnuT1vy_6KXp92J87j1J024H3TCyPnhH7K4YjoUTn6ey3v6-X2ESuR2Mrqnr2haPGQjsNEV9UxKrYsqidR2BkOkkGsx3E4Gen951CwarueSSO8rndEGi3m7HdeItGVsvwdUtIj2B2itrI-hgYvNTiIN8xrmmBSVqgLDfZ76vNbgz2J1swrFCqPrE7gSY0cJ1k5J2s66IpgIkqyVb3AbJSjVOIlOVEEEa9Est-QDTkEWfdVfhGFWj-F27QzjrsG7nj2Y57ZZLQ9nH0GzXJR8nMgIjKbvscK28EiLi-YjckIDTzX4kHAC_MCWn_raf23eAmHwtTiIcKiV9CsP1b8GkNzzW6kTb8BlU6Tvg |
link.rule.ids | 230,309,786,891,25594,76906 |
linkProvider | European Patent Office |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LT8MwDLam8Rg3KCAYryCh3iradU3XQ4W0dtOAvWAb2m1q2hTBoZtoJ8S_x8kyxgWujmQllhzbyWd_ADe1OEqxTKaGZ8bMqEfMMSIaNQzacFjCYjupyf6KXp92JvWHqTMtwfu6F0bOCf2UwxHRo2L090Le14vNI1YosZX5LXtD0fyuPfZDXVXHqNiyqB42_dZwEA4CPQj8yUjvP_uCVdtxTdrE63rLFdN5Rer00hRdKYvfIaW9D9tD1JYVB1DimQaVYM28pgkmZYW60GC3pz6_NdiRaM04R6HyyPwQntTIcRJlCVnRQecEE1ESLYs5ZqMcpRIvyYkiiHglkv2GrMkhyKqv8oso1PoRXLdb46Bj4I5nP-aZTUabw9nHUM7mGT8BIiKz2XBZatexiItTZmMyQj3Xsbjn8dQ8herfeqr_LV5BpTPudWfd-_7jGewJs4tHCYueQ7n4WPILDNMFu5T2_QansJar |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Apatent&rft.title=Systems+and+methods+for+automated+machine+learning+model+training+quality+control&rft.inventor=Habermehl%2C+Kyle&rft.inventor=Hellman%2C+Scott&rft.inventor=Baikadi%2C+Alok&rft.inventor=Foltz%2C+Peter&rft.inventor=Rosenstein%2C+Mark&rft.inventor=Hopkins%2C+Stephen&rft.inventor=Becker%2C+Lee&rft.inventor=Budden%2C+Jill&rft.date=2024-01-16&rft.externalDBID=B2&rft.externalDocID=US11875706B2 |