TY - JOUR
T1 - Early Detection Prediction of Learning Outcomes in Online Short-Courses via Learning Behaviors
AU - Chen, Weiyu
AU - Brinton, Christopher G.
AU - Cao, Da
AU - Mason-Singh, Amanda
AU - Lu, Charlton
AU - Chiang, Mung
N1 - Funding Information:
This version includes a more comprehensive set of algorithms, evaluation, and corresponding discussion. This work was in part supported by Zoomi Inc. The authors thank the anonymous reviewers for their valuable comments.
Publisher Copyright:
© 2008-2011 IEEE.
Copyright:
Copyright 2019 Elsevier B.V., All rights reserved.
PY - 2019/1/1
Y1 - 2019/1/1
N2 - We study learning outcome prediction for online courses. Whereas prior work has focused on semester-long courses with frequent student assessments, we focus on short-courses that have single outcomes assigned by instructors at the end. The lack of performance data and generally small enrollments makes the behavior of learners, captured as they interact with course content and with one another in Social Learning Networks (SLN), essential for prediction. Our method defines several (machine) learning features based on the processing of behaviors collected on the modes of (human) learning in a course, and uses them in appropriate classifiers. Through evaluation on data captured from three two-week courses hosted through our delivery platforms, we make three key observations: (i) behavioral data contains signals predictive of learning outcomes in short-courses (with classifiers achieving AUCs ≥ 0.8 after the two weeks), (ii) early detection is possible within the first week (AUCs ≥ 0.7 with the first week of data), and (iii) the content features have an 'earliest' detection capability (with higher AUC in the first few days), while the SLN features become the more predictive set over time as the network matures. We also discuss how our method can generate behavioral analytics for instructors.
AB - We study learning outcome prediction for online courses. Whereas prior work has focused on semester-long courses with frequent student assessments, we focus on short-courses that have single outcomes assigned by instructors at the end. The lack of performance data and generally small enrollments makes the behavior of learners, captured as they interact with course content and with one another in Social Learning Networks (SLN), essential for prediction. Our method defines several (machine) learning features based on the processing of behaviors collected on the modes of (human) learning in a course, and uses them in appropriate classifiers. Through evaluation on data captured from three two-week courses hosted through our delivery platforms, we make three key observations: (i) behavioral data contains signals predictive of learning outcomes in short-courses (with classifiers achieving AUCs ≥ 0.8 after the two weeks), (ii) early detection is possible within the first week (AUCs ≥ 0.7 with the first week of data), and (iii) the content features have an 'earliest' detection capability (with higher AUC in the first few days), while the SLN features become the more predictive set over time as the network matures. We also discuss how our method can generate behavioral analytics for instructors.
KW - Clickstream data
KW - data mining
KW - learning outcome prediction
KW - predictive learning analytics
KW - social learning networks.
UR - http://www.scopus.com/inward/record.url?scp=85041242053&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85041242053&partnerID=8YFLogxK
U2 - 10.1109/TLT.2018.2793193
DO - 10.1109/TLT.2018.2793193
M3 - Article
AN - SCOPUS:85041242053
VL - 12
SP - 44
EP - 58
JO - IEEE Transactions on Learning Technologies
JF - IEEE Transactions on Learning Technologies
SN - 1939-1382
IS - 1
M1 - 8259019
ER -