TY - JOUR
T1 - Predicting learning outcome in a first-year engineering course
T2 - 129th ASEE Annual Conference and Exposition: Excellence Through Diversity, ASEE 2022
AU - Castro, Laura Melissa Cruz
AU - Li, Tiantian
AU - Ciner, Leyla
AU - Douglas, Kerrie A.
AU - Brinton, Christopher Greg
N1 - Publisher Copyright:
© American Society for Engineering Education, 2022.
PY - 2022/8/23
Y1 - 2022/8/23
N2 - First-year engineering courses are relatively large with several sections; thus, it can be rather difficult for an individual instructor to recognize when a particular student begins to lose engagement. Learning management systems (LMS) (e.g., Canvas, Blackboard, Brightspace) can be valuable tools to provide a consistent curriculum across several sections of a course and generate data regarding students' engagement with course materials. However, a human-centered approach to transform the data needs to be utilized to extract valuable insights from LMS data. The purpose of this Complete Research paper is to explore the following research questions: What type of LMS objects contain information to explain students' grades in a first-year engineering course? Is the inclusion of a human operator during the data transformation process significant to the analysis of learning outcomes? For this, data from LMS is used to predict the learning outcome of students in a FYE course. Two predictive models are compared. The first model corresponds to a usual predictive model, using the data from the LMS directly. The second model considers the specifics of the course, by transforming the data from aggregate user interaction to more granular categories related to the content of the class by a human operator. A logistic regression model is fitted using both datasets. The comparison between predictive measures such as precision, accuracy, and recall are then analyzed. The findings from the transformed dataset indicate that students' engagement with the career exploration curriculum was the strongest predictor of students' final grades in the course. This is a fascinating finding because the amount of weight the career assignments contributed to the overall course grade was relatively low. Additionally, while both models produced adequate fit indices, the human-informed model performed significantly better and resulted in more interpretable results.
AB - First-year engineering courses are relatively large with several sections; thus, it can be rather difficult for an individual instructor to recognize when a particular student begins to lose engagement. Learning management systems (LMS) (e.g., Canvas, Blackboard, Brightspace) can be valuable tools to provide a consistent curriculum across several sections of a course and generate data regarding students' engagement with course materials. However, a human-centered approach to transform the data needs to be utilized to extract valuable insights from LMS data. The purpose of this Complete Research paper is to explore the following research questions: What type of LMS objects contain information to explain students' grades in a first-year engineering course? Is the inclusion of a human operator during the data transformation process significant to the analysis of learning outcomes? For this, data from LMS is used to predict the learning outcome of students in a FYE course. Two predictive models are compared. The first model corresponds to a usual predictive model, using the data from the LMS directly. The second model considers the specifics of the course, by transforming the data from aggregate user interaction to more granular categories related to the content of the class by a human operator. A logistic regression model is fitted using both datasets. The comparison between predictive measures such as precision, accuracy, and recall are then analyzed. The findings from the transformed dataset indicate that students' engagement with the career exploration curriculum was the strongest predictor of students' final grades in the course. This is a fascinating finding because the amount of weight the career assignments contributed to the overall course grade was relatively low. Additionally, while both models produced adequate fit indices, the human-informed model performed significantly better and resulted in more interpretable results.
UR - http://www.scopus.com/inward/record.url?scp=85138241731&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85138241731&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85138241731
SN - 2153-5965
JO - ASEE Annual Conference and Exposition, Conference Proceedings
JF - ASEE Annual Conference and Exposition, Conference Proceedings
Y2 - 26 June 2022 through 29 June 2022
ER -