Predicting learning outcome in a first-year engineering course: a human-centered learning analytics approach

First-year engineering courses are relatively large with several sections; thus, it can be rather difficult for an individual instructor to recognize when a particular student begins to lose engagement. Learning management systems (LMS) (e.g., Canvas, Blackboard, Brightspace) can be valuable tools to provide a consistent curriculum across several sections of a course and generate data regarding students' engagement with course materials. However, a human-centered approach to transform the data needs to be utilized to extract valuable insights from LMS data. The purpose of this Complete Research paper is to explore the following research questions: What type of LMS objects contain information to explain students' grades in a first-year engineering course? Is the inclusion of a human operator during the data transformation process significant to the analysis of learning outcomes? For this, data from LMS is used to predict the learning outcome of students in a FYE course. Two predictive models are compared. The first model corresponds to a usual predictive model, using the data from the LMS directly. The second model considers the specifics of the course, by transforming the data from aggregate user interaction to more granular categories related to the content of the class by a human operator. A logistic regression model is fitted using both datasets. The comparison between predictive measures such as precision, accuracy, and recall are then analyzed. The findings from the transformed dataset indicate that students' engagement with the career exploration curriculum was the strongest predictor of students' final grades in the course. This is a fascinating finding because the amount of weight the career assignments contributed to the overall course grade was relatively low. Additionally, while both models produced adequate fit indices, the human-informed model performed significantly better and resulted in more interpretable results.