Using Machine Learning to Predict Engineering Technology Students' Success with Computer Aided Design

TL;DR

This study demonstrates how machine learning models can predict engineering students' success in CAD design tasks, enabling real-time assistance and improved learning outcomes.

Contribution

It introduces a novel approach using student interaction data and machine learning to predict CAD design success, facilitating automated real-time support.

Findings

Models using design action sequences are highly predictive.

Logistic regression achieved over 60% success prediction accuracy.

Early design data can effectively forecast student performance.

Abstract

Computer-aided design (CAD) programs are essential to engineering as they allow for better designs through low-cost iterations. While CAD programs are typically taught to undergraduate students as a job skill, such software can also help students learn engineering concepts. A current limitation of CAD programs (even those that are specifically designed for educational purposes) is that they are not capable of providing automated real-time help to students. To encourage CAD programs to build in assistance to students, we used data generated from students using a free, open source CAD software called Aladdin to demonstrate how student data combined with machine learning techniques can predict how well a particular student will perform in a design task. We challenged students to design a house that consumed zero net energy as part of an introductory engineering technology undergraduate course. Using data from 128 students, along with the scikit-learn Python machine learning library, we tested our models using both total counts of design actions and sequences of design actions as inputs. We found that our models using early design sequence actions are particularly valuable for prediction. Our logistic regression model achieved a >60% chance of predicting if a student would succeed in designing a zero net energy house. Our results suggest that it would be feasible for Aladdin to provide useful feedback to students when they are approximately halfway through their design. Further improvements to these models could lead to earlier predictions and thus provide students feedback sooner to enhance their learning.