OU Analyse

New OU Analyse Team Wins Top Prize for Educational Dataset

The OU Analyse team has been awarded the prestigious Educational Dataset Prize by the Educational Data Mining Society for their remarkable work with the OULAD dataset. The OULAD dataset, which consists of anonymised student interactions with the Virtual Learning Environment (VLE) at The Open University, combined with demographics and study results, has had a major impact on the field of learning analytics.

OU Analyse is a system powered by machine learning methods for early identification of students at risk of failing. All students with their risk of failure in their next assignment are updated weekly and made available to the course tutors and the Student Support Teams to consider appropriate support. The overall objective is to significantly improve the retention of OU students.

Demographic and VLE data

Machine learning techniques are applied to two types of data: student demographic data and dynamic data represented by their VLE activities. Records of previous presentations are used to build and validate predictive models, which are then applied to the data of the running presentation. VLE data is collected daily at a very fine grain level, representing individual actions and activity types according to the module study plan.

Predictive models

OU Analyse uses the module fingerprint, demographic data of current students and their VLE activities to build a number of predictive models that take into account different properties of input data. Their conclusions are combined to classify students and predict who is at risk. The list of students is supplemented with justifications to explain the prediction.

Delivering the output

In Spring 2014 the project was piloted and evaluated on two introductory OU modules with about 1500 and 3000 students, respectively. In 2020, tutors from all undergraduate OU modules have access to the dashboard application and predictions are generated for more than 150,000 every year.

Beyond predictions

In addition to the identification of at risk students, we are testing a personalised Activity Recommender that would be available to students to advise them how to improve their performance in the course.

OU Analyse Dashboard

Access OU Analyse Dashboard

Team

Zdenek
Zdrahal

Professor of Knowledge Engineering

Miriam
Fernandez

Professor in Responsible Artificial Intelligence

Martin
Hlosta

Research Fellow

Vaclav
Bayer

Technical Lead

Paul
Mulholland

Senior Research Fellow

Christothea
Herodotou

Professor of Learning Technologies and Social Justice

Alumni team members

Stéphane Carpenter
Jakub Kocvara
David Beran
Milan Lysonek
Jakub Zapletal
Drahomira Herrmannova
Michal Huptych
Petr Knoth
Jakub Kuzilek
Anna Kuzilkova
Andriy Nikolov
Michal Pantucek
Jonas Vaclavek
Lucie Vachova
Annika Wolff
Michal Bohuslavek

Publications

2023

New Herodotou, Christothea; Maguire, Claire; Hlosta, Martin and Mulholland, Paul (2023). Predictive Learning Analytics and University Teachers: Usage and perceptions three years post implementation . In: LAK2023: LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 68–78.

New Rets, Irina; Herodotou, Christothea and Gillespie, Anna (2023). Six Practical Recommendations Enabling Ethical Use of Predictive Learning Analytics in Distance Education Journal of Learning Analytics (Early Access)

2022

Hlosta M., Mutwarasibo, F., Farrell T., Bayer V. and and Fernandez M. (2022). Understanding the BAME awarding gap at The Open University by means of quantitative and qualitative data analytics . 2020-22 eSTEem Project Final Report.

Boroowa, A. and Herodotou, C. (2022). Learning Analytics in Open and Distance Higher Education: The Case of the Open University UK . In: Prinsloo, P.; Slade, S. and Khalil, M. eds. Learning Analytics in Open and Distributed Learning.SpringerBriefs in Education. Singapore: Springer, pp. 46–62.

Rienties, Bart and Herodotou, Christothea (2022). Making sense of learning data at scale. In: Sharpe, Rhona; Bennett, Sue and Varga-Atkins, Tünde eds. Handbook for Digital Higher Education . Cheltenham: Edward Elgar Publishing, pp. 260–270.

Hlosta, Martin; Herodotou, Christothea; Papathoma, Tina; Gillespie, Anna and Bergamin, Per (2022). Predictive learning analytics in online education: A deeper understanding through explaining algorithmic errors . Computers and Education: Artificial Intelligence, 3, article no. 100108.

2021

Herodotou C., Maguire C., McDowell N., Hlosta M., Boroowa A. The engagement of university teachers with predictive learning analytics Computers & Education, Volume 173, November 2021, 104285

Rets I., Herodotou C., Bayer V., Hlosta M., and Rienties B. Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students International Journal of Educational Technology in Higher Education (In Press).

Bayer V., Hlosta M., Fernandez M. Learning Analytics and Fairness: Do Existing Algorithms Serve Everyone Equally? AIED 2021; 22nd International Conference on Artificial Intelligence in Education, 14-18 Jun 2021, ONLINE from Utrecht.

Hlosta M., Herodotou Ch., Fernandez M., Bayer V. Impact of Predictive Learning Analytics on Course Awarding Gap of Disadvantaged students in STEM AIED 2021; 22nd International Conference on Artificial Intelligence in Education, 14-18 Jun 2021, ONLINE from Utrecht.

Herodotou Ch. Predictive learning analytics: An empowering tool in the hands of online teachers - Society for Learning Analytics Research (SoLAR) (2021) SoLAR Nexus series, 29th Apr 2021

Kaliisa, Rogers; Gillespie, Anna; Herodotou, Christothea; Kluge, Anders and Rienties, Bart (2021). Teachers’ Perspectives on the Promises, Needs and Challenges of Learning Analytics Dashboards: Insights from Institutions Offering Blended and Distance Learning . In: Sahin, Muhittin and Dirk, Ifenthaler eds. Visualizations and Dashboards for Learning Analytics.Advances in Analytics for Learning and Teaching. Cham: Springer, pp. 351–370.

2020

Hlosta M., Papathoma T., Herodotou C. Explaining errors in predictions of at-risk students in distance learning education International Conference on Artificial Intelligence in Education (AIED'20), 06-10 Jul 2020, Ifrane, Morocco

Hlosta M., Bayer V., Zdrahal Z. Mini Survival Kit: Prediction based recommender to help students escape their critical situation in online courses Edrecsys@LAK Worskhop at 10th International Conference on Learning Analytics and Knowledge (LAK'20)

Herodotou C., Boroowa A., Hlosta M., Rienties B. What do distance learning students seek from student analytics? International Conference on Learning Sciences (ICLS'20), 19-23 Jun 2020, Nashville, TN, USA

Hlosta M., Zdrahal Z., Bayer V., Herodotou C. Why Predictions of At-Risk Students Are Not 100% Accurate? Showing Patterns in False Positive and False Negative Predictions 10th International Conference on Learning Analytics and Knowledge (LAK20)

Herodotou C., Rienties B., Hlosta M., Boroowa A., Mangafa C., Zdrahal Z. The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study The Internet and Higher Education, Volume 45, April 2020, 100725

2019

Herodotou, Christothea; Rienties, Bart; Verdin, Barry and Boroowa, Avinash (2019). Predictive learning analytics ‘at scale’: Guidelines to successful implementation in Higher Education based on the case of the Open University UK . Journal of Learning Analytics, 6(1) pp. 85–95.

Hlosta M., Kocvara J., Beran D., Zdrahal Z. Visualisation of key splitting milestones to support interventions LAK '19: Companion Proceedings 9th International Conference on Learning Analytics & Knowledge

Herodotou C., Rienties B., Boroowa A., Zdrahal Z., Hlosta M. A large-scale implementation of Predictive Learning Analytics in Higher Education: The teachers’ role and perspective Educational Technology Research and Development

Herodotou C., Hlosta M., Boroowa A., Rienties B., Zdrahal Z., Mangafa C. Empowering online teachers through predictive learning analytics British Journal of Educational Technology pp. 1–17.

2018

Hlosta M., Zdrahal Z., Zendulka J. Are we meeting a deadline? classification goal achievement in time in the presence of imbalanced data Knowledge-Based Systems. ISSN 09507051

Huptych M., Hlosta M., Zdrahal Z., Kocvara J. Investigating Influence of Demographic Factors on Study Recommenders Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science, vol 10948. Springer, Cham

Nguyen, Q., Huptych, M. and Rienties, B. Linking students' timing of engagement to learning design and academic performance LAK ’18: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

2017

Kuzilek J., Hlosta M., Zdrahal Z. Open University Learning Analytics dataset Nature Scientific Data 4:170171 doi: 10.1038/sdata.2017.171 (2017).

Rienties B., Clow D., Coughlan T., Cross S., Edwards C., Gaved M., Herodotou C., Hlosta M., Jones J., Rogaten J., Ullmann T. Scholarly insight Autumn 2017: a Data wrangler perspective The Open University

Herodotou C., Gilmour A., Boroowa A., Rienties B., Zdrahal Z., Hlosta M. Predictive modelling for addressing students’ attrition in Higher Education: The case of OU Analyse CALRG Annual Conference 2017

Herodotou C., Rienties B., Boroowa A., Zdrahal Z., Hlosta M., Naydenova G. Implementing predictive learning analytics on a large scale: the teacher's perspective LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

Huptych M., Bohuslavek M., Hlosta M., Zdrahal Z., Measures for recommendations based on past students' activity LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

Hlosta M., Zdrahal Z., Zendulka J. Ouroboros: early identification of at-risk students without models based on legacy data LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

2016

Rienties B., Cross S., Zdrahal, Z. Implementing a Learning Analytics Intervention and Evaluation Framework: what works? Big Data and Learning Analytics in Higher Education: Current Theory and Practice

2015

Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z. and Wolff, A. OU Analyse: Analysing At-Risk Students at The Open University. Learning Analytics Review, no. LAK15-1, March 2015, ISSN: 2057-7494.

Herrmannova, D., Hlosta, M., Kuzilek, J., Zdrahal, Z. Evaluating weekly predictions of at-risk students at the Open University: results and issues. EDEN 2015 , Barcelona, Spain.

2014

Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J. and Hlosta, M. Developing predictive models for early detection of at-risk students on distance learning modules , Workshop: Machine Learning and Learning Analytics at LAK, Indianapolis

Hlosta, M., Herrmannova, D., Vachova, L., Kuzilek, J., Zdrahal, Z. and Wolff, A. Modelling student online behaviour in a virtual learning environment , Workshop: Machine Learning and Learning Analytics at LAK, Indianapolis

2013

Wolff, A., Zdrahal, Z., Nikolov, A. and Pantucek, M., Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment, Learning Analytics and Knowledge

Wolff, A., Zdrahal, Z., Herrmannova, D. and Knoth, P., Predicting Student Performance from Combined Data Sources, in eds. Alejandro Peña-Ayala, Educational Data Mining: Applications and Trends, 524, Springer

2012

Wolff, A. and Zdrahal, Z., Improving retention by identifying and Supporting 'at-risk' students Case study for EDUCAUSE review online

Open University Learning Analytics dataset

We introduce the anonymised Open University Learning Analytics Dataset (OULAD). It contains data about courses, students and their interactions with Virtual Learning Environment (VLE) for seven selected courses (called modules). Presentations of courses start in February and October - they are marked by “B” and “J” respectively. The dataset consists of tables connected using unique identifiers. All tables are stored in the csv format.

OULAD page Download dataset

OULAD testimonials

Learning Analytics & Open Data Hackathon 3.0 at the University of British Columbia, Canada

The two-day event was held at the University of British Columbia, Canada. Over 100 participants dove into our dataset and experimented with it. Interesting projects in the area of social comparison and visualisation have been developed.

LAK18 Hackathon at Learning Analytics and Knowledge conference (LAK18) in Sydney, Australia

The principal aim of Hack@LAK18 was to enable multi-disciplinary thinking over key open challenges in Learning Analytics based on a problem-oriented, pragmatic approach. OULAD was one of the recommended datasets by the organisers.

More testimonials

Media

Tim Blackman about the OUAnalyse

Tim Blackman, the Vice-Chancellor of The Open University, explains the importance of using The OUAnalyse (Early Alert Indicators) dashboard and how it can enhance ALs current practice to target interventions proactively to support students especially those who may be struggling (OU intranet).

OU Analyse in top four in UNESCO awards

OU Analyse, the OU’s Predictive Learning Analytics system, was selected as a finalist and among the four best projects for the 2020 edition of the UNESCO Prize in Education

Winners at the DataIQ 2020 Awards

September 2020 - winners of the DataIQ 2020 Awards in Best use of data by a not-for-profit organisation.

Excellence in enhancing teaching and learning at the Open University

April 2020 - OUAnalyse awarded Recognition of Excellence in Teaching at the OU.

Recognised by the Open University by receiving the Research Excellence Award 2019

for Outstanding Impact on Teaching, Curriculum and Students.

Address

Knowledge Media Institute,
The Open University,
Milton Keynes,
MK7 6AA,
United Kingdom.

Email: analyse-support@open.ac.uk