OU Analyse

Tim Blackman about the OUAnalyse
`

New Tim Blackman, the Vice-Chancellor of The Open University, explains the importance of using The OUAnalyse (Early Alert Indicators) dashboard and how it can enhance ALs current practice to target interventions pro-actively to support students especially those who may be struggling (OU intranet).

 

OU Analyse is a system powered by machine learning methods for early identification of students at risk of failing. All students with their risk of failure in their next assignment are updated weekly and made available to the course tutors and the Student Support Teams to consider appropriate support. The overall objective is to significantly improve the retention of OU students.

Analysis process

Demographic and VLE data

Machine learning techniques are applied to two types of data: student demographic data and dynamic data represented by their VLE activities. Records of previous presentations are used to build and validate predictive models, which are then applied to the data of the running presentation. VLE data is collected daily at a very fine grain level, representing individual actions and activity types according to the module study plan.

Predictive models

OU Analyse uses the module fingerprint, demographic data of current students and their VLE activities to build a number of predictive models that take into account different properties of input data. Their conclusions are combined to classify students and predict who is at risk. The list of students is supplemented with justifications to explain the prediction.

Delivering the output

In Spring 2014 the project was piloted and evaluated on two introductory OU modules with about 1500 and 3000 students, respectively. In 2020, tutors from all undergraduate OU modules have access to the dashboard application and predictions are generated for more than 150,000 every year.

Beyond predictions

In addition to the identification of at risk students, we are testing a personalised Activity Recommender that would be available to students to advise them how to improve their performance in the course.

OU Analyse Dashboard

Screenshot - Module overview.
Screenshot - Student detail.
Screenshot - Course statistics.

Team

Zdenek Zdrahal
Zdenek
Zdrahal
Professor of Knowledge Engineering
Miriam Fernandez
Miriam
Fernandez
Professor in Responsible Artificial Intelligence
Martin Hlosta
Martin
Hlosta
Research Fellow
Vaclav Bayer
Vaclav
Bayer
Technical Lead
Paul Mullholand
Paul
Mulholland
Senior Research Fellow
Christothea Herodotou
Christothea
Herodotou
Professor of Learning Technologies and Social Justice

Alumni team members

  • Stéphane Carpenter
  • Jakub Kocvara
  • David Beran
  • Milan Lysonek
  • Jakub Zapletal
  • Drahomira Herrmannova
  • Michal Huptych
  • Petr Knoth
  • Jakub Kuzilek
  • Anna Kuzilkova
  • Andriy Nikolov
  • Michal Pantucek
  • Jonas Vaclavek
  • Lucie Vachova
  • Annika Wolff
  • Michal Bohuslavek

Publications

2023

New Herodotou, Christothea; Maguire, Claire; Hlosta, Martin and Mulholland, Paul (2023). Predictive Learning Analytics and University Teachers: Usage and perceptions three years post implementation . In: LAK2023: LAK23: 13th International Learning Analytics and Knowledge Conference, pp. 68–78.
New Rets, Irina; Herodotou, Christothea and Gillespie, Anna (2023). Six Practical Recommendations Enabling Ethical Use of Predictive Learning Analytics in Distance Education Journal of Learning Analytics (Early Access)

2022

Hlosta M., Mutwarasibo, F., Farrell T., Bayer V. and and Fernandez M. (2022). Understanding the BAME awarding gap at The Open University by means of quantitative and qualitative data analytics . 2020-22 eSTEem Project Final Report.
Boroowa, A. and Herodotou, C. (2022). Learning Analytics in Open and Distance Higher Education: The Case of the Open University UK . In: Prinsloo, P.; Slade, S. and Khalil, M. eds. Learning Analytics in Open and Distributed Learning.SpringerBriefs in Education. Singapore: Springer, pp. 46–62.
Rienties, Bart and Herodotou, Christothea (2022). Making sense of learning data at scale. In: Sharpe, Rhona; Bennett, Sue and Varga-Atkins, Tünde eds. Handbook for Digital Higher Education . Cheltenham: Edward Elgar Publishing, pp. 260–270.
Hlosta, Martin; Herodotou, Christothea; Papathoma, Tina; Gillespie, Anna and Bergamin, Per (2022). Predictive learning analytics in online education: A deeper understanding through explaining algorithmic errors . Computers and Education: Artificial Intelligence, 3, article no. 100108.

2021

Herodotou C., Maguire C., McDowell N., Hlosta M., Boroowa A. The engagement of university teachers with predictive learning analytics Computers & Education, Volume 173, November 2021, 104285
Rets I., Herodotou C., Bayer V., Hlosta M., and Rienties B. Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students International Journal of Educational Technology in Higher Education (In Press).
Bayer V., Hlosta M., Fernandez M. Learning Analytics and Fairness: Do Existing Algorithms Serve Everyone Equally? AIED 2021; 22nd International Conference on Artificial Intelligence in Education, 14-18 Jun 2021, ONLINE from Utrecht.
Hlosta M., Herodotou Ch., Fernandez M., Bayer V. Impact of Predictive Learning Analytics on Course Awarding Gap of Disadvantaged students in STEM AIED 2021; 22nd International Conference on Artificial Intelligence in Education, 14-18 Jun 2021, ONLINE from Utrecht.
Herodotou Ch. Predictive learning analytics: An empowering tool in the hands of online teachers - Society for Learning Analytics Research (SoLAR) (2021) SoLAR Nexus series, 29th Apr 2021
Kaliisa, Rogers; Gillespie, Anna; Herodotou, Christothea; Kluge, Anders and Rienties, Bart (2021). Teachers’ Perspectives on the Promises, Needs and Challenges of Learning Analytics Dashboards: Insights from Institutions Offering Blended and Distance Learning . In: Sahin, Muhittin and Dirk, Ifenthaler eds. Visualizations and Dashboards for Learning Analytics.Advances in Analytics for Learning and Teaching. Cham: Springer, pp. 351–370.

2020

Hlosta M., Papathoma T., Herodotou C. Explaining errors in predictions of at-risk students in distance learning education International Conference on Artificial Intelligence in Education (AIED'20), 06-10 Jul 2020, Ifrane, Morocco
Hlosta M., Bayer V., Zdrahal Z. Mini Survival Kit: Prediction based recommender to help students escape their critical situation in online courses Edrecsys@LAK Worskhop at 10th International Conference on Learning Analytics and Knowledge (LAK'20)
Herodotou C., Boroowa A., Hlosta M., Rienties B. What do distance learning students seek from student analytics? International Conference on Learning Sciences (ICLS'20), 19-23 Jun 2020, Nashville, TN, USA
Hlosta M., Zdrahal Z., Bayer V., Herodotou C. Why Predictions of At-Risk Students Are Not 100% Accurate? Showing Patterns in False Positive and False Negative Predictions 10th International Conference on Learning Analytics and Knowledge (LAK20)
Herodotou C., Rienties B., Hlosta M., Boroowa A., Mangafa C., Zdrahal Z. The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study The Internet and Higher Education, Volume 45, April 2020, 100725

2019

Herodotou, Christothea; Rienties, Bart; Verdin, Barry and Boroowa, Avinash (2019). Predictive learning analytics ‘at scale’: Guidelines to successful implementation in Higher Education based on the case of the Open University UK . Journal of Learning Analytics, 6(1) pp. 85–95.
Hlosta M., Kocvara J., Beran D., Zdrahal Z. Visualisation of key splitting milestones to support interventions LAK '19: Companion Proceedings 9th International Conference on Learning Analytics & Knowledge
Herodotou C., Rienties B., Boroowa A., Zdrahal Z., Hlosta M. A large-scale implementation of Predictive Learning Analytics in Higher Education: The teachers’ role and perspective Educational Technology Research and Development
Herodotou C., Hlosta M., Boroowa A., Rienties B., Zdrahal Z., Mangafa C. Empowering online teachers through predictive learning analytics British Journal of Educational Technology pp. 1–17.

2018

Hlosta M., Zdrahal Z., Zendulka J. Are we meeting a deadline? classification goal achievement in time in the presence of imbalanced data Knowledge-Based Systems. ISSN 09507051
Huptych M., Hlosta M., Zdrahal Z., Kocvara J. Investigating Influence of Demographic Factors on Study Recommenders Artificial Intelligence in Education. AIED 2018. Lecture Notes in Computer Science, vol 10948. Springer, Cham
Nguyen, Q., Huptych, M. and Rienties, B. Linking students' timing of engagement to learning design and academic performance LAK ’18: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

2017

Kuzilek J., Hlosta M., Zdrahal Z. Open University Learning Analytics dataset Nature Scientific Data 4:170171 doi: 10.1038/sdata.2017.171 (2017).
Rienties B., Clow D., Coughlan T., Cross S., Edwards C., Gaved M., Herodotou C., Hlosta M., Jones J., Rogaten J., Ullmann T. Scholarly insight Autumn 2017: a Data wrangler perspective The Open University
Herodotou C., Gilmour A., Boroowa A., Rienties B., Zdrahal Z., Hlosta M. Predictive modelling for addressing students’ attrition in Higher Education: The case of OU Analyse CALRG Annual Conference 2017
Herodotou C., Rienties B., Boroowa A., Zdrahal Z., Hlosta M., Naydenova G. Implementing predictive learning analytics on a large scale: the teacher's perspective LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference
Huptych M., Bohuslavek M., Hlosta M., Zdrahal Z., Measures for recommendations based on past students' activity LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference
Hlosta M., Zdrahal Z., Zendulka J. Ouroboros: early identification of at-risk students without models based on legacy data LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge Conference

2016

Rienties B., Cross S., Zdrahal, Z. Implementing a Learning Analytics Intervention and Evaluation Framework: what works? Big Data and Learning Analytics in Higher Education: Current Theory and Practice

2015

Kuzilek, J., Hlosta, M., Herrmannova, D., Zdrahal, Z. and Wolff, A. OU Analyse: Analysing At-Risk Students at The Open University. Learning Analytics Review, no. LAK15-1, March 2015, ISSN: 2057-7494.
Herrmannova, D., Hlosta, M., Kuzilek, J., Zdrahal, Z. Evaluating weekly predictions of at-risk students at the Open University: results and issues. EDEN 2015 , Barcelona, Spain.

2014

Wolff, A., Zdrahal, Z., Herrmannova, D., Kuzilek, J. and Hlosta, M. Developing predictive models for early detection of at-risk students on distance learning modules , Workshop: Machine Learning and Learning Analytics at LAK, Indianapolis
Hlosta, M., Herrmannova, D., Vachova, L., Kuzilek, J., Zdrahal, Z. and Wolff, A. Modelling student online behaviour in a virtual learning environment , Workshop: Machine Learning and Learning Analytics at LAK, Indianapolis

2013

Wolff, A., Zdrahal, Z., Nikolov, A. and Pantucek, M., Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment, Learning Analytics and Knowledge
Wolff, A., Zdrahal, Z., Herrmannova, D. and Knoth, P., Predicting Student Performance from Combined Data Sources, in eds. Alejandro Peña-Ayala, Educational Data Mining: Applications and Trends, 524, Springer

2012

Wolff, A. and Zdrahal, Z., Improving retention by identifying and Supporting 'at-risk' students Case study for EDUCAUSE review online

Open University Learning Analytics dataset

We introduce the anonymised Open University Learning Analytics Dataset (OULAD). It contains data about courses, students and their interactions with Virtual Learning Environment (VLE) for seven selected courses (called modules). Presentations of courses start in February and October - they are marked by “B” and “J” respectively. The dataset consists of tables connected using unique identifiers. All tables are stored in the csv format.
OULAD page Download dataset

OULAD testimonials

Learning Analytics & Open Data Hackathon 3.0 at the University of British Columbia, Canada

The two-day event was held at the University of British Columbia, Canada. Over 100 participants dove into our dataset and experimented with it. Interesting projects in the area of social comparison and visualisation have been developed.

LAK18 Hackathon at Learning Analytics and Knowledge conference (LAK18) in Sydney, Australia

The principal aim of Hack@LAK18 was to enable multi-disciplinary thinking over key open challenges in Learning Analytics based on a problem-oriented, pragmatic approach. OULAD was one of the recommended datasets by the organisers.

More testimonials

Media

OUAnalyse UNESCO Awards

New OU Analyse in top four in UNESCO awards

OU Analyse, the OU’s Predictive Learning Analytics system, was selected as a finalist and among the four best projects for the 2020 edition of the UNESCO Prize in Education

OUAnalyse winners at the DataIQ 2020 awards

Winners at the DataIQ 2020 Awards

September 2020 - winners of the DataIQ 2020 Awards in Best use of data by a not-for-profit organisation.

OUAnalyse Recognition of Excellence in Teaching

Excellence in enhancing teaching and learning at the Open University

April 2020 - OUAnalyse awarded Recognition of Excellence in Teaching at the OU.

 
OUAnalyse OU Research Awards

Recognised by the Open University by receiving the Research Excellence Award 2019

for Outstanding Impact on Teaching, Curriculum and Students.

THE logo

Shortlisted for THE awards 2019

5 September 2019 - OU Analyse was shortlisted for the Times Higher Education Awards 2019 in the category Technological or Digital Innovation of the Year.

Bricks to Click logo

From Bricks to Clicks - The Potential of Data and Analytics in HE

26 January 2016 - OU Analyse mentioned as the UK's only significant headway in Predictive Analytics

 
BBC logo

OU students' progress to be monitored by software

28 July 2015 - OU Analyse was featured on the BBC News website.

FT logo

Students under surveillance

24 July 2015 - Predicting at-risk students at the Open University was part of the weekend article in The Financial Times.

THE logo

The week in higher education – 30 July 2015

30 July 2015 - Times Higher Education mentioned OU Analyse in their weekly overview in academia.

Address

Knowledge Media Institute,
The Open University,
Milton Keynes,
MK7 6AA,
United Kingdom.