Presentation on the potential of Learning Analytics for supporting disabled students and identifying accessibility deficits:
Today I submitted and internal Open University project bid to a programme called eSTEeM.
I post here the project description. N.B. at this stage this is just a proposal. However we should hear by 31 October 2012 if this has been supported as an eSTEeM project and funded. If so I might be blogging much more about this work and its findings.
May I remind readers I set up a LinkedIn Group to try and tease out if there was anyone worldwide doing anything in the area of Learning Analytics and Accessibility. There has been some interest (the group currently has 75 members) but no one has yet shared that they are doing substantive work. So you never know LA4STEM but in the future be seen as seminal. 😉
If you are interested in this field may I commend to you SoLAR – The Society of Learning Analytics Research: http://www.solaresearch.org/
I will be giving a 30 min presentation about this work at this event – it’s a long way to travel for me 😉 – it’s OU main campus where I work :
SoLAR Flare UK (19 Nov 2012) #flareUK
Mon 19 Nov 2012, The Open University
Jennie Lee Building, Walton Hall, Milton Keynes, MK7 6AA [map]
Feel free to post comments or questions!
LA4STEM Project Description
The LA4STEM project will review the potential of Learning Analytics in higher education, specifically in STEM, and with an emphasis on supporting disabled students and facilitating accessibility enhancements.
Learning Analytics is defined as the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs. Learning analytics is a “hot topic” in eLearning and was the second headline topic in the 2-3 year time to adoption section in the 2012 NMC Horizon Report on Higher Education:
“The larger promise of learning analytics, however, is that when correctly applied and interpreted, it will enable faculty to more precisely understand students’ learning needs and to tailor instruction appropriately far more accurately and far sooner than is possible today.”
The LA4STEM project will specifically explore the following STEM application areas for Learning Analytics:
- Student support (with an emphasis on support for disabled students)
- Tutor support (facilitating their support of disabled learners)
- Module review (identifying accessibility enhancements)
- Retention and attainment (focussing on where disabled students appear disadvantaged)
- Learning analytics in remote labs (because of their potential for enhancing access to STEM)
- Recommender systems (the timely direction of disabled students to support and study skills aids; including scaffolding of STEM specific learning activities)
A key output of the project will be an external funding bid for a larger-scale collaborative project. The work of LA4ALL will inform pilots in this project. Provide envisaged benefits are confirmed, this should lead to enterprise level implementation within the OU and across HE.
The findings of the LA4STEM project will be disseminated, firstly throughout the Science and MCT faculties, then to the wider university. External dissemination will highlight the OU’s lead in this field.
 Johnson, L., Adams, S. and Cummins, M. (2012) The NMC Horizon Report: 2012 Higher Education Edition. The New Media Consortium, Austin, Texas: http://www.nmc.org/publications/horizon-report-2012-higher-ed-edition
This blog post makes available publicly simple use cases and scenarios we are using in various internal projects and discussions at the Open University on the application of Learning Analytics to support disabled students and to identify accessibility deficits in online teaching and learning. Scenarios based on potential use cases are useful as design tools and in communicating ideas around complex systems with multiple actors. Three scenarios are presented here that were adapted from those created for a concept paper  for the Society for Learning Analytics Research (SoLAR – www.solaresearch.org). Credit goes to my colleagues Rebecca Ferguson and Annika Wolff for writing them. Discussion of these use case scenarios is deferred to subsequent blog posts but comments and questions on them would be most welcome.
Kris is a student who has access to a dashboard of analytics that provide him with feedback when he is at his computer or using a mobile device. He has set the dashboard to send him a weekly summary of his activity on university sites and on a set of external sites where he has chosen to share his data with the analytics system. He receives basic statistics on attendance, participation and marks on his formal assignments and exams. He receives personalised recommendations suggesting resources and contacts available at his location and relevant to his range of learning interests. However, what he finds most useful for reflection are the visual ‘mirrors’ that the system presents to him, plus suggestions of ways in which he might become a more effective, strategic learner.
Jenny teaches one of Kris’ courses. Her dashboard is designed for educators, and can be configured to illuminate problems and progress on the course. Visualisations provide an overview of the course’s social network and indicate students ‘at risk’, as defined by a range of algorithms that match online behaviour to predictive models based on past cohorts. There are different categories of risk, so Jenny can easily filter to see more details behind each student’s classification. One risk category flags students who have a disability and who are nearing a point in the course that previous analysis has identified as being a potential problem area for such students. Kris has a visual impairment and Jenny sees that this is a good time to enquire whether he needs any additional support. She logs all contact (including the form it takes), generating data that can be used to evaluate the effectiveness of the predictive methods. Jenny has also chosen to view aggregates of all the personal Learning Analytics on her students’ dashboards, providing a deeper level of insight into how they are self-reporting and evidencing their progress.
Natalie is a module manager at the OU. She is exploring statistics from a previous module presentation in order to discuss changes for the following presentation. Natalie can see from the visualisations of VLE data combined with assessment data that students who performed poorly on the second assessment had engaged significantly less with quizzes than those who did well. The system flags up that a significant proportion of these students have declared a disability. Natalie compares datasets and finds a similar pattern on other modules, indicating a potential accessibility issue with the quizzes presented alongside module materials. She undertakes some follow-up activities, contacting a percentage of students from several modules with a questionnaire to try to identify the cause of the problem before passing her findings to the university’s accessibility experts for detailed evaluation and remedial action.
 George Siemens, Dragan Gasevic, Caroline Haythornthwaite, Shane Dawson, Simon Buckingham Shum, Rebecca Ferguson, Erik Duval, Katrien Verbert, Ryan S. J. d. Baker, (2011), Open Learning Analytics: an integrated & modularized platform – Proposal to design, implement and evaluate an open platform to integrate heterogeneous learning analytics techniques. Available at: http://www.solaresearch.org/ (accessed 12 October 2012)
This week I have been writing an introductory section for a paper on models of disability and accessibility. This has led me to think again about the relationship between the two.
The Medical Model of Disability
Disabilities have traditionally been described with reference to medical conditions that they were seen to arise from. This is known as the medical model of disability and was encapsulated in the 1980 World Health Organisation’s (WHO) International classification of impairments, disabilities, and handicaps  which included the following definitions:
- Impairment = a loss or abnormality of physical bodily structure or function, of logic-psychic origin, or physiological or anatomical origin
- Disability = any limitation or function loss deriving from impairment that prevents the performance of an activity in the time-lapse considered normal for a human being
- Handicap = the disadvantaged condition deriving from impairment or disability limiting a person performing a role considered normal in respect of their age, sex and social and cultural factors
The Social Model of Disability
The main alternative to the medical model of disability is the social model. This has been highly influential, over the last 30 years, in shaping policy, practice and attitudes to disabled people. The social model stemmed from the publication of Fundamental Principles of Disability in 1976.  This revolutionised the understanding of disability arguing that it was not mainly caused by impairments but by the way society was organised and responded to disabled people.
In the social model, disability is caused by society and is not the ‘fault’ of an individual disabled person, or an inevitable consequence of their limitations. Disability is the product of the physical, organisational and attitudinal barriers present within society. The social model takes account of disabled people as part of the economic, environmental and cultural society.
The WHO revised its definitions of disability, in part as a response to this social model, and from the realisation that the medical model was of very limited use in defining effective responses in meeting the needs of disabled people. In 2001 WHO published the International Classification of Functioning, Disability and Health (ICF) . In the ICF disability is described as:
… the outcome or result of a complex relationship between an individual’s health condition and personal factors, and of the external factors that represent the circumstances in which the individual lives.
Building on the social model the IMS Global Learning Consortium, introducing its work developing technical standards for accessibility in e-learning, offered a more education specific definition of both disability and accessibility :
… the term disability has been re-defined as a mismatch between the needs of the learner and the education offered. It is therefore not a personal trait but an artifact of the relationship between the learner and the learning environment or education delivery. Accessibility, given this re-definition, is the ability of the learning environment to adjust to the needs of all learners. Accessibility is determined by the flexibility of the education environment (with respect to presentation, control methods, access modality, and learner supports) and the availability of adequate alternative-but-equivalent content and activities. The needs and preferences of a user may arise from the context or environment the user is in, the tools available (e.g., mobile devices, assistive technologies such as Braille devices, voice recognition systems, or alternative keyboards, etc.), their background, or a disability in the traditional sense. Accessible systems adjust the user interface of the learning environment, locate needed resources and adjust the properties of the resources to match the needs and preferences of the user.
Post Social Models of Disability
The social model of disability has been criticised and various moves instigated to move beyond it. For an example see Torn Shakespeare and Nicholas Watson (2001) . They argue instead for an Embodied Ontology: “we are our body, with all of its imperfections and impairments”. Further, they assert that “there is no qualitative difference between disabled and non-disabled people because we are all impaired in some form, some more than others”. They consider the idea of a normal/perfect person as mythical. However, this discussions has been more within the academic world of disability studies and I would contest has yet to have widespread impact beyond this, and particularly relevant to this post, on accessibility. That being said one direct relation to accessibility is that accessibility accommodations have benefits for many who do not consider themselves disabled. An example of this is the feature present in most modern browsers to enlarge the display of web pages in response to a short cut key, usually Ctrl +. This was originally introduced for those with a visual impairment but at times is useful to all. A major piece of research undertaken by Forrester for Microsoft in 2003  supports this case of the wider benefit of accessibility accommodations. It found that 57% of working-age computer users are likely to benefit from accessible technology (where accessible technology is understood as technical responses to promote access for disabled people to computer hardware and software).
Functional Models of Disability
The term accessibility is widely used in the context of web design. The W3C describes web accessibility thus:
Web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the Web, and that they can contribute to the Web. 
This is in essence based on a functional model of disability. Generally in Human Computer Interaction (HCI) a functional approach is most useful. What is important in the design of web-based applications or content is how the diversity of users access the computer. This design can be said to be accessible if it facilitates full interaction by all users irrespective of assistive technologies or access approaches that may be adopted by some.
The AccessForAll 3.0 Personal Needs and Preferences (PNP) provides a specification that enables comprehensive profiles of individuals’ access approaches and assistive technologies to be stored based on a functional model. This specification is being developed within the IMS Global Learning Consortium and went to public draft in September 2012 . These functional profiles could be generated by disabled people themselves, possibly with the help of advisors, inputting their specific access approaches and requirements to a web-form. Such profiles have great potential in personalisation approaches to accessibility and in analytics based approaches to identifying accessibility issues, as discussed elsewhere in this blog.
A Note on WCAG and Models of Disability
The Web Content Accessibility Guidelines (WCAG) 2.0 , a formal recommendation of the web standards body the W3C, are the de facto international standard on web accessibility. These are targeted at web developers and cover what is normally referred to as technical accessibility. They are organised according to four top-level principles of web accessibility: that web pages should be perceivable, operable, understandable, and robust. WCAG are focussed at the properties of a web page and in so doing might be considered to be based on a functional model. However, the user is deliberately subsumed in their formulation; their concern is the functional properties of the web page not the person accessing them. This ignores the consequences of the social model of disability of the importance of context and the relational nature of accessibility.
The development of web assets or applications is a process. Accessibility considerations need to be built into the everyday practices across the web product life-cycle from conception and specification through development to delivery and maintenance. Recognising this, the British Standards Institute developed BS 8878: 2010 Web Accessibility Code of Practice . This facilitates a pragmatic application of WCAG 2.0 within a process based approach and reasserts a user focus.
Our models of disability are important, they shape our attitudes and impact on how effectively the needs and preferences of disabled people are met in design. The medical model is now widely seen as outmoded and a perpetuator of discriminatory attitudes. The social model has had widespread influence. It is important in accessibility considerations because it recognises the importance of the context of the users and supports the view of accessibility as a relationship property; in the case of web accessibility the relationship being between the diversity of users and the web resource or application. Functional models have been asserted as the most useful in design and development and the potential of these for personalisation and analytics highlighted.
(All web-links checked 10 October 2012)
 World Health Organization, (1980) International classification of impairments, disabilities, and handicaps. A manual of classification relating to the consequences of disease. Geneva, WHO
 UPIAS, (1976) Fundamental Principles of Disability, London: Union of Physically Impaired against Segregation, available on-line at:
 World Health Organization. (2001) International Classification of Functioning, Disability and Health. Geneva, WHO, Searchable online versions available at: http://www.who.int/classifications/icf/en/
 IMS Global Learning Consortium (2004) IMS AccessForAll Meta-data Overview. http://www.imsglobal.org/accessibility/accmdv1p0/imsaccmd_oviewv1p0.html
 Torn Shakespeare, Nicholas Watson, (2001) The social model of disability: An outdated ideology?, in Sharon N. Barnartt and Barbara M. Altman (ed.)Exploring Theories and Expanding Methodologies: Where we are and where we need to go (Research in Social Science and Disability, Volume 2), Emerald Group Publishing Limited, pp.9-28. Availible on-line at:
 Microsoft (2004) The wide range of abilities and its impact on computer technology. Available on-line at:
 World Wide Web Consortium (2005). Introduction to Web Accessibility, available at: http://www.w3.org/WAI/intro/accessibility.php
 IMS Global Learning Consortium (2012) Access for All (AfA), Version 3.0 Specification, Public Draft 1.0. Primer and specification documents available from: http://www.imsglobal.org/accessibility/index.html
 W3C (2008), Web Content Accessibility Guidelines 2.0 (WCAG 2.0), available at: http://www.w3.org/TR/WCAG/
 British Standards International (2010). BS 8878:2010 Web Accessibility – Code of Practice, (charged for publication available through http://www.bsi-publications.com and by subscription through BSOL https://bsol.bsigroup.com/)
Today I have been writing a section on Disability and Accessibility for a paper for LAK13 entitled “What Can Learning Analytics Contribute to Disabled Students’ Learning and to Accessibility in e-Learning Systems?”. In doing so I had cause to check on the status of the long promised Code of Practice for Higher Education covering the UK’s Equality Act 2010 . I discovered this on the Equality and Human Rights Commission’s web site:
Other Codes of Practice
We were intending to produce further statutory codes of practice on the Public Sector Equality Duty (PSED), which came into force on 5 April 2011, and codes for the Further and Higher Education (FEHE) sector and schools.
Unfortunately, we are no longer able to proceed with these plans. The Government is keen to reduce bureaucracy around the Equality Act 2010, and feels that further statutory guidance may place too much of a burden on public bodies. Although the Commission has powers to issue codes, it cannot do so without the approval of the Secretary of State, as we are reliant upon government to lay codes before parliament, in order for them to be statutory.
It is the Commission’s view that, rather than creating a regulatory burden, statutory codes have a valuable role to play in making clearer to everyone what is and is not needed in order to comply with the Equality Act. However, as this is no longer an option, we feel the best solution is to issue our draft codes as non statutory codes instead. These non statutory codes will still give a formal, authoritative, and comprehensive legal interpretation of the PSED and education sections of the Act and will make it clear to everyone what the requirements of the legislation are.
These now non-statutory codes do not seem to be published yet and with the further discouragement from Government who knows when they will be. I and many others had been eagerly hoping that among other things the statutory codes would have provided clear legal guidance on “reasonable adjustments” generally and web accessibility specifically. It was hoped that they would include reference to the key external accessibility standards: WCAG 2.0 and BS8878.
To my view this is a very retrograde step. The old CoP relating to the previous legislation, the Disability Discrimination Act (1995 as amended 2005) is still available but now has no statutory basis and is outdated in terms of educational practice, web accessibility standards, technology and the law. Available at: http://www.equalityhumanrights.com/uploaded_files/code_of_practice__revised__for_providers_of_post-16_education_and_related_services__dda_.pdf
I am now chairing the newly formed Open University Web Accessibility Standards Working Group defining a common web accessibility standard for the OU and developing associated support documentation for managers and developers. This is part of a overall Web Governance Review. This work needs a secure legal underpinning which I had hoped would come from the CoP. It would be helpful is we could authoritatively point to a statutory statement of what is considered as the appropriate level of web accessibility under the term “reasonable adjustment”. That being said it is probably optimistic to think the CoP would have given that.
As commented elsewhere in this blog defining levels of accessibility is problematic. This follows from the fact that accessibility is a property of the relationship between the user and the web resource and depends on the circumstances in which and technology they use to access it. More generally it is a summation of these relationships for the full diversity of potential users. Web accessibility is not, as usually inferred from WACG2.0 and in most work on accessibility metrics, a property of the resource alone. However, organisations in education, commerce and the public sector are longing for a way of authoritatively asserting that they have sufficiently addressed accessibility in terms of their legal obligations.
I have now submitted two papers that cover Learning Analytics and Disabled Students.
1. Cooper, M., Sloan, D., Kelly, B. and Lewthwaite, S., 2012. A challenge to web accessibility metrics and guidelines: putting people and processes first. In: W4A 2012: 9th International Cross-Disciplinary Conference on Web Accessibility, 16-18 April 2012, Lyon.
2. Cooper, M., Ferguson, R., Wolff, A., 2012 What Can Learning Analytics Contribute to Disabled Students’ Learning and to Accessibility in e-Learning Systems? In HCI2012: 26th Annual Conference of the Specialist HCI group of the BCS, 12th-14th September 2012, Birmingham, UK.
Paper 1. is available for download from: http://opus.bath.ac.uk/29190/. Paper 2. is still under peer review.
The more I think about the topic the more I am sure there is significant potential here.
This arises from the fact that at a university, for example, it is possible to know which users (students) have a disability and indeed in many cases what disability or functional impairment. Thus the behaviour and attainment of disabled students can be monitored in comparison to the general student population across the e-learning provision, in many cases in real-time. Therefore modules, or part of modules that seem to be problematic for disabled students can be identified. Learning Analytics is unlikely to be able to identify what the problem is, but by indicating where it is, can trigger further investigation and remedial action. Such problems may be in the learning design, the technical accessibility, the assessment, etc.
Do readers of this blog have any examples envisaged or implemented of such approaches they would like to share? This area is just under consideration at the Open University where I work.
With 3 colleagues from other UK universities I have just had the following paper accepted for W4A2012:
A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First
In this paper we argue that web accessibility guidelines such as WCAG 2.0 are insufficient in ensuring accessibility is achieved in any web-based resource or service. A key deficiency is in an appropriate level of understanding of the users, their needs and behaviors. In a higher education context one approach to address this that I am currently exploring is based on Learner Analytics. This blog posts expands on the ideas floated in the above paper and invites comment on them. I am just beginning to draft a project proposal to fund a pilot project exploring these ideas with real data and real students in their learning contexts. If this project might be of interest then please e-mail me: email@example.com.
The 1st International Conference on Learning Analytics & Knowledge defined Learner Analytics as:
… the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.
The Open University is currently hosting various projects exploring how such approaches may be adopted to enhance its educational offering and support an ethos of continued improvement. A survey report on this area, Ferguson, R. (2012). “The State Of Learning Analytics in 2012: A Review and Future Challenges.” Technical Report KMI-12-01, Knowledge Media Institute, The Open University, UK, is freely available at:
But how can Learner Analytics be used to enhance accessibility?
A possible starting point was outlined in the above cited W4A2012 paper and is quoted here:
One area of interest is withdrawals – e.g. when students stop study before completion of a module towards a degree. Such “drop-outs” are a high-stakes issue for universities, because they form part of the assessment of the quality of their teaching, which in turn impacts on levels of funding from government and from student-paid fees.
It is envisaged that, with a learner analytics approach, it will be possible to map for the whole student body what points on paths of study withdrawals occur. It will be further possible to analyse this data comparing withdrawals by disabled students in particular with the student body in general. It is worth noting that the Open University is a large institution with more than 200,000 students, more than 12,000 of whom declare a disability. Hence there is a reasonable chance of data about the point of withdrawal across the educational context revealing that there is something of significance relating to that context over and above the more random distribution of withdrawals for non-study related issues such as health problems and family circumstances.
An analysis has been made of completion and pass rates on all undergraduate modules presented in 2010-11, with a minimum of 10 students who had declared at least one disability (164 modules). The differences in module completion and pass rates between disabled students and non-disabled students falls in the range [-33% to +29%]. On the majority of modules (67%) disabled students fail to complete or do not pass the course in a greater proportion than non-disabled students. If this data was routinely reviewed in a learner analytics approach investigations could be triggered of what might be the factors in module design that might be leading to poor completion and pass rates for disabled students. Further, what are factors in the modules where disabled students are doing as well or better than the non-disabled members of their cohort? The reasons for these could be diverse, but might include issues of accessibility at the teaching and learning level or at the technical level of how the teaching and learning is mediated; which increasingly is web-based. The learner analytics here only indicate where there might be a problem, not what it is.
Now the above approach is only possible in a university context because data is held as to which students identify themselves as having a disability. This would not normally be the case with most public websites for example. However knowing which students have a disability says little about what their needs and preferences might be in interacting with eLearning resources for example. The university does collect slightly refined data but this is to meet the requirements of national statistics agencies. This is coarse grained and based on medical model classifications of disability and not functional requirements with respect to interaction with computer environments. To illustrate the most recent data for all OU disabled students is shown here:
[Source: OU internal data. N.B. some students declare more than one disability; the actual total of disabled students currently registered with the OU is 13,884.]
Now we can infer that students in the Sight, Hearing, Manual Skills, and Dyslexia categories are likely to have web access needs, those in the other categories may too, but we do not know anything about the detail of their needs. Further the access needs will be diverse within any given category. Probably the most effective way of addressing this problem is asking the students to create for themselves on initial registration a profile of the detailed access needs and approaches. On candidate standard to base such a set of profiles on is AccessForAll 3.0 which is currently near finalisation within the IMS Accessibility Working Group. Note I will blog about the AccessForAll 3.0 specification when it goes to public draft which I am informed is imminent. Suffice it to say for this discussion, a learner’s needs and preferences with respect to how the learner can best interact with digital resources is represented using the IMS GLC Access For All Personal Needs and Preferences (PNP) v3.0 specification. This specification includes descriptors for all envisaged access approaches that can be encoded in a variety of ways; probably most likely in the application considered here as user profiles made up of sets of RDF Triples as defined by the vocabulary of the specification. We can set aside the “under the bonnet” discussion for now. All we need to know is that from the student perspective they can complete (once and for all but amend if necessary) a web form detailing their access needs. Then the university has this information, mapped to the PI (Personal Identification) number for each student that does so. Thus for any Learner Analytics approach we now know not just which students have a disability but specifically the nature of their access needs and preferences. These profiles could also be used for other purposes such as personalisation; managing alternative formats and quality assurance of services to disabled students but I will not discuss those here.
Another type of information that is being collected for Learner Analytics purposes is “Click Rate”. This is generated from the automatic monitoring of the frequency of clicks of individual students on all learning resources on the VLE. This gives a reasonable measure of what resources each student accessed, for what period and how actively they interacted with them. This information is stored against PI number for each student.
Now in the 3 sets of data described above we have some powerful tools to assess what is the actual performance and attainment of disabled students compared with their non-disabled peers. Where there appears to be a disparity here we can analyse as to whether web accessibility is likely to key factor. If so targeted remedial action can be instigated to improve accessibility. Further this accessibility improvement is strategically focused where it will have greatest impact on student learning and attainment. This makes best use of the limited resources and staff expertise to address accessibility issues.
In summary I remind you what the 3 sets of data here are:
- Information about the students
Disability flag, disability type, and access needs and preferences profiles completed by the students
- Progression and attainment information
Student module pass rates, grades, and withdrawal data
- Information about activity in the VLE
Information about individual student interaction levels with all specific learning resources
Given ready and simultaneous access to this data it will be possible to construct a wide range of specific accessibility investigations that will identify issues which when addressed will have real impact of the learning of disabled students. What is more, these will be based on actual student interactions with the resources and not just measures of accessibility focussed on the properties of the resources. This approach directly takes into account user experience and context both of which are excluded in approaches based just on evaluation against WCAG2.0.
Comments, questions, discussion and suggestions of collaboration are all welcome.