Learning Objectives in MOOCs

We are learning more and more about who enrolls in Massive Open Online Classes (MOOCs) and how those students behave.  For example, Harvard and MIT recently released de-identified data from their first 16 MOOCs that ran in 2012-2013 (read more about the Harvard and MIT data sets here and access the actual data here).  The data set includes several variables relating to student activities – for example, whether students visited the course website, watched videos, or completed exams.  These types of measures can tell us a lot about what students do, but it is not clear how much they learned as a result of those actions.

We were interested to find out how much students gained in specific learning objectives as a result of participating in a MOOC.  To do this, we asked students to rate their learning gains in five areas that roughly correspond to Bloom’s taxonomy of learning outcomes.  Bloom’s taxonomy is a classification of learning outcomes that includes both lower-level outcomes (remembering, understanding) and a progression towards higher-level outcomes (evaluating, creating).

6700283_orig

We sent a post-course survey to students in Dorian’s Canelas’ Introduction to Chemistry course asking them to indicate the extent to which the course contributed to their progress on five different learning objectives.  The survey was completed by 382 students, and it should be noted that this does not represent a random sample of students in the course.  Rather, the findings generated from this survey are indicative of what some of the more-engaged students experienced.

Of the students who completed the survey, 62% earned a Statement of Accomplishment.  However, even those who did not earn a Statement reported having a very positive experience in the course.  As shown below, when asked to rate their overall experience with the course, the overwhelming majority of all students rated it highly.

Slide03

The graph below shows student’s self-reported progress on the five learning objectives we asked about.  The percents reported are the percent of students who said that the course contributed “highly” or “very highly” to their progress (other options were “not at all”, “a little”, and “moderately”).

Slide12

As is typical in a traditional class, most students made significant progress on the lower-level outcomes of gaining knowledge and understanding basic concepts.  However, over half the students reported that they made progress with the higher-level outcomes of applying knowledge to other situations and synthesizing information.  Finally, 42% of students made progress learning to conduct their own inquiry, an objective at the top of the taxonomy.  This group does not include only those who completed the course and earned a Statement.  Of those who did not earn a Statement, 33% said that they made significant progress learning to conduct inquiry.

These numbers highlight the need to think carefully about how we define success in a MOOC.  It is increasingly clear that students who do not earn certificates at the end and who do not meet the traditional metrics of completion are still having meaningful engagements with the course material and accomplishing learning gains.

2 thoughts on “Learning Objectives in MOOCs

  1. Pingback: Education & Instructional Technology Tweet Wrap for the Week of 06-09-14

  2. Maureen Coffey

    “… they made progress with the higher-level outcomes of applying knowledge to other situations and synthesizing information …” – Were these student-answered questions only? Would it not be likely that -since everyone wants the most positive outcome from an effort they spend time on- that students themselves would evaluate themselves more positively than e.g. an exam would if taken at the same time? This is not to relativize the quality or impact of MOOCs. Being an autodidact myself, I have long advocated for things like MOOCs (and used to do the “pen-and-paper form” of it back when no Internet existed – correspondence courses) but in the end most of these courses end with multiple choice questionnaires at best. And these types of test in my opinion cannot capture the higher level of Bloom’s paradigm on learning results?!

Comments are closed.