A common conundrum for educators in computer science or electrical and computer engineering is how to assess students’ mastery of programming concepts. Assignments are often project-based—after all, a student has to write a program to demonstrate competency thinking through programming structures and implementation.
One of Duke’s newest Coursera specializations, Java Programming: An Introduction to Software, takes a varied approach, with several question types:
- Multiple choice concept questions—learners choose code that makes sense in a certain context
- Multiple choice programming output questions—learners need to identify a certain output from their program
- Free response—learners must have achieved a specific numerical or text answer as a result of building their program correctly
An example of a question that asks a learner to use their programming knowledge to make changes to a program is shown below.
Question 1 comes from a practice quiz, in which learners are prompted to demonstrate that they understand the logic of the class they have just written (in this case, MarkovZero). A Programming Exercise document precedes each practice quiz and gives exact details for algorithms and naming conventions.
The sample questions below illustrate examples of a specific numerical answer generated by a working program and a task that is accomplished by hand, respectively.
To ask the kind of question shown in number 3, it is crucial to provide learners every detail of the algorithm they are implementing, as well as common data files. In Java Programming, we specify the method and parameter names, so that the quiz questions can be unambiguous. We have learned that great attention must be paid to how data files work on different platforms, even accounting for details like how whitespace at the end of a file is interpreted.
Question number 4 requires a student to work a problem by hand, something they are encouraged to do iteratively as they refine their understanding of the algorithm being described. This knowledge is a major step in helping them learn to debug their code systematically, a focal point of this specialization.
The Coursera platform also has options to use machine-graded programming assessments, where the instructor can build a custom grader that tests code that learners submit, but this option was not used in this course, due to the time requirements of specifying a grading algorithm. Instead, the assessments used in this specialization demonstrate that common machine-grading tools, such as multiple choice or numeric answers can be appropriate for programming problems when used carefully.
These examples demonstrate that both formative and summative programming assessment is possible online with commonly available machine-graded question types. It is possible to ask students to make predictions, follow programming logic, or demonstrate that their program works as desired, without having them submit their own code to a human or machine grader. Special care must be taken in wording questions to avoid ambiguity, such as “run the method getMapInfo() immediately after the method buildMap() is called.” Specifying class, method, and variable names is also essential to provide clarity. Testing the assessments—and especially data files—on multiple platforms is also important, although, at least in MOOC world, no bug will go unnoticed for very long!