Coding Overview
course goals
in program
number of students
programming language
how open-ended?
why this open-ended?
assessment experience
rubric content
rubric shared with students
rubric structure
rubric usage
formative assessment plan
summative assessment plan
Cases:
01
02
03
04
05
06
07
08
09
10
11
course goals
Code
01
02
03
04
05
06
07
08
09
10
11
programming paradigms
–
–
–
–
–
–
–
–
–
–
object-oriented programming
–
–
–
–
–
–
–
–
–
functional programming
–
–
–
–
–
–
–
–
–
distributed programming
–
–
–
–
–
–
–
–
–
–
broadened programming skills
–
–
–
–
software project engineering
–
–
–
–
–
–
design
–
–
–
–
–
–
–
–
web
–
–
–
–
–
–
–
–
databases
–
–
–
–
–
–
–
–
games
–
–
–
–
–
–
–
–
–
–
formal methods
–
–
–
–
–
–
–
–
–
–
in program
Code
01
02
03
04
05
06
07
08
09
10
11
BSc CS
MSc CS
–
–
–
–
–
–
–
–
–
–
first-year course
–
–
–
–
–
advanced course
–
–
–
–
–
–
first programming course
–
–
–
–
–
–
–
–
–
–
prior programming experience
–
–
–
–
–
–
number of students
Code
01
02
03
04
05
06
07
08
09
10
11
<50
–
–
–
–
–
–
–
50-100
–
–
–
–
–
–
–
–
–
–
100-200
–
–
–
–
–
–
>200
–
–
–
–
–
–
–
–
–
programming language
Code
01
02
03
04
05
06
07
08
09
10
11
any
–
–
–
–
–
–
–
Python
–
–
–
–
–
–
–
–
–
Java
–
–
–
–
–
–
–
–
–
C#
–
–
–
–
–
–
–
–
–
Haskell
–
–
–
–
–
–
–
–
–
C++
–
–
–
–
–
–
–
–
–
–
Erlang
–
–
–
–
–
–
–
–
–
–
JavaScript
–
–
–
–
–
–
–
–
–
Kotlin
–
–
–
–
–
–
–
–
–
–
Agda
–
–
–
–
–
–
–
–
–
–
how open-ended?
Code
01
02
03
04
05
06
07
08
09
10
11
freedom
–
–
–
–
–
–
–
–
–
–
–
client
–
–
–
–
–
–
–
–
–
own ideas
–
–
–
–
open bonus
–
–
–
–
–
–
chosen bonus
–
–
–
–
–
–
–
–
dissimilar results
–
–
–
–
–
–
start from scratch
–
–
–
–
–
–
–
–
–
freedom of implementation
–
–
–
–
–
–
freedom of technology
–
–
–
–
–
–
–
structure
–
–
–
–
–
–
–
–
–
–
–
specified functionality
–
–
–
–
limited answer possibilities
–
–
–
–
–
–
–
–
–
–
given template
–
–
–
–
–
–
–
technical requirements
–
–
–
–
–
–
incremental steps
–
–
–
–
–
–
–
–
–
structured process
–
–
–
–
–
–
–
–
–
why this open-ended?
Code
01
02
03
04
05
06
07
08
09
10
11
benefits of freedom
–
–
–
–
–
–
–
–
–
–
–
design decisions
–
–
–
–
–
–
–
–
–
realistic
–
–
–
–
fun
–
–
–
–
–
–
–
–
fraud resistance
–
–
–
–
–
–
–
–
flexibility for students
–
–
–
–
–
–
–
–
–
–
learning experience
–
–
–
–
–
showing genuine learning
–
–
–
–
–
–
–
–
–
convenient
–
–
–
–
–
–
–
–
–
–
matches goals
–
–
–
–
–
–
–
–
–
downsides of freedom
–
–
–
–
–
–
–
–
–
–
time-intensive
–
–
–
–
–
–
benefits of structure
–
–
–
–
–
–
–
–
–
–
–
guidance
–
–
–
–
easier checks
–
–
–
–
–
–
–
–
considerations
–
–
–
–
–
–
–
–
–
–
–
no structure required
–
–
–
–
–
–
–
–
–
–
fits course context
–
–
–
–
–
–
–
–
–
legacy
–
–
–
–
–
–
–
–
–
assessment experience
Code
01
02
03
04
05
06
07
08
09
10
11
challenges
–
–
–
–
–
–
–
–
–
–
gen AI
–
–
–
–
–
–
–
–
differentiation in collaboration
–
–
–
–
–
–
–
consistency
–
–
–
–
time intensive
–
–
–
–
product vs process
–
–
–
–
–
–
–
feedback
–
–
–
–
–
–
–
–
transparency
–
–
–
–
–
–
works well
–
–
justification
–
–
–
–
–
–
–
–
–
covers common cases
–
–
–
–
–
–
–
–
–
rubric challenges
–
–
–
–
–
–
–
–
–
–
–
overspecified
–
–
–
–
–
–
–
outcome not matching intuition
–
–
–
–
–
–
–
–
–
–
no matching level
–
–
–
–
–
–
–
–
tooling is lacking
–
–
–
–
–
–
–
–
–
rubric content
Code
01
02
03
04
05
06
07
08
09
10
11
functionality
code constructions
–
–
–
–
–
–
–
code quality
–
–
–
–
–
–
–
code style
–
–
–
design patterns
–
–
–
–
–
–
–
–
design
–
–
testing
–
–
–
–
–
process
–
–
–
–
product vision
–
–
–
–
–
–
–
reporting
–
–
–
–
reflection
–
–
–
–
–
–
–
–
open bonus
–
–
–
–
–
–
–
–
–
rubric shared with students
Code
01
02
03
04
05
06
07
08
09
10
11
rubric shared
–
–
–
–
–
–
rubric not shared
–
–
–
–
–
–
–
–
–
modified rubric shared
–
–
–
–
–
–
–
box ticking
–
–
–
–
–
–
–
students don't look at rubric
–
–
–
–
–
–
–
–
–
rubric structure
Code
01
02
03
04
05
06
07
08
09
10
11
marks
–
–
–
–
–
–
–
level descriptors
–
–
–
–
plain criteria
–
–
–
–
–
–
–
–
–
test cases
–
–
–
–
–
–
–
–
–
common levels
–
–
–
–
–
–
differing levels
–
–
–
–
–
–
–
–
–
list
–
–
–
–
table
–
–
–
–
–
–
analytic grading
–
–
–
holistic grading
–
–
–
–
–
–
–
–
–
rubric usage
Code
01
02
03
04
05
06
07
08
09
10
11
grading
–
–
automated grading
–
–
–
–
–
–
–
–
–
holistic grading
–
–
–
–
–
–
–
–
–
analytic grade suggestion
–
–
–
–
–
–
–
–
–
–
analytic grading
–
–
–
–
–
–
–
–
feedback
–
–
–
–
–
–
–
–
–
automated feedback
–
–
–
–
–
–
–
–
–
rubric not used for feedback
–
–
–
–
–
–
–
teaching assistants
–
–
–
–
–
–
–
further coordination
–
–
–
–
–
–
–
formative assessment plan
Code
01
02
03
04
05
06
07
08
09
10
11
project plan
–
–
–
–
–
–
–
intermediate submission
–
–
–
–
regular session with tutor
–
–
–
–
–
–
–
on request
–
–
–
–
–
–
–
peer feedback
–
–
–
–
–
–
–
–
practice
–
–
–
–
–
–
–
–
–
automation
–
–
–
–
–
–
–
–
–
summative assessment plan
Code
01
02
03
04
05
06
07
08
09
10
11
programming assignment
–
–
–
–
–
–
–
other assignment
–
–
–
–
–
–
–
–
–
–
exam
–
–
–
–
–
–
–
–
integrated project
–
–
–
–
–
–
–
–
–
–
–
report
–
–
–
–
–
–
–
–
peer review
–
–
–
–
–
–
–
–
–
–
product
–
–
–
–
presentation
–
–
–
–
–
–
–
–
meeting
–
–
–
–
–
–
–
–
observations
–
–
–
–
–
–
–
–
reflection
–
–
–
–
–
–
–
–
–
individual contribution
–
–
–
–
–
–
–
–
–