grader.sh is executed when a learner clicks the submit button, and it runs grader.py to start the grading process. In order to write grader.py, we will explain the commonly used functions in elice_utils.py along with their general formats.
secure_send_grader(): Sends a message to the learner through the terminal.
secure_send_score(): Sets the final score for the practice.
from grader_elice_utils import EliceUtilselice_utils =EliceUtils()elice_utils.secure_init()# Import and initialize elice_utils.try:# Perform the grading. elice_utils.secure_send_grader('string')# Sends the string to the learner's terminal. elice_utils.secure_send_score(100) # Sends the practice score to the learner. When they score 100, a correct answer pop-up will appear.
exceptException:# If an error occurs during the grading process. elice_utils.secure_send_score(0)# Set the practice score to 0. sys.exit(1)
You can modify grader.py to grade a student's code in various ways.
You can check how to write grader.py through the examples below.
When writing grader.py, please note the following points.
Note 1
Sometimes, when comparing values using if statements, if the values being compared do not match in format and structure, errors such as TypeError or RangeError may occur. In this case, the Grader immediately stops working and the grading process cannot proceed normally. Therefore, it is recommended to use the following code format when using if statements.
try:if (submission.drink_relfreq == drink[drink['Attend']==1]['Name'].value_counts(normalize=True)).all(): total_score +=100 elice_utils.secure_send_grader('You calculated the relative frequency correctly!\n')exceptException: elice_utils.secure_send_grader('The relative frequency was not calculated correctly. Would you like to try calculating it again?\n')
Note 2
In some cases, specific error messages need to be provided for detailed error information. In these cases, please use internal exceptions to write the messages. You can refer to the following link for Python internal exceptions: https://docs.python.org/3/library/exceptions.html#concrete-exceptions
import student_codestry:if student_codes.answer ==12: total_score +=100 elice_utils.secure_send_grader('Correct answer!\n')# index range errorexceptIndexError: elice_utils.secure_send_grader('The index is out of range.\n')# name errorexceptNameError: elice_utils.secure_send_grader('You used a variable that was not declared.\n')# zero division errorexceptZeroDivisionError: elice_utils.secure_send_grader('You cannot divide by zero.\n')
Note 3
If the Grader provides specific error messages during grading, the student can potentially deduce the test cases based on the message content. Therefore, it is recommended to write error messages in the following format.
# Not recommendedtry:# Write grading codeexceptExceptionas err: elice_utils.secure_send_grader('Internal Error:\n%s\n'%str(err)) elice_utils.secure_send_score(0) sys.exit(1)# Recommendedtry:# Write grading codeexceptException: elice_utils.secure_send_grader('An error occurred during the grading process. Please check if your code runs correctly.\n')
elice_utils.secure_send_score(0) sys.exit(1)"""What if you wrote the grading code in the not recommended way?"""# Full output of grader.pyraiseException(open('.elice/grader.py').read())