Python Script-based Grading

grader.sh is executed when a learner clicks the submit button, and it runs grader.py to start the grading process. In order to write grader.py, we will explain the commonly used functions in elice_utils.py along with their general formats.

• secure_send_grader(): Sends a message to the learner through the terminal.

• secure_send_score(): Sets the final score for the practice.

from grader_elice_utils import EliceUtils
elice_utils = EliceUtils()
elice_utils.secure_init() # Import and initialize elice_utils.

try: # Perform the grading.
	elice_utils.secure_send_grader('string') # Sends the string to the learner's terminal.
	elice_utils.secure_send_score(100) # Sends the practice score to the learner. When they score 100, a correct answer pop-up will appear.

except Exception: # If an error occurs during the grading process.
	elice_utils.secure_send_score(0) # Set the practice score to 0.
	sys.exit(1)

You can modify grader.py to grade a student's code in various ways.

You can check how to write grader.py through the examples below.

pagecase1. Input/output based gradingpagecase2. Check Specific Conditions

When writing grader.py, please note the following points.

Note 1

Sometimes, when comparing values using if statements, if the values being compared do not match in format and structure, errors such as TypeError or RangeError may occur. In this case, the Grader immediately stops working and the grading process cannot proceed normally. Therefore, it is recommended to use the following code format when using if statements.

try:
	if (submission.drink_relfreq == drink[drink['Attend'] == 1]['Name'].value_counts(normalize=True)).all() :
		total_score += 100
		elice_utils.secure_send_grader('You calculated the relative frequency correctly!\n')
except Exception:
	elice_utils.secure_send_grader('The relative frequency was not calculated correctly. Would you like to try calculating it again?\n')

Note 2

In some cases, specific error messages need to be provided for detailed error information. In these cases, please use internal exceptions to write the messages. You can refer to the following link for Python internal exceptions: https://docs.python.org/3/library/exceptions.html#concrete-exceptions

import student_codes

try:
	if student_codes.answer == 12:
		total_score += 100
		elice_utils.secure_send_grader('Correct answer!\n')

# index range error
except IndexError:
	elice_utils.secure_send_grader('The index is out of range.\n')

# name error
except NameError:
	elice_utils.secure_send_grader('You used a variable that was not declared.\n')

# zero division error
except ZeroDivisionError:
	elice_utils.secure_send_grader('You cannot divide by zero.\n')

Note 3

If the Grader provides specific error messages during grading, the student can potentially deduce the test cases based on the message content. Therefore, it is recommended to write error messages in the following format.

# Not recommended
try:
	# Write grading code
except Exception as err:
	elice_utils.secure_send_grader('Internal Error:\n%s\n' % str(err))
	elice_utils.secure_send_score(0)
	sys.exit(1)

# Recommended
try:
	# Write grading code
except Exception:
	elice_utils.secure_send_grader('An error occurred during the grading process. Please check if your code runs correctly.\n')
	elice_utils.secure_send_score(0)
	sys.exit(1)

"""
What if you wrote the grading code in the not recommended way?
"""
# Full output of grader.py
raise Exception(open('.elice/grader.py').read())