Taskgrader

The taskgrader tool manages every step of grading a contest task, from the generation of test data to the grading of a solution output.

It allows for a wide variety of contest task types and languages to be evaluated, and is meant to be used both locally for tests and in contest evaluation settings.

It uses isolate as a sandbox to run solutions to limit execution time and memory, as well as their access to the environment.

This documentation covers:

• Getting started:
  • Installation
  • Basic usage of the taskgrader
  • How to write tasks for use with the taskgrader
  • Input JSON to send to the taskgrader
• Reference:
  • taskSettings.json, the settings for auto-detection of the task components and parameters
  • defaultParams.json, the full description of the task for direct use by evaluation processes
• Error messages and their meaning
• Further information on the taskgrader internals