Writing your first script

This tutorial will guide you through discovering Quixote and its concepts, in order to allow you to write your own moulinettes easily.

Its only pre-requisites are a working installation of Quixote (see Installing), and basic knowledge of programming.

What is a moulinette ?

A moulinette describes operations that should be applied to a particular piece of data, in order to produce a result. A typical example would be in the context of the validation of an exercise.

With Quixote, a moulinette is a set of scripts and resources specifying the job’s execution environment and behavior. All of these are described in a blueprint.py file which is called the moulinette’s blueprint.

The steps to be taken during a moulinette’s execution are grouped into three phases:

The build phase, which specifies how the job’s environment should be configured (installed applications, dependencies, …)
The fetch phase, which takes care of fetching the data to be processed by the moulinette
The inspection phase, which analyzes the data and produces a result

Each phase can of course include multiple steps.

How does it look like on my computer ?

From the Quixote user’s point-of-view, the moulinette is simply a directory containing the blueprint.py script. Optionally, the moulinette can also come with resources, which are to be put into a resources subdirectory.

The following output of the tree command illustrates an example of the directory structure of a moulinette:

$> tree my_moulinette
my_moulinette
├── blueprint.py
└── resources
    ├── a_cool_resource.txt
    └── a_subdirectory
        └── another_resource.jpg

2 directories, 3 files

Writing a blueprint file

The metadata

The first element of a blueprint is its metadata, which includes its name, its author, etc. Within the blueprint.py file, this is represented by an instance of the quixote.Blueprint class. It can be created like shown below:

import quixote

blueprint = quixote.Blueprint(
    name="my_is_even_demo_moulinette",
    author="Clément 'Doom' Doumergue",
)

The phases and the steps

The second part of the blueprint is about describing the different steps of the moulinette.

Since a step is basically a sequence of instructions to perform, it is implemented by defining a Python function inside the blueprint script.

def almost_a_cool_step():
    pass # This step doesn't do anything :/

As we said before, each step must be part of one of the three phases of moulinette execution. Thus, we also need to mark our step functions appropriately, in order to point out the phase they belong to.

In the script file, we mark functions as part of a phase like so (using one of the three Python decorators provided by Quixote):

import quixote

@quixote.builder
def bonkers_builder():
    pass # Nothing to configure

@quixote.fetcher
def funky_fetcher():
    pass # No data to fetch

@quixote.inspector
def incredible_inspector():
    pass # Nothing to analyze

def not_a_step():
    pass # Not marked, so not a step

Note

Functions that are not marked are not executed by Quixote, but they still can be called by other functions.

When the moulinette is run, phases are executed in order: first, the build phase, then the fetch phase, and finally the inspection phase.

Note

It is possible to mark multiple functions as steps for one phase. For a given phase, the functions are executed in the order in which they are defined (i.e. from top to bottom).

Getting information from the context

In order to reach resource files or other information about the current moulinette, Quixote provides a context as a dictionary, and allows the steps to retrieve data from it.

For example, we can use the following snippet to retrieve the path to a particular resource:

resources_path = get_context()["resources_path"]
file_path = f"{resources_path}/a_cool_resource.txt"

Some values are always guaranteed to be found in the context, such as the activity_id, the module_id, and the targeted group_id, as integers.

Some other values can only be found during particular phases, like the delivery_path, at which the delivery files are stored (available during the fetch and inspection phases).

Note

A cheat sheet of the available values for each phase is available here.

Implementing a step in the build phase

Steps belonging to the build phase shall describe the environment and configuration required in order to run the moulinette. When the moulinette is prepared, instructions in these steps are translated into concrete instructions (suitable for a Dockerfile, or a shell script for instance), in order to build the moulinette’s environment.

For example, one can use the apt built-in to download and install software into the moulinette’s environment. Here is an example of a builder step that installs GCC:

import quixote.build.apt as apt

@quixote.builder
def install_gcc():
    """
    Install the GCC compiler using the APT package manager.
    """

    apt.update()
    apt.install("gcc")

Note

The quixote package offers three sub-packages named respectively build, fetch and inspection, each providing built-ins for the corresponding phase.

The quixote.build module offers a wide range of built-ins for this phase, but it can also be easily extended (see Writing your own “build” built-in).

Implementing a step in the fetch phase

Steps marked as part of the fetch phase are in charge of fetching all the required data that must be analyzed by the moulinette. (These are run outside of the moulinette’s environment).

In most cases, the data is a student delivery that must be fetched from a GitLab repository. This simple task can be achieved using the GitLab built-in.

import quixote.fetch.gitlab as fetch

@quixote.fetcher
def fetch_delivery():
    fetch.gitlab()

Note

That built-in uses the moulinette’s context to figure out which delivery should be fetched in which module.

Implementing a step in the inspection phase

The inspection steps are the most feature-rich in Quixote. They contain the core purpose of the moulinette, which is the processing to perform on the fetched data.

Quixote still provides built-ins for this phase, but the full power of the Python language and its modules can be used.

Our example here shows how we can implement an inspection step that compiles the C files contained in the delivery into a program.

import quixote.inspection.build.gcc as gcc

@quixote.inspector
def compile_delivery():
    """
    Compile all the C files in the student delivery to make a program.
    """

    delivery_path = get_context()["delivery_path"]
    gcc(f"{delivery_path}/*.c", output_file="student")

Wrap-up

Congratulations, you just learned the basics of Quixote! You can download the full blueprint.py for this tutorial. However, so far it is not that useful: it does not verify anything yet! Next, we suggest you take a look at the Validating an exercise tutorial to learn about the different ways to ensure a delivery conforms to an exercise’s requirements.