.. _experiment-workspaces:

==========================
Workspaces for experiments
==========================

One of the purposes of Debusine is to allow people to run experiments such as
mass-rebuilds with different parameters, and an easy way to model an experiment
is via a single-use workspace.

In other words, it should be possible to create a purpose-specific workspace to
run and hold results of an experiment, which uses collections and
WorkflowTemplates present in another workspace.

The workspace that acts as environment for an experiment is set up by a
workflow, run in the source workspace.


Parameters for the experiment
=============================

An experiment needs:

* a source workspace
* a name
* (optional) an existing group
* a list of WorkflowTemplates (present in the source workspace) needed to run
  it
* an expiration delay

Collections are inherited from the source workspace via workspace inheritance.


Experiment names
================

Experiment names need to be able to be embedded in workspace names and group
names, and should be validated accordingly.


Ephemeral groups
================

It should be possible to create a single-use group tied to the experiment. This
can be the default if no existing group is provided when creating the
experiment.

The single use group should be marked ephemeral, and should be garbage
collected once it is found with no roles assigned.

The name of a single-use group can be autogenerated in a way that ties it to
the source and experiment workspace names, and minimizes conflicts with
other group names.

The user that creates an experiment is the only member of the newly created
group.

The user that creates an experiment should be able to manage the group
membership, once we have UI for it.


Experiment workspace name
=========================

The workspace that holds the experiment is automatically named by the workflow
that creates it after the source workspace name and the experiment name.

The name is created in a way that minimizes conflicts with other workspace
names.


Experiment workspace contents
=============================

A newly created experiment workspace:

* inherits from the source workspace, getting readonly access to its collections
* has the experiment group set as ``OWNER``
* contains a copy of all WorkflowTemplates selected from the source workspace


Expiration of experiment workspaces
===================================

Experiments are temporary by nature and we need the possibility of experiment
Workspaces to expire. We can reuse the same expiration modeling we have in
``Artifact`` and ``WorkRequest``, adding the corresponding fields to
``Workspace``::

    created_at = models.DateTimeField(auto_now_add=True)
    expiration_delay = models.DurationField(blank=True, null=True)

If feasible, workspaces should expire ``expiration_delay`` days after
``max(created_at, last_task_completion_time)``, instead of after
``created_at``.


Copying WorkflowTemplates
=========================

It makes sense for a copied WorkflowTemplate to be a verbatim copy of the
version in the source workspace at the time of cloning, to avoid an update in
the source workspace to change the way the experiment works mid-way.

We currently have no need for tracking provenance of copies or annotating
them in any special way.


New server tasks
================

The workflow that creates the experiment needs one or more server tasks to
create and provision the new workspace.

As a start it could delegate creation and provisioning to a single server task,
once performing validation of its arguments.

Later, this can be later split into multiple general purpose tasks as the need
arises for them.


Permissions
===========

Any user that is at least ``CONTRIBUTOR`` in a workspace is allowed to create
an experiment from it.

Since this introduces the possibility for users to create workspaces, the
`Workspace` model also gains a ``created_by`` field::

    created_by = models.ForeignKey("User", on_delete=models.PROTECT)

Users creating an experiment workspace can decide to assign one of their groups
the ``OWNER`` role on it, or to have a newly created ephemeral group, of which
they are member, be ``OWNER`` instead.

In both cases, this makes the user have all permissions in the experiment
workspace that derive from the ``OWNER`` role, even if they were only
``CONTRIBUTOR`` on the source workspace.

This means that they will have more permissions in the experiment workspace
than they had in the initial one, and we need to be careful that this does not
grant unexpected rights, like use of signing keys.


Further desired features
========================

When running experiments, it is useful to have features such as:

* pause/resume scheduling of work requests in the workspace
* interrupt a work request or a workflow


Proposed changes
================

* Define an experiment name as needing to satisfy the same regexp as workspace
  and group names
* Define the experiment workspace name as ``{orig_workspace_name}-{experiment_name}``
* Define ephemeral group name the same as the experiment workspace name
* Define the ``WorkflowTemplate`` arguments to create an experiment:

    * experiment name (see above)
    * name of an existing group in the current scope (optional)
    * list of names of WorkflowTemplates present in the current workspace
    * expiration delay (optional integer, with default of 60 days)

* add an "ephemeral" flag to ``Group``, default to ``False``

   * in regular maintenance, check for ephemeral groups with no roles assigned,
     and delete them
   * set the user as a member of the newly created group
   * eventually set the user as an admin of the newly created group

* create a server task that:

   * creates a new workspace, with an expiration_delay
   * sets the new workspace as inheriting an existing workspace
   * creates a new group containing a given user
   * sets the new group as ``OWNER`` of the new workspace
   * copies a list of WorkflowTemplates from an existing workspace to the new
     workspace

* test that one cannot use a signing key from a ``WorkflowTemplate`` copied
  from the source workspace

* show a workspace expiration in the UI, and allow workspace owners to
  change/extend it

* start a new blueprint to design pause/resume/interrupt features.
