The Coq mailing list

[Jannis Limperg <jannis AT limperg.de>]

Dear all,

We are looking for motivated people to be members of the ICFP 2022
Artifact Evaluation Committee (AEC); students, researchers and people
from the industry or the free-software community are all welcome. The
artifact evaluation process aims to improve the quality and
reproducibility of research artifacts for ICFP papers.

You can either nominate yourself or nominate someone else (students,
colleagues, etc; we will of course check later that this person is
willing to be a committee member) by filling the nomination form.

Nomination form:
  https://framaforms.org/icfp22-aec-nomination-form-1649450477

For more information, see the AEC webpage:
  https://icfp22.sigplan.org/track/icfp-2022-artifact-evaluation

The primary responsibility of committee members is to review the
artifacts submitted corresponding to the already accepted papers in
the main research track. In particular, run the associated tool, check
whether the results in the main paper can be reproduced, and inspect
the tool and the data.

We expect evaluation of one artifact to take about a full day, and
each committee member to receive 2 to 3 artifacts to review.

All of the AEC work will be done remotely/online. The ICFP AEC
committee will work in June, with the review work happening between
June 2nd and June 28th.

Here are some reasons why we think nominees may like it:

1. Reviewing artifacts helps our community improve the quality of its
   artifacts, such as research software and mechanized
   proofs. Participating to this effort feels valuable and useful.

2. Nominees may learn some nice tricks by studying how other researchers
   implement, evaluate and document their ideas.

3. There is some evidence that members of the artifact-evaluation
   committee like the job: about a third of last year's evaluators
   volunteered to join the committee again this year.

(The cons: unpaid work, less prestigious than reviewing papers, the
evaluation process is less clearly-defined and well-understood than
for papers.)

Come join us in improving the quality of research in our field!

Best,
the Artifact Evaluation chairs: Jannis Limperg and Gabriel Scherer