PPoPP 2016
Sat 12 - Wed 16 March 2016 Barcelona, Spain

Traditionally, technical research papers are published without including any artifacts (such as tools, data, models, videos, etc.), even though the artifacts may serve as crucial and detailed evidence for the quality of the results that the associated paper offers. They support the repeatability of experiments and precise comparison with alternative approaches, thus enabling higher quality in the research area as a whole. They may also make it easier for other researchers to perform their own experiments, thus helping the original authors disseminating their ideas in detail. Hence, artifacts should be taken seriously and recognized separately.

Last year, PPoPP 2015 successfully ran artifact evaluation process for the first time. PPoPP 2016 continues this experiment.

Artifacts have been already evaluated at several other conferences and workshops including recent CGO 2015, PLDI 2014, OOPSLA 2014 and ADAPT 2014. Our eventual goal is to develop common evaluation methodology gradually and collaboratively.

Mon 14 Mar

Artifact Evaluation
18:00 - 18:30: Artifact Evaluation - Artifact Evaluation at Mallorca+Menorca
Artifact Evaluation145797480000018:00 - 18:30

Call for Artifact Evaluations

Updated: The Artifact Evaluation procedure for PPoPP 2016 has finished. The list of accepted artifacts can be found at http://ctuning.org/ae/artifacts.html

Authors of accepted papers will be invited to formally submit their supporting materials to the Artifact Evaluation process. The Artifact Evaluation process is run by a separate committee whose task is to assess how the artifacts support the work described in the papers. This submission is voluntary and will not influence the final decision regarding the papers. Papers that go through the Artifact Evaluation process successfully will receive a seal of approval printed on the papers themselves. Further information is available on the PPoPP AEC web page. Authors of accepted papers are encouraged (but not obliged) to make these materials publicly available upon publication of the proceedings, by including them as “source materials” in the ACM Digital Library.

Important dates

  • Paper decision: 10 Nov 2015
  • Artifact submission: 20 Nov 2015 AoE
  • Technical clarification: 14-20 Dec 2015
  • Decision announced: 22 Dec 2015
  • Public discussion: 14 March 2016

How to pack and submit

Please read the guidelines on what to submit. Please upload your submission to EasyChair (joint CGO/PPoPP submission).

Review process

The review process is described in detail here.

Artifact Evaluation Committee (AEC)


Committee (joint with CGO'16):


We consider AE as a continuous learning curve. If you have questions or comments and suggestions on how to improve packaging and reviewing process, please get in touch!

Our presentation about Artifact Evaluation Experience at CGO/PPoPP'15 is available online here.

Artifact Evaluation

Important Dates
Tue 22 Dec 2015
Mon 14 - Sun 20 Dec 2015
Technical clarification
Fri 20 Nov 2015