Material used in the article:

S. Segura, A. Durán, Ana B. Sánchez, D. Le Berre, E. Lonca and A. Ruiz-Cortés. Automated Metamorphic Testing of Variability Analysis Tools. Submitted to Software Testing Verification and Reliability. 2014.


Test data generators (Eclipse projects)

Feature Model Test Data Generator
CUDF Test Data Generator
CNF Formula Test Data Generator

Test data (summary)

Feature Models: 1,000 feature models in XML format plus their corresponding set of products in CSV format.
Feature Models (edit operations): 1,000 couples of feature models in XML format plus their corresponding set of products in CSV format.
Feature Models (prolog): Prolog code with input feature models and their expected outputs. One file per analysis operation under test.
CUDF Documents: 1,000 CUDF documents and their respective set of configurations in CSV format.
CNF Formulas: 10,000 CNF formulas en DIMACS format plus their respective set of solutions in CSV format.

Test results

Reasoner Test Data Test Results
 
FaMa 1.1.2
FLAME 1.0
SPLAR (SAT) 05/04/2013
SPLAR (BDD) 05/04/2013
p2cudf 1.14
aspcudf 1.7
cudf-check 0.6.2-1
Sat4j 2.3.1
Lingeling ala-b02
Minisat 2.2
Clasp 2.1.3
Picosat 535
Rsat 2.0
March_ks (SAT Competition 2007)
March_rw (SAT Competition 2011)
Kcnfs 1.2