Abstract
We develop an approach to test suite generation for Constrained Combinatorial Testing (CCT), one of the most widely studied combinatorial testing techniques, based on Answer Set Programming (ASP). The resulting catnap system accepts a CCT instance in fact format and combines it with a first-order encoding for generating test suites, which can subsequently be solved by any off-the-shelf ASP systems. We evaluate the effectiveness of our approach by empirically contrasting it to the best known bounds obtained via dedicated implementations.
This work was partially funded by JSPS (KAKENHI 15K00099) and DFG (SCHA 550/11).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Balduccini, M.: Representing constraint satisfaction problems in answer set programming. In: Faber, W., Lee, J. (eds.) Proceedings of the Second Workshop on Answer Set Programming and Other Computing Paradigms (ASPOCP 2009), pp. 16–30 (2009)
Banbara, M., Inoue, K., Kaufmann, B., Schaub, T., Soh, T., Tamura, N., Wanko, P.: teaspoon: Solving the curriculum-based course timetabling problems with answer set programming. In: Proceedings of the Eleventh International Conference of the Practice and Theory of Automated Timetabling (PATAT 2016), pp. 13–32 (2016)
Banbara, M., Tamura, N., Inoue, K.: Generating event-sequence test cases by answer set programming with the incidence matrix. In: Technical Communications of the 28th International Conference on Logic Programming (ICLP 2012), LIPIcs, vol. 17, pp. 86–97. Schloss Dagstuhl (2012)
Baral, C.: Knowledge Representation, Reasoning and Declarative Problem Solving. Cambridge University Press, Cambridge (2003)
Brain, M., Erdem, E., Inoue, K., Oetsch, J., Pührer, J., Tompits, H., Yilmaz, C.: Event-sequence testing using answer-set programming. Int. J. Adv. Softw. 5(3–4), 237–251 (2012)
Bryce, R.C., Colbourn, C.J.: Prioritized interaction testing for pair-wise coverage with seeding and constraints. Inf. Softw. Technol. 48(10), 960–970 (2006)
Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: an approach to testing based on combinatiorial design. IEEE Trans. Softw. Eng. 23(7), 437–444 (1997)
Cohen, M.B., Dwyer, M.B., Shi, J.: Constructing interaction test suites for highly-configurable systems in the presence of constraints: a greedy approach. IEEE Trans. Softw. Eng. 34(5), 633–650 (2008)
Colbourn, C.J., Dinitz, J.H.: Handbook of Combinatorial Designs, 2nd edn. Chapman & Hall/CRC, Boca Raton (2006)
Czerwonka, J.: Pairwise testing in real world. Practical extensions to test case generators. In: Proceedings of the 24th Annual Pacific Northwest Software Quality Conference, pp. 419–430 (2006)
Erdem, E., Inoue, K., Oetsch, J., Pührer, J., Tompits, H., Yilmaz, C.: Answer-set programming as a new approach to event-sequence testing. In: Proceedings of the 3rd International Conference on Advances in System Testing and Validation Lifecycle (VALID 2011), pp. 25–34. Xpert Publishing Services (2011)
Garvin, B.J., Cohen, M.B., Dwyer, M.B.: An improved meta-heuristic search for constrained interaction testing. In: Proceedings of the first International Symposium on Search Based Software Engineering (SSBSE 2009), pp. 13–22. IEEE (2009)
Gebser, M., Ostrowski, M., Schaub, T.: Constraint answer set solving. In: Hill, P.M., Warren, D.S. (eds.) ICLP 2009. LNCS, vol. 5649, pp. 235–249. Springer, Heidelberg (2009). doi:10.1007/978-3-642-02846-5_22
Lin, J., Luo, C., Cai, S., Su, K., Hao, D., Zhang, L.: TCA: An efficient two-mode meta-heuristic algorithm for combinatorial test generation. In: Proceedings of 30th IEEE/ACM International Conference on Automated Software Engineering (ASE 2015), pp. 494–505. IEEE (2015)
Mandl, R.: Orthogonal latin squares: an application of experiment design to compiler testing. Commun. ACM 28(10), 1054–1058 (1985)
Nanba, T., Tsuchiya, T., Kikuno, T.: Constructing test sets for pairwise testing: a SAT-based approach. In: Proceedings of the Second International Conference on Networking and Computing (ICNC 2011), pp. 271–274. IEEE (2011)
Nie, C., Leung, H.: A survey of combinatorial testing. ACM Comput. Surv. 43(2), 11:1–11:29 (2011). http://doi.acm.org/10.1145/1883612.1883618
Petke, J., Yoo, S., Cohen, M.B., Harman, M.: Efficiency and early fault detection with lower and higher strength combinatorial interaction testing. In: Proceedings of ESEC/FSE 2013, pp. 26–36. ACM (2013)
Tatsumi, K.: Test case design support system. In: Proceedings of the International Conference on Quality Control (ICQC 1987), pp. 615–620 (1987)
Yamada, A., Kitamura, T., Artho, C., Choi, E., Oiwa, Y., Biere, A.: Optimization of combinatorial testing by incremental SAT solving. In: Proceedings of 8th IEEE International Conference on Software Testing, Verification and Validation (ICST 2015), pp. 1–10. IEEE (2015)
Yu, L., Lei, Y., Nourozborazjany, M., Kacker, R., Kuhn, D.R.: An efficient algorithm for constraint handling in combinatorial test generation. In: Proceedings of the Sixth IEEE International Conference on Software Testing, Verification and Validation (ICST 2013), pp. 242–251. IEEE (2013)
Zhang, Z., Yan, J., Zhao, Y., Zhang, J.: Generating combinatorial test suite using combinatorial optimization. J. Syst. Softw. 98, 191–207 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Banbara, M. et al. (2017). catnap: Generating Test Suites of Constrained Combinatorial Testing with Answer Set Programming. In: Balduccini, M., Janhunen, T. (eds) Logic Programming and Nonmonotonic Reasoning. LPNMR 2017. Lecture Notes in Computer Science(), vol 10377. Springer, Cham. https://doi.org/10.1007/978-3-319-61660-5_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-61660-5_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61659-9
Online ISBN: 978-3-319-61660-5
eBook Packages: Computer ScienceComputer Science (R0)