publications

2022

1. FSE

   Perfect Is the Enemy of Test Oracle

   Ibrahimzada, Ali Reza, Varli, Yigit, Tekinoglu, Dilara, and Jabbarvand, Reyhaneh

   In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’22), November 14–18, 2022, Singapore, Singapore

   Abs DOI PDF Code Slides Video

   Automation of test oracles is one of the most challenging facets of software testing, but remains comparatively less addressed compared to automated test input generation. Test oracles rely on a ground-truth that can distinguish between the correct and buggy behavior to determine whether a test fails (detects a bug) or passes. What makes the oracle problem challenging and undecidable is the assumption that the ground-truth should know the exact expected, correct or buggy behavior. However, we argue that one can still build an accurate oracle without knowing the exact correct or buggy behavior, but how these two might differ. This paper presents SEER, a Deep Learning-based approach that in the absence of test assertions or other types of oracle, can automatically determine whether a unit test passes or fails on a given method under test (MUT). To build the ground-truth, SEER jointly embeds unit tests and the implementation of MUTs into a unified vector space, in such a way that the neural representation of tests are similar to that of MUTs they pass on them, but dissimilar to MUTs they fail on them. The classifier built on top of this vector representation serves as the oracle to generate “fail” labels, when test inputs detect a bug in MUT or “pass” labels, otherwise. Our extensive experiments on applying SEER to more than 5K unit tests from a diverse set of opensource Java projects show that the produced oracle is (1) effective in predicting the fail or pass labels, achieving an overall accuracy, precision, recall, and F1 measure of 93%, 86%, 94%, and 90%, (2) generalizable, predicting the labels for the unit test of projects that were not in training or validation set with negligible performance drop, and (3) efficient, detecting the existence of bugs in only 6.5 milliseconds on average. Moreover, by interpreting the neural model and looking at it beyond a closed-box solution, we confirm that the oracle is valid, i.e., it predicts the labels through learning relevant features.

2. MBEC

   Predicting the Predisposition to Colorectal Cancer based on SNP Profiles of Immune Phenotypes using Supervised Learning Models

   Cakmak, Ali, Ibrahimzada, Ali Reza, Arikan, Soykan, Ayaz, Huzeyfe, Demirkol, Seyda, Sonmez, Dilara, Hakan, Mehmet Tolgahan, Surmen, Saime Turan, Horozoglu, Cem, Kucukhuseyin, Ozlem, Cacina, Canan, Kiran, Bayram, Zeybek, Umit, Baysan, Mehmet, and Yaylim, Ilhan

   Medical & Biological Engineering & Computing, 2022

   Abs DOI PDF Code

   This study explores the machine learning-based assessment of predisposition to colorectal cancer based on single nucleotide polymorphisms (SNP). Such a computational approach may be used as a risk indicator and an auxiliary diagnosis method that complements the traditional methods such as biopsy and CT scan. Moreover, it may be used to develop a low-cost screening test for the early detection of colorectal cancers to improve public health. We employ several supervised classification algorithms. Besides, we apply data imputation to fill in the missing genotype values. The employed dataset includes SNPs observed in particular colorectalcancer-associated genomic loci that are located within DNA regions of 11 selected genes obtained from 115 individuals. We make the following observations: (i) Random Forest-based classifier using one-hot encoding and K-Nearest Neighbor (KNN)-based imputation performs the best among the studied classifiers with an F1 score of 89% and Area Under the Curve (AUC) score of 0.96. (ii) One-hot encoding together with K-Nearest-Neighbor-based data imputation increase the F1 scores by around 26% in comparison to the baseline approach which does not employ them. (iii) The proposed model outperforms a commonly employed state-of-the-art approach, ColonFlag, under all evaluated settings by up to 24% in terms of the AUC score. Based on the high accuracy of the constructed predictive models, the studied 11 genes may be considered as a gene panel candidate for colon cancer risk screening.

2020

1. JBCB

   Scalable classification of organisms into a taxonomy using hierarchical supervised learners

   Sohsah, Gihad N., Ibrahimzada, Ali Reza, Ayaz, Huzeyfe, and Cakmak, Ali

   Journal of Bioinformatics and Computational Biology, World Scientific Publishing Co., Vol. 18, No. 05, 2020

   Abs DOI PDF Code

   Accurately identifying organisms based on their partially available genetic material is an important task to explore the phylogenetic diversity in an environment. Specific fragments in the DNA sequence of a living organism have been defined as DNA barcodes and can be used as markers to identify species efficiently and effectively. The existing DNA barcode-based classification approaches suffer from three major issues: (i) most of them assume that the classification is done within a given taxonomic class and/or input sequences are pre-aligned, (ii) highly performing classifiers, such as SVM, cannot scale to large taxonomies due to high memory requirements, (iii) mutations and noise in input DNA sequences greatly reduce the taxonomic classification score. In order to address these issues, we propose a multi-level hierarchical classifier framework to automatically assign taxonomy labels to DNA sequences. We utilize an alignment-free approach called spectrum kernel method for feature extraction. We build a proof-of-concept hierarchical classifier with two levels, and evaluated it on real DNA sequence data from barcode of life data systems. We demonstrate that the proposed framework provides higher f1-score than regular classifiers. Besides, hierarchical framework scales better to large datasets enabling researchers to employ classifiers with high classification performance and high memory requirement on large datasets. Furthermore, we show that the proposed framework is more robust to mutations and noise in sequence data than the non-hierarchical classifiers.

2019

1. IMMC

   Predicting the Predisposition to Colorectal Cancer based on SNP Profiles of Immune Checkpoints Using Supervised Learning Models

   Cakmak, Ali, Ibrahimzada, Ali Reza, Arikan, Soykan, Ayaz, Huzeyfe, Demirkol, Seyda, Sonmez, Dilara, Hakan, Mehmet Tolgahan, Surmen, Saime Turan, Horozoglu, Cem, Kucukhuseyin, Ozlem, Cacina, Canan, Kiran, Bayram, Zeybek, Umit, Baysan, Mehmet, and Yaylim, Ilhan

   VII. International Molecular Medicine Congress, September 2019, Istanbul, Turkey

manuscripts under review

2021

1. IEEE

   Predicting Student Grades in Courses: A Clustering-Based Approach

   Ibrahimzada, Ali Reza, and Cakmak, Ali

   IEEE Transactions on Learning Technologies

   Abs

   Intelligent systems are getting more popular for predicting student success and obtaining useful insights from educational data. Traditional educational institutions fail to fully utilize their students’ data to enhance their instructional setting. Interpreting large amounts of educational data may be quite essential for institutions. For instance, such intelligent systems may address questions like “how many credits should a particular student be allowed in a semester in order to improve her grade point average with no or minimal extension on her graduation time?” or “which students in a class should be proactively offered additional tutoring to increase the success rate of a class?”. A fundamental problem that needs to be dealt with while addressing such questions is predicting a student’s success in a particular course ahead of time. In this paper, we propose a general clustering-based framework to estimate the grades of students in a course before the course commences. The clustering-based framework allows the grouping of similar students prior to fitting a learning model. Using a separate model for each group of similar individuals can significantly increase the grade estimation accuracy, as it is much easier to find patterns in a more homogeneous student group. We demonstrate the application of the proposed grade prediction model on a real-life dataset. The experimental results show that our proposed framework outperforms the state of the art by approximately 36% improvement in error rates.