Weighted Distance Based Discriminant Analysis: The R Package WeDiBaDis

The WeDiBaDis package provides a user friendly environment to perform discriminant analysis (supervised classiﬁcation). WeDiBaDis is an easy to use package addressed to the biological and medical communities, and in general, to researchers interested in applied studies. It can be suitable when the user is interested in the problem of constructing a discriminant rule on the basis of distances between a relatively small number of instances or units of known unbalanced-class membership measured on many (possibly thousands) features of any type. This is a current situation when analyzing genetic biomedical data. This discriminant rule can then be used both, as a means of explaining differences among classes, but also in the important task of assigning the class membership for new unlabeled units. Our package implements two discriminant analysis procedures in an R environment: the well-known distance-based discriminant analysis (DB-discriminant) and a weighted-distance-based discriminant (WDB-discriminant), a novel classiﬁer rule that we introduce. This new procedure is based on an improvement of the DB rule taking into account the statistical depth of the units. This article presents both classifying procedures and describes the implementation of each in detail. We illustrate the use of the package using an ecological and a genetic experimental example. Finally, we illustrate the effectiveness of the new proposed procedure (WDB), as compared with DB. This comparison is carried out using thirty-eight, high-dimensional, class-unbalanced, cancer data sets, three of which include clinical features.


Introduction
Discriminant analysis (supervised classification) is used to differentiate between two or more naturally occurring groups based on a suite of discriminating features.This analysis can be used as a means of explaining differences among groups and for classification.That is, to develop a rule based on features measured on a group of units with known membership (the so-called training set), and to use this classification rule to assign a class membership to new unlabeled units.Classification is used by researchers in a wide variety of settings and fields including biological and medical sciences.For example, in biology it is used for taxonomic classification, morphometric analysis for species identification, and to study species distribution.Discriminant analysis is applicable to a wide range of ecological problems, e.g., testing for niche separation by sympatric species or for the presence or absence of a particular species.Marine ecologists commonly use discriminant analysis to evaluate the similarity of distinct populations and to classify units of unknown origin to known populations.The discriminant technique is also used in genetic studies in order to summarize the genetic differentiation between groups.In studies with Single Nucleotide Polymorphism (SNP) or re-sequencing data sets, usually the number of variables (alleles) is greater than the number of observations (units), so discriminant methods are available for data sets with more variables than units, as necessary.Furthermore, class prediction is currently one of the most important tasks in biomedical studies.The diagnosis of diseases, as cancer type or psychiatric disorder, has recently received a great deal of attention.With actual data, classification presents serious difficulties, because diagnosis is based on both clinical/pathological features (usually nominal data) and gene expression information (continuous data).For this reason, classification rules that could be applied to all types of data are desirable.The most popular classification rules are the linear (LDA) and quadratic (QDA) discriminant analyses (Fisher, 1936), which are easy to use as they are found in most statistical packages.However, they require the assumption of normally distributed data; when this condition is violated, their use may yield poor classification results.Many distinct classifiers exist, differing in the definition of the classification rule and whether they utilize statistical (Golub et al., 1999;Hastie et al., 2001) or machine learning (Breiman, 2001;Boulesteix et al., 2008) methods.However, the problem of classification with data obtained from microarrays is challenging because there are a large number of genes and a relatively small number of samples.In this situation, the classification methods based on the within-class covariance matrix fail, as an inverse is not defined.This is known as the singularity or under-sample problem (Krzanowski et al., 1995).The shrunken centroid method can be seen as a modification of the diagonal discriminant analysis (Dudoit et al., 2002) and was developed for continuous high-dimensional data (Tibshirani et al., 2002).Nowadays, another issue that requires attention is the class-unbalanced situation, that is, the number of units belonging to each class is not the same.Some classifiers on class-unbalanced data tend to classify most of the new data in the majority class.This bias is higher when using high dimensional data.Recently, a method which improves the shrunken centroid method when the high-dimensional data is class-unbalanced was presented The R Journal Vol.8/2, December 2016 ISSN 2073-4859 (Blagus and Lusa, 2013).Furthermore, some statistical approaches are characterized by having an explicit underlying probability model, but it is not possible to always assume this requirement.One of the most popular nonparametric, machine-learning, classification methods is the k-nearest neighbor classification (k-NN) (Cover and Hart, 1967;Duda et al., 2000).Given a new unit to be classified, this method finds the k nearest neighbors and classifies the new unit in the class to which belong the majority of neighbours.This classification may depend on the selected value for k.As ecologists have repeatedly argued, the Euclidean distance is inappropriate for raw species abundance data involving null abundances (Orloci, 1967;Legendre and Legendre, 1998) and it is necessary to use discriminant analyses that incorporate adequate distances.In this situation, discriminant analysis based on distances (DB-discriminant), where any symmetric distance or dissimilarity function can be used, is a useful alternative (Cuadras, 1989(Cuadras, , 1992;;Cuadras et al., 1997;Anderson and Robinson, 2003).
To our knowledge, this technique is only included in GINGKO a suite of programs for multivariate analysis, oriented towards ordination and classification of ecological data (De Caceres et al., 2003;Bouin, 2005;Kent, 2011).These programs are written in Java language, so it is therefore necessary to have a Java Virtual Machine to execute it.Even though GINGKO is a very useful tool, it does not provide the option of a class prediction for new unlabeled units or feature selection.Recently, data depth was proposed as the basis for nonparametric classifiers (Jornstein, 2004;Ghosh and Chaudhuri, 2005;Jin and Cui, 2010;Hlubinka and Vencalek, 2013).A depth of a unit is a nonnegative number, which measures the centrality of the unit.That is, depth in the sample version reflects position of the unit with respect to the observed data cloud.The so-called maximal depth classifier is the simple and natural classifier defined from a depth function: to allocate a new observation to the class to which it has maximal depth.There are many possibilities how to define the depth of the data (Liu, 1990;Vardi and Zhang, 2000;Zuo and Serfling, 2000;Serfling, 2002), nevertheless the computation of the most popular depth functions is very slow, in particular, for high-dimensional data the time needed for classification grows rapidly.A new less-computer intensive depth function I (Irigoien et al., 2013a) was developed, but the authors did not study its use in relation to the classification problem.
A discriminant method should have several abilities.First, the classifier rule has to be able to properly separate the classes.In this sense, the classifier evaluation is most often based on the error rate, the percentage of incorrect prediction divided by the total number of predictions.Second, the rule has to be useful to classify new unlabeled units.Then, cross validation evaluation is needed.Cross-validation involves a series of sub-experiments, each of which involves the removal of a subset of objects from a data set (the test set), construction of a classifier using the remaining objects in the data set (the model building set), and subsequent application of the resulting model to the removed objects.The leave-one-out method is a special case of cross-validation; it considers each single object in the data set as a test set.Furthermore, other measures, such as the sensitivity, specificity, positive predictive value for each class, and the generalized correlation coefficient, are useful to know the ability of the rule in the prediction task.
Here we introduce WeDiBaDis, an R package which provides a user-friendly interface to run the DB-discriminant analysis and a new classification procedure, the weighted-distance-based discriminant (WDB-discriminant) that performs well and improves the DB-discriminant rule.It is based on both, the DB-discriminant rule and the depth function I (Irigoien et al., 2013a).First, we will describe the DB and WDB discriminant rules.Then, we will provide details about the WeDiBaDis package and will illustrate its use and its main outputs using an ecological and a genetic data set.To compare both DB and WDB rules-and in order to avoid the criticism that artificial data can favour particular methods-we present a large analysis of thirty-eight, high-dimensional, class-unbalanced, cancer gene expression data sets, three of which include clinical features.Furthermore, the data sets include more than two classes.Finally, we conclude the paper presenting the main conclusions.WeDiBaDis is available at https://github.com/ItziarI/WeDiBaDis.

Discriminant rules and evaluation criteria
Let y i (i = 1, 2, . . ., n) be m-dimensional units measured in any kind of features, with associated class labels l i ∈ {1, 2, . . ., K}, where n and K denote the number of units and classes, respectively.Let Y be the matrix of all units and d a distance defined between any pair of units, d ij = d(y i , y j ).Let y * be a new unlabeled unit to be classified in one of the given classes C k , k = 1, 2, . . ., K.

DB-discriminant
The distance-based or DB-discriminant rule (Cuadras et al., 1997) takes as a discriminant score The R Journal Vol.
where n k indicates the number of units in class k.Note that the second term in (2), The DB classification rule allocates y * to the class which has the minimal proximity value: That is, this distance-based rule assigns a unit to the nearest group.Furthermore, using appropriate distances, Equation (3) reduces to some classic and well-studied rules (see Table 1 in Cuadras et al. 1997).For example, under the normality assumption, Equation ( 3) is equivalent to a linear discriminant or to a quadratic discriminant if the Mahalanobis distance or the Mahalanobis distance plus a constant is selected, respectively.

WDB-discriminant
For any unit y, let I k be the depth function in class C k defined by (Irigoien et al., 2013a), Function I takes values in [0, 1] and it verifies the following desirable properties: For a distribution having a uniquely defined "center" I attains maximum value at this center (maximality at center); When one unit moves away from the deepest unit (the unit at which the depth function attains maximum value; in particular, for a symmetric distribution, the center) along any fixed ray through the center, the depth at this unit decreases monotonically (monotonicity relative to the deepest point) and the depth of a unit y should approach zero as ||y|| approaches infinity (vanishing at infinity).According to the distance used, the depth of a unit may or may not depend on the underlying coordinate system or, in particular, of the scales of the underlying measurements.In any case the affine invariance holds for translations and rotations.Thus, according to Zuo and Serfling (2000), I is a type C depth function.As I is a depth function, it assigns to any observation a degree of centrality.While most of the depth functions assign zero depth to units outside a convex hull and then, it is possible that some training units have zero depth, the function in Equation ( 4) attains the zero value if V(C k ) = 0, that is, in presence of a constant distribution.
For each class C k we weight the discriminant score δ 1 k by 1 − I k (y * ), that is, given a new unit y * , we define a new discriminant score for class k by: The shrinkage we use, reduces the proximity values, this reduction being greater for deeper units.Thus, this new classification rule, allocates a new unit y * to the class which has the minimal proximity and maximal depth values.

Evaluation criteria
First consider the case of two classes (K = 2) and the most common measures of performance for a classification rule.As it is usual in medical statistics, for a fixed class k, let TP, FN, FP, and TN denote the true positive (number of units of class k correctly classified in class k), the false negative (number of units of class k misclassified as units in class l, with l = k), the false positive (number of units of class l, with l = k misclassified as units in class k), and the true negative (number of units of The R Journal Vol.8/2, December 2016 ISSN 2073-4859 class l, with l = k correctly classified as units in class l), respectively.Then (Zhou et al., 2002), the sensitivity (recall) for class k is defined as the ability of a rule to correctly classify units belonging to class k, thus Q se k = TP TP+FN .The specificity is the ability of a rule to correctly exclude a unit from class k when it really belongs to another class, thus Q sp k = TN TN+FP .Furthermore, the positive predictive value (precision) is the probability that a classification in class k is correct, thus P + k = TP TP+FP and the negative predictive value is the probability that a classification in class l with l = k is correct, thus P − k = TN TN+FN .However, these measures do not take into account all the TP, FN, FP and TN values.For this reason, in biomedical applications the Matthew's correlation coefficient (Matthews, 1975) MC it is often used.This is defined by: .
It ranges from −1 if all the classifications are wrong to +1 for perfect classification.A value equal to zero indicates that the classifications are random or the classifier always predicts only one of the two classes.
In the general case of K classes with K ≥ 2, one obtains a K × K contingency or confusion matrix Z = (z kl ), where z kl is the number of times that units are classified to be in class l while belonging in reality to class k.Then, z k. = ∑ l z kl and z .l= ∑ k z kl represent the number of units belonging to class k and the number of units predicted to be in class l, respectively.Obviously One standard criterium to evaluate a classification rule is to compute the percentage of all correct predictions, the percentage of units correctly predicted to belong to class k relative to the total number of units in class k (sensitivity for class k), the percentage of units correctly predicted to belong to any class l with l = k relative to the total number of units in any class l with l = k (specificity of class k), and the percentage of units correctly classified to be in class k with respect to the total number of units classified in class k (positive predictive value for class k), However, we also consider a generalization of the Matthew's correlation coefficient, the so called generalized squared correlation GC 2 (Baldi et al., 2000), which is defined by where e kl = z k. z .ln .This coefficient ranges between 0 and 1, and may often provide a much more balanced evaluation of the prediction than, for instance, the above percentages.A value equal to zero indicates that there is at least one class in which no units are classified.
Another interesting coefficient is the Kappa statistic, which measures the agreement of classification to the true class (Cohen, 1960;Landis and Koch, 1977).It can be calculated by: Finally, another measure used as a result of classification is the F 1 statistic (Powers, 2011).For each class, it is calculated based on the precision P + k and the recall Q se k as follows: . However, note that F 1 does not take the true negatives into account.

Distance functions
The DB and WDB procedures require the previous calculation of a distance between units.In biomedical, genetic, and ecological studies different types of dissimilarities are frequently used.For this reason, WeDiBaDis includes several distance functions.Although these distances can be found in other packages they were included for ease their use for non-expert R users.
The package contains the usual Euclidean distance, the well known correlation distance, where r is the Pearson correlation coefficient, and the Mahalanobis distance (Mahalanobis, 1936) with S the variance-covariance matrix, The function named mahalanobis() that calculates the Mahalanobis distance already exists in the stats package, but it is not suitable in our context.While this function calculates the Mahalanobis distance with respect to a given center, our function is designed to calculate the Mahalanobis distance between each pair of units given a data matrix.
Next, we briefly comment on the other distances included in the package.The Bhattacharyya distance (Bhattacharyya, 1946) is a very well-known distance between populations in the genetic context.Each population is characterized by a vector (p i1 , . . ., p im ) whose coordinates are the relative frequencies of the features (usually chromosomal arrangements), with Then, the distance between two units (populations) with frequencies y i = (p i1 , . . ., p im ) and y j = (p j1 , . . ., p jm ) is defined by: The Gower distance (Gower, 1971), used for mixed variables, is defined by: where s(y i , y j ) is the similarity coefficient between unit y i = (x i , q i , b i ) and unit y j = (x j , q j , b j ), and x ., q ., b .are the values for the m 1 continuous, m 2 binary and m 3 qualitative features, respectively.The coefficient s(y i , y j ) is calculated by: with R l the range of the lth continuous variable (l = 1, . . ., m 1 ); for the m 2 binary variables, a and d represent the number of matches presence-presence and absence-absence, respectively; and α is the number of matches between states for the m 3 qualitative variables.Note that there is also the daisy() function in the cluster package, which can calculate the Gower distance for mixed variables.The difference between this function and dGower() in WeDiBaDis is that in daisy() the distance is calculated as d(y i , y j ) = 1 − s(y i , y j ) and in dGower() as d(y i , y j ) = 2(1 − s(y i , y j )).Moreover, dGower() allows us to include missing values (such as NA) and therefore calculates distances based The R Journal Vol.8/2, December 2016 ISSN 2073-4859 on Gower's weighted similarity coefficients.The dGower() function improves the function dgower() included in the package ICGE (Irigoien et al., 2013b).
The Bray-Curtis distance (Bray and Curtis, 1957) is one of the most well-known ways of quantifying the difference between samples when the information is ecological abundance data collected at different sampling locations.It is defined by: where y il , y jl are the abundance of specie l in samples i and j, respectively, and y i+ , y j+ are the total specie's abundance in samples i and j, respectively.This distance can be also found in the vegan package.
The Hellinger (Rao, 1995) and Orloci (or chord distance) (Orloci, 1967) distances are also measures recommended for quantifying differences between sampling locations when the ecological abundance of species is collected.The Hellinger distance is given by: and the Orloci distance that represents the Euclidean distance computed after scaling the site vectors to length 1 is defined by: This distance between two sites is equivalent to the length of a chord joining two points within a segment of a hypersphere of radius 1.
The Prevosti distance (Prevosti et al., 1975) is a very useful genetic distance between units representing populations.Now, we consider that genetic data is stored in a table where the rows represent the populations and the columns represent potential allelic states grouped by loci.The distance between two units at a single locus k with m(k) allelic states is: where ν is the number of loci or chromosomes (in the case of chromosomal polymorphism) considered and p iks , p jks are the sample relative frequencies of the allele or chromosomal arrangement s in the locus or chromosome k, in the ith and jth population, respectively.With presence/absence data coded by 1 and 0, respectively, the term 1 2ν is omitted.
As we explain in the next section, WeDiBaDis allows the user to introduce alternative distances by means of a distance matrix.Therefore, the user can work with any distance matrix that is considered appropriate for their data set and analysis.For this reason, no more distances were included in our package.

Using the package
We have developed the WeDiBaDis package to implement both the DB-discriminant and the new WDB-discriminant.It can be used with different distance functions and NA values are allowed.When an unit has a NA value in some features, those features are excluded in the computation of the distances for that unit and the computation is scaled up to the number m of features involved in the data set.Package WeDiBaDis requires a version 3.3.1 or a greater of R.
The principal function is WDBdisc with arguments: WDBdisc(data, datatype, classcol, new.ind, distance, type, method) where: • data: a data matrix or a distance matrix.If the Prevosti distance will be used, data must The R Journal Vol.8/2, December 2016 ISSN 2073-4859 be a named matrix where the name of the loci and allele must be separeted by a dot (Loci-Name.AlleleName).
• datatype: if the data is a data matrix, datatype = "m"; if the data is a distance matrix datatype = "d".
• classcol: a number indicating which column in the data contains the class variable.By default the class variable is in the first column.
• new.ind: is only required if there are new unlabeled units to be classified; if datatype = "m" it is a matrix containing the feature values for the new units to be classified; if datatype = "d" it is a matrix containing the distances between the new units to be classified and the units in the classes.
• type: is only required if distance = "Gower".The value for type is a list (e.g., type = list(cuant,nom,bin)) indicating the position of the columns for continuous (cuant), nomi- nal (nom) and binary (bin) features, respectively.
• method the discriminant method to be used.This must be either "DB" or "WDB" for the DBdiscriminant and WDB-discriminant, respectively.The default method is WDB.
The function returns an object with associated plot and summary methods offering: • The classification table obtained with the leave-one-out cross-validation.
• The total well classification rate in percentage (Q t ).
• The sensitivity, specificity, and positive predictive values for each class (Q se k , Q sp k , and P + k , respectively).
• The Kappa and F 1 statistics.
• The assigned class for new unlabeled units to be classified.
• A barplot for the classification table.
• A barplot for the sensitivity, specificity, and positive predictive values for each class.

Example 1: Ecological data
We consider the data from Fielding (2007), which relate to the core area (the region close to the nest) of the golden eagle Aquila chrysaetos in three regions of Western Scotland.The data consist of eight habitat variables: POST (mature planted conifer forest in which the tree canopy has closed); PRE (pre-canopy closure planted conifer forest); BOG (flat waterlogged land); CALL (Calluna (heather) heath land); WET (wet heath, mainly purple moor grass); STEEP (steeply sloping land); LT200 (land below 200 m), and L4-600 (land between 200 and 400 m).The values are the numbers of four-hectare grid cells covered by the habitat, whose values are the amounts of each habitat variable, measured as the number of four hectare blocks within a region defined as a "core area."In order to evaluate if the habitat variables allow to discriminate between these three regions, for example, a WDB-discriminant using the Euclidean distance using the following instructions may be performed: library(WeDiBaDis) out <-WDBdisc(data = datafile, datatype = "m", classcol = 1) The summary method shows, as usual, the more complete information:   -----------------------------------

-No predicted individuals
As we can observe, perfect classification is obtained for samples from region 1.For regions 2 and 3, only two samples were not correctly classified.
If we want to obtain the barplot for the classification table (see Figure 1), we use the command plot(out) Finally to perform a DB discriminant using a different distance that the Euclidean, the following commands are used: library(WeDiBaDis) out <-WDBdisc(data = datafile, datatype = "m", distance = "name of the distance", method = "DB", classcol = 1) summary(out) The R Journal Vol.8/2, December 2016 ISSN 2073-4859 The chromosomal polymorphism for inversions is very useful to characterize the natural populations of Drosophila subobscura.Furthermore, lethal genes located in chromosomal inversions allow the understanding of important evolutionary events.We consider the data from a study of 40 samples of this polymorphism for the O chromosome of this species (Solé et al., 2000;Balanyà et al., 2004;Mestres et al., 2009).Four groups can be considered: NLE with 16 no lethal European samples, LE with 4 lethal European samples, NLA with 14 no lethal American samples and LA with 6 lethal American samples.In this example, two samples one of the group NLA and one of the group NLE were randomly selected, and considered as new unlabeled units to be classified.The Bhattacharyya distances between all pairs of units were calculated.Therefore, the input for the WDBdisc function is an n × (n + 1) matrix dat = (l i , d B (y i , y 1 ), . . ., d B (y i , y n )) i=1,...,n where the first column contains the class label and the following columns the distance matrix.Furthermore, xnew is a two-row matrix where each row contains the distances between the new unlabeled units to be classified and the units in the four classes.In this situation, the commands to call the WDB procedure to classify the xnew units and to obtain the available graphics in the package, are: library(WeDiBaDis) out <-WDBdisc(data = dat, datatype = "d", classcol = 1, new.ind = xnew) plot(out) outplot <-summary(out, show = FALSE) plot(outplot) The summary method shows the following information.We can see that the xnew units were correctly classified: summary(out) Discriminant method: WDB Leave-one-out confusion matrix: The R Journal Vol.------------------------------------Prediction for new individuals: Pred.class 1 "NLE" 2 "NLA" Now, the two unlabeled new units were correctly classified.The barplots are in Figure 3 and Figure 4, respectively.

Data files
The package contains some examples of data files, each with a corresponding explanation.The data sets are corearea, containing the data for the example presented in the subsection Example 1: Ecological data; abundances, which is a simulated data set for abundance data matrix; and microsatt, The R Journal Vol.8/2, December 2016 ISSN 2073-4859

Computing time
To illustrate the time consumed by the WDB procedure, which requires more computation than DB, we performed the following simulation with artificial data.We generated multinormal samples containing 50, 100, 200, 300,. . .,900, 1000, 2000, and 3000 units, respectively.Then, for each sample size we created sets containing respectively 50, 100, 500, 1000, 1500, 2000, 2500, . . ., 4500, and 5000 features.For each combination of sample sizes and features, we considered 2, 3, 4, and 10 classes.All the computations presented in this paper have been performed on a personal computer with Intel(R) Core(TM) i5-2450M and 6 GB of memory using a single 2.50GHz CPU processor.The results of the simulation for two classes are displayed in Figure 5, where the elapsed time (the actual elapsed time since the process started) is reported in seconds.We can observe that the runtime is mainly affected by the number of units (Figure 4, top), but affected very little by the number of variables (Figure 4, bottom).This is expected, as the procedure is based on distances and therefore the dimension of the distance matrix (number of units) determines the runtime required.The number of classes also affects the runtime, although its variation with increasing the number of classes is very slight.For example, with 300 units and 4000 variables, the elapsed time for 2, 3, 4, and 10 classes are 3.38, 3.40, 3.62, and 4.82 seconds, respectively.

DB and WDB comparison using cancer data sets
In order to compare the performance of DB and WDB procedures, thirty-eight available cancer data sets were considered in our analysis (Table 1).  1, three of them include clinical features and some of the data sets have unbalanced classes.We performed the evaluation for DB and WDB classifiers using the leave-one-out procedure.We present the total misclassification rate MQ t = 100 − Q t and the generalized squared correlation coefficient GC 2 (Table 2).For simplicity, the sensitivity Q k i se , the specificity Q we considered the Gower distance (Gower, 1971).As we can observe in Table 2, considering only MQ t , the total misclassification percentage rate, WDB was the best classifier in 18 data sets and it shared this quality in 11 data sets with DB (Wilcoxon signed rank test; one side p-value = 0.0265).Using the generalized squared correlation GC 2 coefficient (Table 2), WDB was the best rule in 16 data sets and it shared this quality in 11 data sets with DB (Wilcoxon signed rank test; one side p-value = 0.0378).Note that for data sets 30 and 38 the GC 2 value is 0. For example, in the Risinger-2003 case, all units of the second class (class with 3 units) were badly classified with DB and WDB methods.However, while with the DB method, 4 units belonging to other classes were badly classified in class 2, with the WDB method none of the units of other classes were badly classified in class 2, and for this reason the GC 2 is equal to 0. With the Yeoh-2002-v2 data set something similar happened.For all these results, WDB seems to obtain in general the best results and to be a slightly better in the case where classes are unbalanced with respect to their sizes.

Conclusions
The package WeDiBaDis, available at https://github.com/ItziarI/WeDiBaDis, is an implementation of two discriminant analysis procedures in an R environment.The classifiers are the Distance- The R Journal Vol.8/2, December 2016 ISSN 2073-4859 These commands generate the sensitivity, specificity and positive predicted values barplot (see Figure 2): outplot <-summary(out, show = FALSE) plot(outplot)

Figure 2 :
Figure 2: Plot of the sensitivity, specificity, and positive predicted value for each class for ecological data in Example 1.

Figure 4 :
Figure 4: Plot of the sensitivity, specificity, and positive predicted value for each class for population genetics data in Example 2.

Figure 5 :
Figure5: Artificial data sets with two classes.Top: Elapsed timing in seconds (y axes) for WDB procedure with respect to the number of units (x axes).Each line (colours in the legend) corresponds to the set with identical number of features.Bottom: Elapsed timing in seconds (y axes) for WDB procedure with respect to the number of features (x axes).Each line (colours in the legend) corresponds to the set with identical number of units.
, C k ) is the proximity function which measures the proximity between y * and C k .This function is defined by, φ2 *

Table Classes
Plot of leave-one-out classification table for ecological data in Example 1.
Plot of leave-one-out classification table for population genetics data in Example 2.

Table 1 :
Cancer data sets (ID = identification number).They present different number of classes (K), number of samples (n), number of samples in each class (n i ), number of features (p), number of continuous features (cuant) and number of qualitative features (quali).The percentage corresponding to the number of samples belonging to each class is in brackets in column five.