A suitable classifier mainly depends on the location vector and covariance matrix of the parameters. The linear discriminant analysis (LDA), for example, works by construct ing a suitable linear hyperplane based mainly on the location and covariance among model parameters. The data points in the Xn×p are transformed from R p to R such that the resultant data lead to minimum misclassification rates (MCRs). However, when the class data centroids are not significantly different, linear classifiers usually per form poorly. The support vector machine (SVM), on the other hand suffers from high MCRs, especially for the linearly non-separable and non-linearly separable classifica tion problems. To overcome these issues, in this study, we propose a loss function using the MM principle for the LDA to minimize MCR. Also, a loss function for the linearly non separable SVM is proposed to minimize MCR. Moreover, we propose a generalized transformation method for the non-linearly separable case to minimize MCR. Finally, a covariance estimate for one class is derived given that one class is known. The resultant performance of the proposed methods are validated using simulation studies and real life applications. Findings from simulation studies and real-life applications show that our methods yield more competitive results compared to the classical classification methods.