Breast Cancer is the most common cancer among women worldwide. In Oman, one out of five Omani women was diagnosed with breast cancer in her lifetime as indicated by the National Cancer Registry. Histopathological images are essential re sources for clinical observation to diagnose the presence or the extent of abnormal con ditions. These images are usually large and complex which make the diagnosis process tedious, time-consuming, and prone to errors. Hence, there is a need for automat ing the histopathological image analysis process. Early machine learning methods for histopathological image processing rely on hand-crafted features extracted from these images with the help of domain knowledge experts. The recent Deep Learning (DL) models (such as Convolutional Neural Network (CNN)) are capable of extracting au tomatically, from the raw data, features of different levels of abstraction and use them to accomplish successfully complex computer vision tasks. Traditional CNNs like AlexNet,etc use the back-propagation algorithm for learning the filters during the train ing phase. Such an algorithm requires a huge amount of labeled datasets which leads to high computational processing. Moreover, it may encounter the vanishing gradient problem that deteriorates the quality of learning. Besides, in many domains, acquiring enough labeled images for conducting properly the training phase is a real challenge. The forward supervised filter learning approach was proposed to tackle these problems. On the other hand, Non-Negative Matrix Factorization (NMF) decomposes input data into two latent factors. It has been shown that by enforcing constraints such as sparsity on the latent factors, dominant features that are mostly correlated with tumors types can be extracted. This research focuses on the development of a novel model that learns useful representations from histopathological images for the purpose of their classi fication. We have derived a mathematical model of a novel forward supervised filter learning approach that combines sparse NMF and Support Vector Machine technique. The model is used to design and implement a feed-forward CNN classifier (CSNMF SVM) to classify histopathology images. Moreover, we modified the architecture of an existing feed-forward CNN (CSVM), to classify histopathological images. We named the model, Convolutional Support Vector Machine for Histopathology (CSVM-H). Fi nally, we have modified the loss function used by CSVM-H (replaced the hinge loss by the binary cross entropy) to obtain a classification model, we named Convolutional Logistic Regression for Histopathology (CLR-H). All these models have been eval uated on the histopathology images from Sultan Qaboos University Hospital dataset and the publically available BreaKHis dataset. The experiments we have conducted demonstrate the efficiency of the proposed models, especially on small-sized datasets.