A great deal of research in the statistical literature has given rise to the existence of bias in inferential procedures based on the maximum likelihood estimators. Moreover, this is a matter of serious concern when the data contains some missing values. The magnitude of such bias can significantly affect the study conclusions; if large, misleading conclusions will be drawn, the sample size will be reduced, as well as the statistical power. In the present thesis, a review of some methods for handling missing data and bias reduction with some application has been done. While we attempted to reduce the impact of bias in estimated parameters when the data is incomplete, we proposed a penalized-based method to reduce bias in the estimated model parameters under data missing at random mech anism. We applied the expectation-maximization principle with weights, and penalized the log-likelihood function with LogF(1,1) for the logit model. Using the Louis(1982) method, we derived the information matrix, while a closed form for the exact bias of the estimated parameter was derived following the Cox and Snell (1968) equation. Further, to facilitate usage of the proposed method, a new BRLFP R Package has been developed. A combination of simulation studies and real-life data problems were used to validate the proposed bias reduction method. Findings from these validation studies consistently show significant reductions in the standard errors of the estimates when compared with other classical bias reduction methods for the missing at random data mechanism. Penalizing the log-likelihood function with the LogF(1,1) was found to generate more efficient parameter estimates for the logistic model under the missing at random mechanism.