The objective of any supply chain management is to maximize the overall value generated. The forecast plays a vital role in achieving this objective because it impacts decision making throughout the supply chain. The intermittent and the non-stationary demands patterns are the most common and challenging behaviour observed in the supply chain, particularly spare parts. Given the importance of the forecasts and the associated challenging demand behaviours, several forecasting techniques have been developed in the industries. One class of these forecasting techniques is the statistical forecasting methods, which depend on the historical demands to predict the future. The most widely used statistical forecasting methods for intermittent and non-stationary demand are Croston's methods and ARIMA. However, these methods have shown to have limitations, especially when dealing with highly erratic data. In such a case, the data needs to smoothen first before these forecasting techniques can be applied. On the other hand, there is an evolution of advanced statistical techniques such as ANN in developing forecasting methods. The ANN-based forecasting methods have shown the potential to outperform the classical statistical methods. The supervised ANNs are well-known for abstracting the underlying unknown relationships without prior assumptions about problems under the study. However, the ANN models can only perform well if their hyperparameters were selected correctly. The three most widely applied supervised ANN are the feedforward, LSTM, and CNN. Several studies demonstrated the effectiveness of the three networks in forecasting when their hyperparameters are selected correctly. This study empirically identifies the hyperparameters that these three networks need to forecast the most common and challenging spare part demands. The study includes developing an ANN-based forecast, which then is applied in a case study to forecast real-life spare parts of a domestic oil producing company to advise the company on the potential of the ANN and provide practical recommendations for efficient ANN-based forecast.