This study aimed at building a criterion-referenced scoring rubric for measuring fourth-grade students' Arabic language reading loud skill and examining its characteristics using generalizability theory. The study also aimed at determining the reliability of the holistic scoring rubric used to measure this skill in the Sultanate of Oman’s schools and exploring the sources of variance affecting the reliability of students’ scores. The sample of the study comprised of (205) first teachers and field one teachers of Arabic language subject, as well as (200) fourth-grade students selected through a cluster random sampling method. The performance of the students was assessed by eight field one teachers using both holistic and analytic scoring rubrics. This assessment took place in two basic education schools located in the Al Dhahirah Governorate. The study employed a design based on a partially nested structure (P:R:S) x T. The data were analyzed using the EduG (version 6.1) program. Results from generalizability studies revealed that the 3-way interaction among the task, the student, and the rater was the most influential source of variation in the reliability of students’ scores when applying both the holistic and analytic scoring rubrics. Decision-making studies demonstrated that the generalizability and dependability coefficients for the holistic scoring rubric were (0.93), whereas for the analytic scoring rubric were (0.99). Furthermore, the study found that gender did not have a significant effect on variance components, generalizability and dependability coefficients. The study recommended reconsidering the holistic scoring rubric used to evaluate the reading loud skill of fourth-grade students, so that it is developed to meet the standards that should be present in the scoring rubrics, and the necessity of employing the analytical scoring rubric in the basic education stage due to its distinction and ability to measure the level of students’ performance in the reading loud skill. Keywords: Scoring rubric, Loud reading, Arabic language, Generalizability theory.