Sixty samples of different marble and granite tiles used as building materials, from local and foreign origins, were collected from different factories in the Al-Madinah Al-Munawarah region of Saudi Arabia. Natural radioactivity of the U–Th series and 40K in the samples were measured using an HPGE detector to conduct gamma-ray spectrometry. From the measured γ-ray spectra, the activity concentrations for different marble samples of Turkish, Egyptian, and Omani origin were 232Th (0.10 ± 0.07, 0.9 ± 0.09, 0.31 ± 0.06 Bq/kg), 226Ra (5.7 ± 0.3, 7.3 ± 0.40, 2.0 ± 0.1 Bq/kg), and 40K (0.9 ± 0.2, 0.96 ± 0.1, 0.8 ± 0.09 Bq/kg), respectively, whereas for Saudi, Chinese, and Indian granite, the concentrations were 232Th (48.6 ± 2.5, 22.9 ± 1.20, 80 ± 4 Bq/kg), 226Ra (56.4 ± 3.0, 32 ± 1.7, 10.1 ± 1.1 Bq/kg), and 40K (1492.8 ± 79.7, 762.5 ± 40.7, 1906.0 ± 43.6 Bq/kg), respectively. The calculated values of the radium equivalent activity Raeq for the studied marble samples ranged from 2.504 to 8.66 Bq/kg, and for granite, the values ranged from 127.8 to 244.19 Bq/kg. Moreover, a comparison study of the measured activity concentrations was presented. The values of external Hex and internal Hin hazard indices for the studied marble and granite samples are less than unity. The values of gamma-radiation hazard indices Iγr for the marble and granite samples are below 0.058 and 1.905, respectively. The values of the annual effective dose for the studied marble samples ranged from 0.011 mSv/y to 0.038 mSv/y, and that of granite ranged from 0.577 mSv/y to 1.169 mSv/y. Therefore, the use of these marble and granite samples under investigation in the construction of dwellings is considered to be safe for inhabitants, and the order of preference for marble is first Omani marble, then Turkish, and finally Egyptian marble. For granite, the preference is first Chinese granite, then Saudi granite, and finally Indian granite because the annual effective doses are comparatively lower.