The Fenhe River basin is the main agricultural and industrial developed area in Shanxi province, China. In recent years, agricultural non-point source pollution in the Fenhe River basin intensified, threatening soil quality and safety in the area. Accumulation of eight heavy metals (HMs) including chromium (Cr), nickel (Ni), copper (Cu), zinc (Zn), arsenic (As), cadmium (Cd), lead (Pb) and mercury (Hg) has been detected in soil samples from 50 agricultural sites (0-20 cm) from the middle reaches of the Fenhe River basin. The ecological and human health risk and potential sources of the eight HMs were investigated. In addition, the human health and ecological risks imposed by the possible sources of the eight HMs were quantitatively apportioned. The enrichment factor (EF) values of Cr, Ni, Cu, Pb and Zn were lower than 2, indicating minimal enrichment, while values for As, Cd and Hg were between 2 and 5, exhibiting moderate enrichment. Temporal variation analysis suggested that most HMs in the study area exhibited low concentrations after 2015, except As. The potential ecological risk index was 174.09, indicating low ecological risk. The total hazard index and cancer risk values were 0.395 and 5.35 x 10^-4 for adults and 2.75 and 3.63 x 10^-4 for children, indicating the accepted standard levels were exceeded for non-carcinogenic risk for children and carcinogenic risks for both adults and children. Four potential sources were identified: (1) natural sources, (2) farming activities, (3) coal combustion, and (4) exhaust emissions. Natural sources represented the largest contributor to ecological risk, accounting for 57.42% of the total. Coal combustion was the major contributor to human health risks, accounting for 43.27% and 43.73% of the total non-carcinogenic risk and carcinogenic risk for adults, respectively, and 42.72% and 43.88% for children, respectively.