Agricultural land abandonment has been increasing worldwide for environmental and ocio-economic reasons, and knowledge of its key ecological processes (e.g., carbon (C) and nitrogen (N) input and accumulation) in relation to vegetation succession can provide important information for ecosystem management and greenhouse gas emissions mitigation. In order to better understand the above- and belowground litter dynamics following agricultural abandonment, we simultaneously studied the litter and fine root production, quality, decomposition, C and N input in ecosystems along a secondary successional gradient (i.e., grassland, shrub-grass land, young secondary forest, and mature secondary forest) following agricultural abandonment in China’s Qinling Mountains. Results showed that the significant increase of aboveground woody plant litter and decrease of grass litter during vegetation succession led little changes in total litter production and annual total C and N input in different succession stages, while the fine root production, fine root biomass, C input from fine root production increased significantly with stand age. The initial litter C concentration and fine root carbon: phosphorous ratio (C:P) were the main factors in explaining the variations of decomposition rates of litter and fine root, respectively. The increasing C concentration in litter and the increasing C:P ratio in fine root during vegetation succession had potentially driven the decreases in litter and fine root decomposition rate respectively. The accumulation of litter standing crop during vegetation succession could be attributed to the decreases in litter decomposition rate partly caused by changes in litter quality, rather than the increases in litter production. Our results imply that the changes in vegetation type have a much smaller role in the annual total litter production and the total litter C and N input than previously assumed, while the changes in quality and decomposition rate may have largely influenced C accumulation in stand floor and soil during secondary succession following agricultural abandonment.