Based on the spectral information of remote sensing images during four periods, the remote sensing ecological index (RSEI) and fractional vegetation cover (FVC) were calculated for the poverty-stricken region of the Lüliang Mountains (PSLM), the temporal and spatial variation of regional ecological environment was subsequently evaluated. The results showed that:(1) The FVC in the research area decreased from 0.65 in 1993 to 0.55 in 2018, and the RSEI decreased from 0.47 in 1993 to 0.40 in 2018. (2) The overall ecological environment of the PSLM showed a trend of degradation during the research period, with the most serious degradation happened in the north and south part of the PSLM. Except for the region of Yellow River valley, all the other river valleys experienced the significant degradation in FVC and ecological environment. (3) The RSEI value in the PSLM region was generally lower and showed substantial spatial variations. The ecological environment in the central part of the study area was the most vulnerable and the part of northern Xinzhou and southern Linfen area was deteriorated particularly significantly. (4) The variation pattern of FVC was generally consistent with that of RSEI. The improvement of FVC has a positive influence on the local ecological environment, while the rapid economic development (increasing in roads, construction land and farmland) has negative impacts on the local ecological environment. In the process of regional poverty alleviation and development, more attention should be paid to balance the needs of regional economic development and ecological environment protection in order to avoid further damage to the fragile regional ecological environment.