The slacks-based measure (SBM) with the environmental undesirable outputs was used to estimate the environmental efficiency of the production of wheat, maize and rice in the 31 provinces of China in 1998~2013, and then the dynamic panel GMM method was used to investigate the effect of fertilizer pollution on the environmental efficiency of the grain crop production. The results showed that the environmental efficiency in the grain crop production had increased but fluctuated, and had an upward trend since 2003. However, the level of the environmental efficiency was low. The environmental efficiency in the main grain production areas was higher than in the main grain consumption areas and in the grain production consumption balance areas. The fertilizer pollution had significantly negative effect on the environmental efficiency. An increase of the fertilizer pollution by 1% led to a decrease of the environmental efficiency by 10.38% for wheat, 13.45% for maize and 13.19% for rice. This proved that there existed a significant punishment effect of the fertilizer pollution in the grain production.