The latest generation of global climate models robustly projects that the summer monsoon rainfall in India will significantly increase in the 21st century due to global warming and that rainfall anomalies will occur more often. This raises the question of the impact of these changes on the agricultural yield. Based on annual district data for the years 1966-2014, we estimate the relationship between weather indices (amount of seasonal rainfall, number of wet days, average temperature) and the most widely grown kharif crops, including rice, in a flexible non-parametric way. We use this relationship in order to predict district-specific crop yield based on the climate projections of eight different climate models of the Coupled Model Intercomparison Project - phase 6 (CMIP6) under two global warming scenarios (Shared Socioeconomic Pathways SSP1-2.6 & SSP5-8.5) for the years 2021-2100 (short-term, mid-term, long-term). We find that the loss in rice yield by the end of the 21st century lies on average between 3 - 22% depending on the underlying emission scenario. Potential gains due to increasing rainfall are more than offset by the negative impacts of increasing temperature. Adaptation efforts in the worst case scenario (SSP5-8.5) would need to cut the negative impacts of temperature by 50% in order to reach the outcome of the sustainable scenario (SSP1-2.6).