Purpose The purpose of this paper is to examine the role of sheepskin effects in the return to education in Japan. Design/methodology/approach The paper provides a short description of the Japanese schooling and recruitment system. It then describes the data set and the empirical approach. Estimation results are presented for the various specifications. The baseline specification closely follows existing studies for the USA to facilitate comparability across the two countries. The paper further investigates whether there are significant firm‐size differences in the estimated sheepskin effects and whether sheepskin effects disappear with increasing job tenure. Findings The estimation results indicate that sheepskin effects explain about 50 percent of the total returns to schooling. The paper further finds that education as a signal is only important for workers in small firms with the size of these effects being similar to comparable estimates for the USA. Finally, the estimated degree effects decrease with firm tenure, in particular for small firms. These results could be explained by the particular recruitment system of large firms in Japan, which makes university diploma as a screening device unimportant for large firms and the admission policy of Japanese universities. Originality/value By investigating the role of sheepskin effects in a labor market that differs substantially from the labor market in the USA, the paper provides additional insights to the human capital theory‐screening hypothesis debate.