Transcriptional initiation is a key step in the control of mRNA synthesis and is intimately related to chromatin structure and histone modification. Here, we show that the ubiquitylation of H2A (ubH2A) correlates with silent chromatin and regulates transcriptional initiation. The levels of ubH2A vary during hepatocyte regeneration, and based on microarray expression data from regenerating liver, we identified USP21, a ubiquitin-specific protease that catalyzes the hydrolysis of ubH2A. When chromatin is assembled in vitro, ubH2A, but not H2A, specifically represses the di- and trimethylation of H3K4. USP21 relieves this ubH2A-specific repression. In addition, in vitro transcription analysis revealed that ubH2A represses transcriptional initiation, but not transcriptional elongation, by inhibiting H3K4 methylation. Notably, ubH2A-mediated repression was not observed when H3 Lys 4 was changed to arginine. Furthermore, overexpression of USP21 in the liver up-regulates a gene that is normally down-regulated during hepatocyte regeneration. Our studies revealed a novel mode of trans-histone cross-talk, in which H2A ubiquitylation controls the di- and trimethylation of H3K4, resulting in regulation of transcriptional initiation.