Recent studies have suggested distinctive biological properties and signaling mechanisms between human and mouse embryonic stem cells (hESCs and mESCs). Herein we report that Shp2, a protein tyrosine phosphatase with two SH2 domains, has a conserved role in orchestration of intracellular signaling cascades resulting in initiation of differentiation in both hESCs and mESCs. Homozygous deletion of Shp2 in mESCs inhibited differentiation into all three germ layers, and siRNA-mediated knockdown of Shp2 expression in hESCs led to a similar phenotype of impaired differentiation. A small molecule inhibitor of Shp2 enzyme suppressed both hESC and mESC differentiation capacity. Shp2 modulates Erk, Stat3 and Smad pathways in ES cells and, in particular, Shp2 regulates BMP4-Smad pathway bi-directionally in mESCs and hESCs. These results reveal a common signaling mechanism shared by human and mouse ESCs via Shp2 modulation of overlapping and divergent pathways.