PTPMT1 (PTP localized to the Mitochondrion 1) is a member of the protein tyrosine phosphatase superfamily that is localized exclusively to the mitochondrion. We recently reported that PTPMT1 dephosphorylates phosphatidylglycerol phosphate, an essential intermediate of cardiolipin biosynthesis. To gain further insights into the molecular basis of PTPMT1 function, we determined the crystal structures of the phosphatase domain of PTPMT1. PTPMT1 exhibits a canonical protein tyrosine phosphatase domain fold, resembling many dual-specificity phosphatases such as phosphatase and tensin homolog and vaccinia H1-related phosphatase. We also determined the structure of the catalytically inactive phosphatase in complex with a surrogate substrate, phosphatidylinositol 5-phosphate, which sheds light on the substrate recognition and specificity of PTPMT1. Comparison of the apo and substrate-bound structures of PTPMT1 suggests that it undergoes significant conformational change during catalysis, and we further demonstrated that an evolutionarily conserved EEYE loop is important for its activity.