Deubiquitinating enzymes (DUBs) regulate ubiquitin signaling by trimming ubiquitin chains or removing ubiquitin from modified substrates. Similar activities exist for ubiquitin-related modifiers, although the enzymes involved are usually not related. Here, we report human ZUFSP (also known as ZUP1 and C6orf113) and fission yeast Mug105 as founding members of a DUB family different from the six known DUB classes. The crystal structure of human ZUFSP in covalent complex with propargylated ubiquitin shows that the DUB family shares a fold with UFM1- and Atg8-specific proteases, but uses a different active site more similar to canonical DUB enzymes. ZUFSP family members differ widely in linkage specificity through differential use of modular ubiquitin-binding domains (UBDs). While the minimalistic Mug105 prefers K48 chains, ZUFSP uses multiple UBDs for its K63-specific endo-DUB activity. K63 specificity, localization, and protein interaction network suggest a role for ZUFSP in DNA damage response.