Interferon-induced transmembrane proteins (IFITM1, 2, and 3) are important antiviral proteins that are active against many viruses, including influenza A virus (IAV), dengue virus (DENV), Ebola virus (EBOV), Zika virus (ZIKV), and severe acute respiratory syndrome coronavirus (SARS-CoV). IFITM proteins exhibit specificity in activity, but their distinct mechanisms of action and regulation are unclear. Since S-palmitoylation and cholesterol homeostasis are crucial for viral infections, we investigated IFITM interactions with cholesterol by photoaffinity cross-linking in mammalian cells along with molecular dynamic simulations and nuclear magnetic resonance analysis in vitro. These studies suggest that cholesterol can directly interact with S-palmitoylated IFITMs in cells and alter the conformation of IFITMs in membrane bilayers. Notably, we discovered that the S-palmitoylation levels regulate differential IFITM protein interactions with cholesterol in mammalian cells and specificity of antiviral activity toward IAV, SARS-CoV-2, and EBOV. Our studies suggest that modulation of IFITM S-palmitoylation levels and cholesterol interaction influence host susceptibility to different viruses.