Mesangial cells share features with contractile smooth muscle cells and mechanically support the capillary wall. The role of vitamin D compounds and the transforming growth factor-beta (TGF-beta) type II receptor in modulating the smooth muscle phenotype of cultured mesangial cells was examined. Cell proliferation was significantly inhibited by the vitamin D analog 22-oxa-1,25-dihydroxyvitamin D(3) (22-oxacalcitriol; OCT) rather than by 1,25-dihydroxyvitamin D(3) (1, 25(OH)(2)D(3)) in a dose-dependent manner. OCT-treated early passage mesangial cells (MC-E cells) had increased expression levels of type IV collagen and smooth muscle alpha actin mRNA, but 1, 25(OH)(2)D(3)-treated MC-E cells did not. The addition of a TGF-beta(1)-neutralizing antibody to the OCT-treated MC-E cells blocked this inhibitory effect for cell proliferation and attenuated the up-regulated mRNA levels. However, after exposure to 1, 25(OH)(2)D(3) or OCT, there was no significant difference in the secretion of active TGF-beta. We next investigated whether TGF-beta type II receptor (RII) was involved in this regulation. OCT treatment significantly increased the expression of the RII mRNA in MC-E cells. These results suggest that the vitamin D analog OCT induces smooth muscle phenotypic alterations and that this phenomenon was mediated through the induction of RII in cultured mesangial cells.