The binding of transcription factor AP-1 and vitamin D receptor (VDR) to the composite AP-1 plus vitamin-D-responsive promoter region (AP-1 + VDRE) of the human osteocalcin gene was characterized in osteocalcin-producing (MG-63) and non-producing (U2-Os, SaOs-2) human osteosarcoma cell lines. In mobility-shift assays with AP-1 + VDRE, AP-1, and VDRE probes and nuclear extracts from these cells, one AP-1-specific and two VDR-specific (fast and slow mobility) interactions were observed. Characterization of the complexes indicated that AP-1 and VDR do not bind simultaneously to the AP-1 + VDRE oligonucleotide. Intensity of the complexes was greatly influenced by cell density: in MG-63 and SaOs-2 cells, AP-1 binding was strong during the proliferative period disappearing at confluency whereas, in U2-Os cells, AP-1 binding was prominent also at the confluent stage. Furthermore, MG-63 cells possessed the faster migrating VDR complex at all stages of confluency whereas, in U2-Os and SaOs-2 cells, it was very weak or absent. There were no detectable differences in the levels of VDR protein between these cell lines. In U2-Os cells, the level of c-jun mRNA was higher than in the other two cell lines, whereas none of these cell lines exhibited detectable levels of c-fos mRNA at the confluent stage. Exogenous c-Jun protein effectively blocked the VDR-DNA interaction. Further, all these cell lines expressed mRNA for retinoid X receptor alpha (RXR alpha), the factor suggested to be required for the VDR-DNA interaction. The presence of an accessory factor in the VDR-DNA complexes was indirectly shown by treatment of the cells with 9-cis retinoic acid and by cycloheximide. Both treatments reduced VDR binding without affecting the VDR protein level. These results suggest that AP-1 interferes with VDR binding to the AP-1 + VDRE element and that the vitamin D responsiveness of the osteocalcin gene correlates with weak AP-1 binding and strong binding of the faster migrating VDR complex.