Current cancer diagnosis employs various nuclear morphometric measures. While these have allowed accurate late-stage prognosis, early diagnosis is still a major challenge. Recent evidence highlights the importance of alterations in mechanical properties of single cells and their nuclei as critical drivers for the onset of cancer. We here present a method to detect subtle changes in nuclear morphometrics at single-cell resolution by combining fluorescence imaging and deep learning. This assay includes a convolutional neural net pipeline and allows us to discriminate between normal and human breast cancer cell lines (fibrocystic and metastatic states) as well as normal and cancer cells in tissue slices with high accuracy. Further, we establish the sensitivity of our pipeline by detecting subtle alterations in normal cells when subjected to small mechano-chemical perturbations that mimic tumor microenvironments. In addition, our assay provides interpretable features that could aid pathological inspections. This pipeline opens new avenues for early disease diagnostics and drug discovery.