We present a machine learning method to directly estimate viscoelastic moduli from displacement time-series profiles generated by viscoelastic response (VisR) ultrasound excitations. VisR uses two colocalized acoustic radiation force (ARF) pushes to approximate tissue viscoelastic creep response and tracks displacements on-axis to measure the material relaxation. A fully connected neural network is trained to learn a nonlinear mapping from VisR displacements, the push focal depth, and the measurement axial depth to the material elastic and viscous moduli. In this work, we assess the validity of quantitative VisR (QVisR) in simulated materials, propose a method of domain adaption to phantom VisR displacements, and show in vivo estimates from a clinically acquired dataset.