Understanding plasmon damping in gold nanorods (AuNRs) is crucial for optimizing their use in photochemical processes and biosensing. This study used dark-field microscopy and spectroscopy to explore plasmon damping in single AuNRs on graphene monolayers (AuNR@GL) with pyridine derivatives as adsorbates. The Au-graphene heterostructure caused a Fermi-level downshift, making graphene a dominant electron acceptor. Hot electrons transferred from AuNR to graphene, leading to a redshift and broadening of the LSPR spectrum. Pyridine adsorption at the AuNR-graphene interface induced a redshift and LSPR line width narrowing due to competing surface damping pathways involving graphene and adsorbate molecules (chemical interface damping, CID). The electron-donating groups of pyridine derivatives on AuNR@1GL caused further LSPR narrowing and decreased plasmon dephasing time. Additionally, thicker graphene layers suppressed electron transfer, highlighting a dominant CID effect. This study, therefore, provides detailed insights into controlling competing plasmon decay mechanisms at the adsorbate-AuNR-graphene interface at the single-particle level.