Residual adhesive and cement (the "glue") can be missed out during the initial debonding (when the braces are removed). This is because the glue is transparent or the same colour as the enamel surface and is practically invisible when wet.
After a few years, the adhesives undergo degradation and absorb stains to change colour and may appear yellowish.
At this stage, it is still possible to remove the adhesive.
This should be done very carefully so as to not damage the enamel surface. Magnification (loupes, operating microscope), very fine diamond burs, non-enamel cutting tungsten carbide burs and a selection of silicone or rubber polishers are essential for removing adhesives safely.
This is a procedure that I commonly do as part of routine scaling and polishing, when required. This should also be done prior to any sort of dental bleaching as the adhesive may affect the penetration of the bleaching chemical into the enamel.