"All" is relative. Most is probably a better term to use. There can be residue left over, but there can't be large channels and gulleys, as the tiles will not be able to sit evenly. The rule of thumb is, as long as your new tiles are going to be HIGHER than the remaining mortar, you'll be fine. But remember, the cleaner the surface, the easier the install, and the less thin-set you'll use.