If the paper isn't wet, you could just leave it. Finch nests can get 'nasty' during laying and covered with poop so it could be it's in a normal state.
I don't think I'd use the moss.....are you referring to the gray type moss? Lint, dried clean grass, or shredded burlap is the safest.










