Will washing my adams apple damage my voice or anything?

i got in chamber choir and so i do what i can to protect my voice. unfortunately as i play sports naturally dirt and sweat builds up on my throat. so when i take a shower i get the wash clothe and scrub my throat area. i have to put pressure to get the dirt off and at times i accidently shove my adams apple about, not too hard. i feel fine so far but i dont want to hurt my throat, is there an easier way of maintaining healthy hygiene on my neck and not hurt my throat?