It's not hard to remember if you remember the rules. An apostrophe replaces a missing letter/number or shows possession.
The temperature was in the 80s today. (Plural just like eighties. You don't say eightie's, do you? )
The music is from the '70s. (Missing numbers as in 19-apostrophe-70s)
Same with CDs, no apostrophe.
DVDs, no apostrophe.
I got all A's and B's in school. Yes, because it's a SINGLE character and needs to separated to avoid confusion.
45s is plural, not possessive.
You can blame my teachers for pounding it into my head so much. They made sure I'd never forget it.
You know how people that misuse "they're, their, there" drives you nuts? Same with apostrophes for me.