just go buy all their stuff starting from Exile on Main Street and work backwards...
the Stones sucked donkey cock after exile IMO...
but all their stuff is pretty much dollarbin common to $10 for everything from 'England's Newest Hitmakers' on up, provided you are satisfied w/US pressings. You may even luck into some cheap UK press if you hit up stores with good rock selection.
Early Stones is always some great raw shit, all the way up thru Exile. Not every LP was great, but most were good or better, and some are sublime (like 'Aftermath' and '12x 5' and several more.
Beggar's Banquet
Let It Bleed
Sticky Fingers
Exile on Mainstreet
I also dig:
Some Girls
Tatoo You
Some of:
It's Only Rock n' Roll
Goat's Head Soup
and lots of their early stuff. They do one of the best covers of Not Fade Away and It's All Over Now is nice. Many people think their mid 60s work is really interesting and their most original work. Wes Anderson films celebrate this period thoroughly.