Because the pit is so much bigger, the XL uses way more lump than the Large does for otherwise similar cooks. I agree with others that the Large is fine for pretty much anything you might cook for about 4 people. I'd say you'd need to be regularly cooking for more than 6 to really needs the space of the XL.
As for accessories, I highly recommend
Grill Grates for all grills. The platesetter is pretty much required for smoking or any other direct cooking. As for other gear, I highly recommend the thermapen, a good remote thermometer (like the Maverick), and once you've got the hang of smoking it the old fashioned way, the BBQ Guru.
Got the Large myself a few years ago. Finally sold my Weber kettle recently. The kettle was great, but it barely used it once I got the BGE. I love that thing.