What I Did:

We had no groomsmen, only the best man. We asked him to wear a costume, but didn't specify any particular style. He and his wife came to the party in matching jumpsuits with "Area 51 Maintenance Crew" patches. They were catering the party, so they spent the first half-hour setting up the buffet. Then they vanished upstairs...

(Click the thumbnails below for larger views.)

...and returned wearing alien masks and gloves that looked like alien hands. Hilarious, and quite unexpected. (He removed the mask and gloves for the actual ceremony.)

Other Suggestions:

• If the groom is renting a tail coat/morning coat ensemble, the groomsmen can rent the same style and wear a less-fancy waistcoat or accessories.

• Undertakers traditionally wore swallowtail coats, so put the groomsmen and ushers in tails. If your wedding has a Victorian theme, consider going with the full top-hat-and-coat ensemble of a Victorian undertaker.

• Get a little wacky and put each of your groomsmen in a different "classic monster" costume: Dracula, the Wolfman, the Mummy.... Go all out and dress your Best Man and Maid of Honor as Frankenstein and his Bride.

• For a more restrained look, put your groomsmen in standard tuxes, but have each one's waistcoat, bow tie, or cummerbund be a different autumn color.