I imagine this question is impossible to answer in full, but there are few examples I can think of off the top of my head or I was able to find fairly easily via Google.
Guitar
"Give It Away" by the Red Hot Chili Peppers features two different sections with a backwards guitar.
"Are You Experienced?" by Jimi Hendrix actually has an entirely backwards solo, rather than just backward effects like "Castles Made Of Sand".
"Cosmic Dancer" by T. Rex closes with a minute-long backwards guitar solo.
I've managed to find a thread on Steve Huffman Music Forums that discusses other songs with backwards guitar.
Bass
Though the bass solo on "You Can Call Me Al" by Paul Simon is brief, the second half of it is actually a backwards recording of the first half.
Vocals
A few songs I can think of with reversed vocals that are solo length, rather than just a line or two, are "Shimmy Shimmy Ya" by Ol' Dirty Bastard and "Gloryhallastoopid (Pin The Tale On The Funky)" by Parliament. The former features a verse by ODB in reverse, while the latter features George Clinton repeating different versions of the phrase "I'm Wellington Wigout".
The Beatles' "Rain" has a few lines of backwards vocals at the end of it.
The song "Empty Spaces" by Pink Floyd also features reversed voices, but they aren't sung. It's actually a phone message that contains the following message:
– Hello, Luka [hunters]... Congratulations. You have just discovered the secret message. Please send your answer to Old Pink, care of the Funny Farm, Chalfont...
– Roger! Carolyne's on the phone!
– Okay