ThisThat innermost looping groove is called a locked groove.
While it is difficult to track the first record to employ this technique, one of the earliest and most well known examples is The Beatles' Sgt. Pepper's Lonely Hearts Club Band from 1967. It consisted of multi-layered collage of randomised chatter that would repeat endlessly:
Two variations [of Sgt. Pepper's Lonely Hearts Club Band] were made: the original British pressing (black label with gold logo) has the "inner groove" play through the entire locked groove and does not include the laughter at the beginning of the piece. The re-issue of the British pressing (black label with silver logo) starts playing the "inner groove" long before the needle reaches the locked groove, includes the laughter and, once the needle hits the locked groove, the listener only hears the last two seconds of the piece played over and over again.
Many consider this the first example of using such technique for a major release.
There is an article on Wikipedia that gives other examples of such releases and explains what this technique was used for.