"It was the most ridiculous name we could think of and it stuck," says frontman and multi-instrumentalist Stu Mackenzie.
Then there is the title of their new album, set to be released next month - Flying Microtonal Banana.
It is named after the custom- made guitars that the members specially created to come up with the unique, singular sounds of their new songs.
Mackenzie, 26, says: "I had a friend build a guitar for me and when we were working on some rough sketches, it was going to be yellow, we got it banana-coloured."
The shape of the guitar was similar to a classic guitar design called The Flying V, he adds in the telephone interview from Melbourne.
"And then later on, we ended up adding some microtonal frets, some extra frets in between the frets, so it became the Flying Microtonal Banana."
He and the other six band members then modified more guitars to suit the new instrument and played them with microtonal harmonicas and keyboards for the new songs. The result is an expanded palette of unusual sounds.
"This is the first time we've done something like that," he says. "Every record we made was an experiment."
The members were playing in various Melbourne-based bands when they formed King Gizzard & The Lizard Wizard.
The band's far-out, experimental style was already very much evident in their early days.
The title track of their 2012 debut album, 12 Bar Bruise, for example, was recorded with four iPhones placed around a room, instead of with regular microphones in a proper studio.
Last year's release, Nonagon Infinity, featured songs that seamlessly blend into one another and the whole album can be listened to as an unending loop.
That album is their most commercially successful one to date, reaching No. 19 on the Australian charts.
The band's creativity is bolstered by the fact that they have more musicians than the average rock band. While Mackenzie is the principal songwriter, all the members contribute to the creative process.
"It can be tricky to juggle all that," he admits. "But it all works out in the end."
The band are more than just musically adventurous, they are also known to be highly prolific. The upcoming album is their eighth in four years.
They intend to raise the bar this year - they recently set up their own studio and Mackenzie says they are working on five different albums.
"For me, recording has always been my favourite part of the whole thing. The band started as a recording project and we've always been centred on recorded music."
Nick Murphy as himself