The poem Mike Milligan (Bokeem Woodbine) recites is called "Jabberwocky" and is taken from the novel "Through the Looking-Glass and What Alice Found There" written by Lewis Carroll in 1872.
The song playing during the credits is a different version of "I Am a Man of Constant Sorrow", a song made popular in the Coen Brothers (Joel Coen & Ethan Coen) film O Brother, Where Art Thou? (2000).
Ed Blumquist (Jesse Plemons) tells Lou Solverson (Patrick Wilson) the story of Sisyphus, a Greek mythological figure condemned to push a boulder up a hill for eternity. This is not the first time the story of Sisyphus is referenced. Previously, in The Myth of Sisyphus (2015), Noreen, (Em Haine) can be seen reading the book of the same name by Albert Camus. Camus uses the story of Sisyphus in his book to explain the absurdity of life.
Karl's (Nick Offerman) attempt to barricade the door with a bench could be a reference to The Dude doing the same thing in The Big Lebowski (1998). In both cases the barricade is on the wrong side of the door.
The episode's title, Rhinoceros, likely comes from the play of the same name by Eugène Ionesco. In the play, the inhabitants of a small, French town turn into rhinoceroses; ultimately the only human who does not succumb to this mass metamorphosis is the central character, Bérenger, a flustered everyman figure who is initially criticized for his drinking, tardiness, and slovenly lifestyle and then, later, for his increasing paranoia and obsession with the rhinoceroses. The play is often read as a response and criticism to the sudden upsurge of Communism, Fascism, and Nazism during the events preceding World War II, and explores the themes of conformity, culture, mass movements, mob mentality, philosophy, and morality.