In musical structure, he was inspired by Beethoven's repeating thematic material in separate movements and the unity this gave the work as a whole. Hector Berlioz's work Symphony Fantastique was also an influence. Liszt liked the work so much that he helped to promote the work by making a piano transcription of it.
The tone poem is a direct descendant of the operatic and concert overture. As operatic overtures signaled a beginning of an opera and often quoted the main themes that were to be heard, the concert overture was usually written in sonata form and could be a singular piece of music that began a dramatic play, or the music could have a descriptive setting. Liszt took these qualities, plus the feeling of the first movement in a symphony (usually written in sonata form and many times was the movement that held the most musical weight). He took the idea of thematic unity further than Beethoven, and used what is called cyclic form, where the entire piece has common themes repeated, sometimes verbatim, sometimes varied. Like many other 'new' things, Liszt did not invent cyclic form so much as revive it for there is examples of it in Renaissance music. There are also examples of it in Haydn's music, and of course Beethoven made use of it.
“I saw in my mind’s eye an Etruscan vase in the Louvre, representing the first poet-musician. I thought to see round about him wild beasts listening in ravishment: man’s brutal instincts quelled to silence.... Humanity today, as formerly and always, preserves in its breast instincts of ferocity, brutality and sensuality, which it is the mission of art to soften, sweeten and ennoble.”
Orpheus is written in a loose sonata form. The piece begins with two harps playing ascending passages in imitation of Orpheus' lyre.The piece is restrained and contemplative, Liszt's tribute to the depth of feeling and redeeming qualities he heard and saw in music and art in general.
Liszt wrote most of his tone poems while he was kappelmeister in Wiemar in 1852-1854. Most of them are not in the immediate concert repertoire except for Les Preludes. They were experiments in form, structure, orchestration and material. Some are more satisfying musically than others, but all of them are at the least interesting. That more of them are not heard in the concert hall may say something about the tone poems, but it surely says something about the state of the modern day concert hall.