All three of these terms are in acceptable use, but first is more common than firstly. A speaker or writer will often start with first and then move on to secondly and thirdly. Although these -ly words are in respectable use, it is simpler and more economical to employ shorter forms (first, second, third, fourth), especially since these short forms can be used adverbially or adjectivally: "Several points need to be stressed: first (or firstly) there is the matter of money." "Who came in first?" "Let us first (or firstly) consider the refugees." Numbers greater than four usually come after first (the first twenty applicants); numbers smaller than four may precede or follow first (the first three applicants, the three first applicants). Recommendation: drop the -ly forms and always use first, second, third, etc., or one, two, three, etc. No need or excuse exists for such phrases as "first of all" and "second of all." Use first or firstly, second or secondly, and drop the useless of all. See also of all.
Dictionary of problem words and expressions. Harry Shaw. 1975.