The first two words of this group apply only to persons, whereas older and oldest may apply to persons or things. Also, elder and eldest (much less common than the other two terms) apply principally to members of a given family or business establishment and indicate age or seniority (elder brother, eldest partner). However, say "He is older (not elder) than his brother."
Dictionary of problem words and expressions. Harry Shaw. 1975.