See can't hardly. Couldn't is a contraction of could not. Drop the negative part of the phrase when using it with scarcely or hardly: "He could hardly see the road through the heavy fog."
Dictionary of problem words and expressions. Harry Shaw. 1975.