He had really only one flaw and that was an inability to calculate the correct age of the Earth. The question occupied much of the second half of his career, but he never came anywhere near getting it right. His first effort, in 1862 for an article in a popular magazine called Macmillan's, suggested that the Earth was 98 million years old, but cautiously allowed that the figure could be as low as 20 million years or as high as 400 million. With remarkable prudence he acknowledged that his calculations could be wrong if "sources now unknown to us are prepared in the great storehouse of creation"—but it was clear that he thought that unlikely.
With the passage of time Kelvin would become more forthright in his assertions and less correct. He continually revised his estimates downward, from a maximum of 400 million years, to 100 million years, to 50 million years, and finally, in 1897, to a mere 24 million years.
Kelvin wasn't being willful. It was simply that there was nothing in physics that could explain how a body the size of the Sun could burn continuously for more than a few tens of millions of years at most without exhausting its fuel. Therefore it followed that the Sun and its planets were relatively, but inescapably, youthful.
The problem was that nearly all the fossil evidence contradicted this, and suddenly in the nineteenth century there was a lot of fossil evidence.