Jefferson was the second president to live in the White House after John Adams, but construction started in George Washington's term.