Echoing Raffey, Sally Hemings was essentially Jefferson's sister-in-law, as she was his wife's half sister. She was also 1/4 black, or quadroon. Her children by Jefferson were 1/8 black. He did not "free" two of his children, because their being "freed" would indicate that they were once enslaved. Jefferson, instead, allowed those two to disappear North and blend in with the white population. Beverly's and Harriet's children would have been 1/16 black, aka, "who the hell can tell?"
One of the reasons people so dismiss the Hemings story is because they are looking for black people. There are some white people out there today who are descendants of Jefferson and Hemings, and they would never guess it.