The nation’s largest mortgage insurer has rejected $334 million in claims since early 2008, contending the loan documents contained false or flimsy information.