When trying to decide what to do, compare the cost of refinancing with what it would cost you in additional interest to hold on to your existing loan. While the breakdown is different for every borrower, generally, you’ll need to keep your current house and loan for anywhere from three to six years to break even on the costs of refinancing.