New England MP Barnaby Joyce fears federal governments pandemic leave may be "a mighty task" to repay.