The rejection rate for Americans looking for a loan hits a 5-year high, Fed data show. Banks are turning cautious as rates rise and the economy slows.