As baby boomers downsize and millennials finally enter the housing market, they are in direct competition for similarly priced and sized homes.