While the pandemic has not led to a jump in unemployment, its impact on the over-50s and public sector staff is particularly troubling