There is a darn good reason why people avoid heavy lifting, whether they are cardio fitness nuts or just people looking for work. Heavy lifting is stressful, unpleasant - other than facing the challenge.
Many people avoid stress, some to the point of avoiding life itself. The thing is, painful stress, whether emotional, physical, or intellectual, is what builds strength and effectiveness for life.
If you lift enough, at some point that 60 lb. bag of cement mix will feel light as a feather. Same goes for all other stressors.