For many people, their home is their biggest investment, so it only makes sense to think of it as part of what you have 'set aside.' But while your home is definitely part of your overall net worth, how it factors into your retirement savings depends on more than your current equity. It depends on whether or not you're going to turn that equity into cash when you retire.
The death of a spouse is probably one of the most stressful times in life at any age. Being young, and now solely responsible for your kids, makes it all the more so. There's so much to think about and so many details to handle, that I totally understand your delaying dealing with Social Security. But in spite of the complications, it's worth the effort.