Should we take measures to thwart an impending asteroid collision if we're able to do so? Of course. But if we expend too much energy in anticipation of such a rare and unlikely event, then we're drawing resources away from more homegrown challenges that are far less remote and much more likely to occur in our lifetimes.
Will we again be hit by the sort of asteroid that ended the dinosaurs? Probably, but we'll never see it coming; in fact, we already missed it if we can assume that maybe we've been threatened by menacing space junk in the past and our modern technologies might have already prevented us from a painful case of mass extinction.