I don't think it is intended to be bigotry.

But it's kind of like talking about the infrastructure needed to wage war on another country: tanks, artillery, hospitals, and bombs. We must eliminate the ability of aggressors to fund the infrastructure of war!

Personally, I would try to be careful to not seem to call for defunding hospitals unless I actually meant it.

"That wasn't racist--all my (white) friends understood and thought it was fine" generally doesn't generally meet with a very sympathetic attitude when said by a white person, and it probably shouldn't (details matter, of course). Same deal here.