indigenous tribes do invite outsiders in to festivals and events to share their culture. doesn't grant everyone immunity from scrutiny while they parade around in sacred garb they may only see as costume, or recreational.
whitey doesn't like it when things they hold sacred are used in a way they find disrespectful. like flag burning. "Do you know how many men and women have died for those colors" or something to that effect. and how many were massacred for the right to just be and continue on living here in their own land? wearing their traditional and sacred garments? if native peoples take issue with outsiders doing anything, I am willing to hear them out before anyone else. especially those being dismissive because they feel they can get away with it.
I'm not sure what kind of sitcomesque scenario you're envisaging where setting fire to a cultural artifact (or smashing it, or otherwise destroying it) is accidentally mistaken by the person doing it as anything other than an attempt at cultural erasure or defacement, but you're right that most cultures will happily invite outsiders in to partake and share of that culture, because its part of who they are and its a way keeping those traditions alive.
People who inherently believe that 'outsiders' are
inherently 'disrespecting' a culture by partaking of it in an act of
appreciation are the same people who think its outrageous that blacks are allowed on 'their' golf course, or jews are allowed in 'their' country club, or gays are allowed in 'their' military, or whites are allowed to make 'their' cuisines.
The 'disrespect' comes from their own bigotry - that
outsiders are worth less in their eyes, and it 'devalues' their culture to see it shared outside 'the tribe'.
Those people can fuck right off, tbh.