I could have swallowed the alien involvement quite happily. I don't think it has anything to do with WHY the movie is bad. The Roswell Incident (which is referred to in the movie) happened in 1947, and thereafter the world became more obsessed with the idea of visitors... HG Wells' War of the Worlds, The Day the Earth Stood Still, Invasion of the Body Snatchers -- they all had releases in the 50s. I didn't mind the refridgerator thing. I gave it a free pass because the shot of him standing with the mushroom cloud looming looked so awesome. Again - the atom bomb was big news back then. The H-Bomb tested in 1952 had 1000 times the power of the A-bomb. Obviously, the cold war was big news too. The McCarthy style accusations flying around (alluded to pretty badly in the Indy movie) were standard fare too.
So all of that is fine in my book. Had they made a good movie, they really could have seperated this from the 'religious / divine' Indy trilogy, and made something that would stand alone as something different but cool. But they didn't.
What sucked:
Not shooting in proper locations -- The first scene at Area 51 just looked wrong. And the dialogue sounded so tinned and rubbish.
-- The terrible jungle chase WAS actually partly filmed in a proper jungle location (in Hawaii or something) but they merged it with film/photo assets from Argentina, which resulted in the very fake look.
-- There was no sense of geography. The waterfall seemed to lead them through a mountain under a river, only for them to come out by a big hidden temple / sand sentinel thing which was basically out in the open, and surely viewable from the air (like the Nazca lines)... their escape from that huge structure at the end seemed dubiously fast.
Irina Spalko -- Not as good as any prior indy villain. She was REALLY hammy in fact. Terrible. Stalin probably did have all sorts of weird research projects, but her attempting to mind read at the beginning of the movie was the first warning that this movie was going to disappoint.
Vine swinging, sword fighting Shia. -- Had they toned this down from Attack of the Clones levels of improv action, not only would Mutt have been a more popular character, but the whole movie wouldn't have sucked so badly.
Driving off a cliff, landing on a tree. -- dumb dumb dumb.
Oxley generally being annoying -- If Connery had agreed to be in it, Henry Jones Snr was going to have Oxley's annoying afflictions apparently
Marion Ravenwood. -- Karen Allen basically had this look on her face through the entire thing like "

yay I'm in a movie again"
I don't understand how - in a film crew of hundreds - with such massive companies behind it: no-one stood up to some of that BS.