Azazel
The so-called scapegoat was originally never actually blamed for causing any problem. In fact the source ritual brings forward a pair of equally unblemished goats and selects, at random (by lottery), one of them to be sacrificed on the temple altar. The other goat is the "scape" goat, given a transmission by the high priest, who lays his hands on the goat's head in a sort of magical act of transferring the sins of the nation, to carry the pollution away ... to Azazel, the desert, wilderness, wasteland.
The goat is simply a messenger--again, he's never considered responsible for committing the transgressions of the nation (or of anyone, really--he's just a goat, after all, capable of capricious mischief but nothing comparable to what humans can do). He carries his burden away from the community, out to the harsh, uninhabitable zone, where he's set loose to wander and survive as well as he can (or, according to some accounts, pushed off a cliff--but even then, goats are excellent climbers...).
The desert, for this culture, was a demonic realm--people forced out into it go crazy, wandering lost and alone, have nightmarish visions, uncertain if they are alive or dead. As such it's entirely receptive to the gift the goat conveys, especially considering that the "sins" of the people probably originated in the demonic realms in the first place. And considering that the proportion of wilderness to habitable land is heavily weighted to the wild, there's little danger of the sins returning within the first day or so (though of course demons travel quickly, not necessarily confined to the rocky path).
The removal of the collective sins (if effective at all) is temporary--just because you washed the floor, in no way is it protected from the crap you track in the following day, though you may be inspired to be a bit more careful for a brief moment. It's an annual attempt to clear a ritual space for a specific purpose, to enter a direct unification with divine reality without being dangerously distracted.
It's also worth remembering that the displacement of the collective pollution does not absolve individuals of their responsibility for wrongdoing. It's still necessary to correct one's own misdeeds in the usual tiresome ways, such as paying fines and seeking forgiveness from the person you were rude to. Whether it's successful in clearing the collective is another question entirely. It may constitute a sacrifice, but the goat gets the worst end of the deal.
Gateway
The holy of holies was considered a portal, a conduit unifying heaven and earth, a very potent spot where matter and energy are actively interchangeable. Attempting to enter the without attending to every preparatory detail would not only defile the space, but would make the the people susceptible to immolation, insanity, plague. That's the sort of danger they were worried about. Not simply whether they had been disproportionately grumpy that morning, but whether they would be competent to withstand the enormity of their lapses.
Without veering too far into superstitions about the dangers of psychic pollution, I'll just mention that, though metaphorical, there's also reasonable evidence suggesting that people are susceptible to physical (as well as emotional, social, psychological) ailments as a result of psychic toxins. And certainly, to be effective in one's endeavors, it's important to have clarity of purpose, but also to not be bogged down by too much unresolved trauma, grief guilt and other distracting, soul-clogging flotsam. So even from a purely practical standpoint, finding ways to clear up the debris--at least once a year--would seem important. A goat's life might depend on it.
