When "Why" is used as an exclamation, it would normally be followed by a comma instead of an exclamation mark, it's an interjection used to express surprise, disagreement, indignation, hesitation, impatience etc:
//Why, don't be silly! (Collins)
//Why, here's what I was looking for! (Merriam Webster)
In J. B Priestley's An Inspector Calls, when Arthur Birling talked about the greatest technological progress of the time such as airplanes, automobiles, and ships, he said:
And then ships. Why, a friend of mine went over this new liner last week – the titanic – she sails next week – forty-six thousand eight hundred tons – new york in five days – and every luxury – and unsinkable, absolutely unsinkable.
He use "Why" to express surprise. Arthur was born into a humble background and became a prosperous manufacturer, he was pompous but rather "provincial" in his speech.