Why do we allow the members of the electoral college vote which ever way they want?

if we got rid of the electoral college, big states like CA & NY would elect the president and small states would have no say. I have NEVER heard of a electoral voter going against what they are supposed to do.