The analogy I use is this: Certain elements, like mercury, are poisonous in their elemental form. However, when mercury is mixed with other metals, it creates a stable and safe compound. Think of it this way. Chlorine gas is highly toxic. However, a compound of Sodium and Chloride is what we eat every day: table salt. Amalgam has a track record of over fifty years and no one has yet proved any adverse affects. They may take one anecdotal example, and claim it as a fact. For example, my aunt is highly allergic to some material in amalgam. This is not necessarily mercury, because there are other metals. Does this mean we should not use amalgam for the millions of people it is safe for? Some people have worse reactions to peanuts. Should we deprive those that use it as a healthy protein source? The answer to both of these is no. We should not let manipulation of the facts feed our fears. If you want another anecdote, my grandmother has had her amalgam fillings for over fifty years, and she is healthier than you my friend. But, hopefully, you aren't listening to anecdotes anymore.
As you can tell, I feel very strongly that the non-dental public is well informed. As stated previously, if you want to change them so your teeth look more natural, that is fine, but know that amalgam has the longer track record.