Expand your question to many different substances if it will help. For instance, you could ask the same question of ammonia (which can lose a proton to become NH2- or gain a proton and become NH4+). In water it acts like a base because is a stronger base than it is an acid. You should also note that it is a stronger base than water, so it removes some of the protons from water. But it is a weaker acid than water, so it's not going to give up a proton when H2O is there to provide protons.
You can also consider a triprotic amino acid such as glutamic acid added in its zwitterion state. The side chain starts protonated, but will lose some of that proton when added to water, because the carboxylic acid sidegroup is a stronger acid than water.
Along with the bicarbonate example you proposed, these are all cases of amphoteric compounds in water acting predominantly in one fashion. It's a trend in chemical behavior and nothing unique for HCO3-.
The pKa2 for H2CO3 is 10.8 while the pKb2 of CO32- is 7.6. Because pKb2 is lower than pKa2, the compound when added to a neutral aqueous solution is more likely to act like a base than an acid. Basically, HCO3- is a slightly stronger base (albeit it is a weak base) than it is an acid.
