When you first finish residency, you are typically board eligible for a period of time and can take the exam a certain number of times before becoming not eligible. At that point you become board certified for a period of time. If you do not become board certified before your eligibility expires, you have to do remedial training which can vary from supervised study programs all the way up to repeating a residency year to become eligible again.
For some specialties you cannot even take the exam for the first time until you have been in practice for a period of time. This is more prevalent in the surgical specialties. Also, in the surgical specialties, two part exams (written and oral) are not uncommon, so one could be board-eligible for sometimes up to a couple of years before being able to be board-certified even if that person doesn't fail any of the exams.
For office based practices, having a board certification is worthwhile as some insurances require it to join their panels, but otherwise not essential. For those needing hospital privileges (e.g. surgeons) board certification becomes very important as many hospitals will not give privileges to those not board certified or at least board eligible. Since you can't stay board eligible forever, you need to get board certified.