You are right in that power will keep increasing as you apply a force, this is because the velocity increases and with increasing velocity you are able to do more work (remember power is work/time). A car travelling at 100 m/s can do more work than a car travelling at 50 m/s.
Here is an example: You're pushing a block with 10 newtons of force and it is CURRENTLY moving at 3 m/s. You have 10*3 = 30 Watts of power AT THE INSTANT when velocity is 3m/s and force is 10 newtons. You could do 30 Joules of work in one second at that instant (because power = w/time). I understand this is probably not a very good explanation but it's the best I could come up with at the moment. Hope this helps!