Your job as the fellow will vary depending on the culture at your program. In large part your responsibility will be as the de-facto leader of the team. You should have a plan and know the course of all your patients, while being aware of your consults, admissions and discharges. You don't necessarily need to know the potassium of the guy in bed 4, your resident should know that (which isn't to say there aren't times when you'll have to know it).
You need to have the larger view - where are things heading, what do you need to plan for, how can you ensure the team is running smoothly. For some people the transition from resident to fellow is difficult, but you need to let your residents learn and make decisions on their own so that they also learn. However, you need to be aware of the decisions they're making so that you can step in when they're getting off track. Your attending will do the same for you from afar.
So think of yourself as the manager. You're managing the patients, and you're managing your residents. Teach when you're able (it'll also help you learn). Generally when I come in I look at all the labs, imaging and I/O very quickly on my patients and then I do a quick lap. Also you'll get a sense of what each of your patients actually needs, and what is important. So maybe for bed 3 I'll pay close attention to his renal function, bed 4 I'll see what his FiO2/PEEP are and what are his vent pressures, bed 5 I'll see if her HgB is stable after her variceal bleed etc etc. Then when we round I listen to all the details from the resident, let them tell me their plan, and then I change the plan as I see fit/necessary.
I also talk to the patients and families, though usually I'll do it after rounds so that things move quickly - nobody likes long rounds, nobody. I hold all the family meetings, I do the vent management, I make the plan and ensure it's being followed. If my residents are getting guff from a consultant or the hospitalist then I will call them and see if we can get on the same page. There's a variety of reasons this happens, and often it's just a miscommunication. If I anticipate there is going to be a difficult consult (for whatever reason) I'll usually just call it myself to spare the resident the indignity.
The first day on service is hard because everyone is new to you. By the time you're a day or two in you'll be surprised at how much you remember and understand what's happening with your patients. Keeping track becomes easier. I used to round with a huge sign-out sheet. Now if it doesn't fit on 1 piece of paper I don't print it.
Lastly, for each patient try and approach it as though you were alone. "If I didn't have so-and-so here, what would my plan be?" This will help you identify weaknesses and then you must actively work to correct them. Have fun also, the ICU is a fascinating physiology lab and you have the potential or proclivity to do well by your patients or to harm them. Identify your deficits, actively correct them and have respect for the unknown. Also, don't forget that like the ED, this is commonplace to you. Your patient's families will not be as comfortable, always try to remember what the experience is like for them and it'll help explain their behavior. This will help reduce your burnout, of which you are certain to experience more than once. Best of luck to you!