TES4: Oblivion is lagging for me....why?

I have done a bunch to help stop my lag overall. I changed settings, got Operation Optimization and Streamline 3.1. They have helped a lot, but I still lag when I look in the direction of NPCs. If I am by myself there is no lag. When I'm by NPCs, I lag/stutter. Is there any setting I can change to fix this? Or a program to get that would help?