Without going into detail, I'd say that a preliminary problem is that the process is wrong. Read Don Gale's site on how to mount rotors.
The rotor could be defective. Remember Josh's new warped rotors of a few weeks back? I've been reading about a lot of such incidents with new brembo rotors as of recent. But it's a far cry, I'd say. Check your process. The set screws aren't even necessary.
best, whit