I'd probably say the same, but if I'm being honest, I think Djokovic is the one Rafa is really trying to hold off because I think Nadal has a lot more arguments for Federer than he does with Djokovic.
With Federer, Rafa can say he has more Masters, Olympics, H2H advantage, similar longevity, better win percentage, winning 2+ slams on each surface, and his age group was tougher than Fed's original generation. They're tied with 5 Year-End Number One's but Rafa is the one more likely to add to it in the future. There's probably more as well but those are the few that pop into mind.
With Novak, a lot of the above can't be used in his favor. Also, it is sadly more than likely that Fed is essentially done adding to his resume for the most part as he's approaching 40 (!!!) while Novak is far from being finished.