This happens every time on my Td4, it's quite a strong sound on mine. I asked some folks and they said it was something in the oil system spinning down.
I doubt it's the 'correct' working of the crankcase valve, because mine hasn't been cleaned out, probably ever. That's a new thing I've just learned about and will look at it this week.
You mention cleaning the crankcase breather AND the PCV (Positive Crankcase Ventilation), I thought these were the same thing? Or are they separate parts contained within that same asembly?