since the tree is yours and the leaves are falling in their land under normal wind (not storm) then law requires you not to cause any discomfort to your neighbours and the leaves are indeed making their property less enjoyable and the action is bordering on nuisance. hence you are liable to remove the leaves as per my understanding.
You can send legal notice through your advocate
If you want legal advise then you can call me on mobile no 9359867870
Advocate IRFAN SHAIKH