Since you are already using a dynamic block, how about making the leader part of the block definition, with a polar stretch parameter.
That would work out well. How did you make the leader part of that?
Created an MLeader with the desired appearance.
Exploded the MLeader and made a block from components.
Added Linear Stretch Parameter with two grips to the annotation side of the leader.
Added Linear Stretch Parameter with one grip to the leader dogleg.
Added Polar Stretch Parameter with one grip to the leader line & arrowhead.
I can upload a drawing later if you want.