Assuming the variables
pt1 &
pt2 represent 3D points, the expression
(mapcar '- pt1 pt2) is returning the vector from
pt2 to
pt1. Since a vector is simply a displacement, the vector can also be viewed as a 3D point relative to the origin, as shown by the following diagram:
Now
(distance '(0.0 0.0 0.0) (mapcar '- pt1 pt2)) will return the distance from the origin to this point, i.e. the length of the vector from
pt2 to
pt1 (or indeed
pt1 to
pt2).