OK so if I have a function Like this:
...
Which create a selections set of handles stored in an Xrecord. I wouldn't have to worry?
If all you are doing is storing handle in Xdata, then using the list of handles to recreate selection sets, it looks like it should work. Keep in mind that you list of handles may be out-of-date - objects regularly get created and destroyed. Break a line, and your old handle is no longer valid.
There is some question about why you are doing this. It sounds like you are trying to use selection sets to mimic the functionality of a group. So why not just use a group?
There is no danger in using a group, just so long as you realize you are using a group, and not a block. Groups have their uses, and blocks have their uses. Some of these uses sort-of overlap, but Groups are Groups and Blocks are Blocks. The only time I really see people get into trouble is when they don't know the difference, and try to work with a group as if it is a block.