The ObjectId you passed to function "ChangeNestedEntitiesToColor()" is the ObjectId of an Entity (line, circle, Blockreference...), while it seems that the function expects an ObjectId of a block definition (BlockTableRecord). That is why the error.
Assume the function is correct (e.g. it needs a BlockTableRecord's ObjectId to work with), then you must pass in an ObjectId of BlockTablerecord, not an Entity. As matter of fact, BlockTableRecord is not selectable (not directly).
You need to do it this way:
1. Get a SelectionSet as you did, but better yet, you may want to use a selection filter to only select blocks (BlockReference, that is). This way, anything in the SelectionSet is a BlockReference;
2. Looping through the selected entities (BlkockReference), for each BlockReference, you find its block definition's ObjectId (ObjectId of a BlockTableRecord. Since it is possible several BlockReferneces may be based on the same BlockTableRecord, you better end up with a list/collection of ObjectId of BlockTableRecords.
3. Then loop through the collection of ObjectIds of BlockTableRecord with the function "Change....()"