If you want say group1 all, group2 only these people, group3 these people, then you may have to do just that make multiple directories and assign read writes to particular people. Thinking some more may be easier to have multi directories, even duplicating code but the directory is linked to a group so easier to add people to a group with certain rights.
Where I worked 1000 pc's so Group privileges was extensively used.