Maybe once the creator saves the map, they have the option to enable their guild or certain friends to have access to it no matter what, but say, if its your guild, you can't let anybody else in not in your guild, only the creator of the map may. Same with the friends list.
Maybe there can be a 'Kick' feature in private rooms. Say if someone is being rude and won't leave - Wala! Problem fixed.
Maybe there can be a 'Private Map Scroll'. Its a scroll that has all your private maps on it, or maps people allow you in! That way if your in the Fruit Bat Territory - And you want to go to one of your private rooms, but your town/dungeon scroll is cooling down? Then, no need to get flustered!