The portals are loaded separately from the map itself, I've actually had one case where I was in north konoha with no portals anywhere, effectively trapping me.... Going into group and then exit fixed that XD. Game be freaky bruh