@alogim
You're sort of naively using the simplest form of the tree data structure. If I understand your question correctly you should look at some BSP tree structure implementations, as they seem to suit your case quite well.