I am working on a directory application using Google Apps API to store all data(users, groups, org units) from APIs into data store and then query the data store to display the user for searching and viewing.
I am doing the loading of users and groups using tasks. The approach that I am following is to read data from API and then create an entity of every user in a loop and then look for nextPageToken and if it is not null, assign another task of loading the users. The same approach is being followed for groups and OU as well.
Now the problem is if I run it on a google domain having 2k users, it works fine, however when I run it on environment having 90K users, it works fine till it reaches 12-13k users and then it stops responding and tasks stop responding and the memory usage on my machine goes higher and this is the case on my local devserver as I havent deployed it yet on appengine.
There is lots of backend code following the approach that I described above, however I am not sure what to provide here, so please ask questions that you may think could be causing the problem. I will paste the snippet here, whatever is asked for!
The actual production server will have double the amount of users , i.e. close to 200K, which concerns me a lot. Please help!
Aucun commentaire:
Enregistrer un commentaire