Orphaned callings at LDS.org that are no longer in MLS
Posted: Fri Jan 04, 2013 8:43 am
Description of Problem:
There have been several reports of a ward's calling positions list getting out of synch between MLS and LDS.org. In particular extra or redundant or obsolete callings showing up on LDS.org with no immediate way to delete them. I recently requested a Unit Data Refresh of the membership data to resolve some of these redundant callings. After going through all of that, I was able to immediately duplicate the problem again, though, by adding a new custom calling and then removing it from MLS, thus causing it to be orphaned on the list at LDS.org with no way to delete it without having to go through a full Unit Data Refresh again. This problem does appear to be readily repeatable, so should be fixable.
Here are the steps to Duplicate the Problem:
1. In MLS, go to the "Other Callings" tab under Organization.
2. On the Custom Positions subtab, click "Add New Position" and name it, say, Testing3. Then press Save.
3. On the "Other Callings" subtab, click "Add Position", select this new Testing3 position, then add a person to the calling. Then press Save.
4. Press Send/Receive Changes button
5. At this point, check LDS.org to see if the new Testing3 position shows up under Other. It may not show up right away. I did not see it right away.
6. Now go back to Custom Positions subtab. Click "Remove" for the Testing3 position. Then press OK.
7. Press Send/Receive Changes button again.
8. At this point, check LDS.org again to see if Testing3 position shows up. You may need to exit LDS.org and close the browser and open it again and sign in to LDS.org to the directory. When I did that, I now see the new Testing3 position there. But it no longer exists in MLS, so there is no longer a way to delete it from LDS.org other than doing a "Unit Data Refresh", which is time consuming and wipes out placeholders for unfilled positions in MLS.
Preliminary Analysis of Problem and Potential Solutions:
The root of the problem is that LDS.org and MLS are getting out of synch in the first place. The Unit Data Refresh is just a brute force workaround that works temporarily until the problem shows up again. The problem appears to be most likely a race condition on posting database changes from MLS to LDS.org via Send/Receive Changes. One theory is that the delete command for the calling arrives at the central database before the add command of the calling arrives there, causing the delete to be ignored, the add to be accepted, and now with no way for MLS to command the delete again. There are several possible solutions. One solution A is to ensure that any pending commands to add callings to the CDOL database are executed before any commands to delete callings. Another solution B would be for MLS to continue to try to delete the calling with each subsequent Send/Receive until there is positive feedback from the server that the delete was successfully executed. Another solution C would be for the server to keep a queue of delete comands and keep trying periodically for a day or more until the command is successfully executed, rather than just immediately drop the delete command if the object being deleted is not currently in the database. Somehow need to make it very hard for the delete command to be missed due to a race condition. Another quick user workaround solution D would be for MLS to keep track of anytime that a custom position is deleted to send a command to LDS.org at the next Send/Receive Changes to have any position with a name that matches the deleted custom position to be deleted. At least that way the user could re-create a custom position with the exact name as the orphaned one, do Send/Receive, then deleted that custom position, do another Send/Receive, which would then send another delete command for that position name for that ward organization in the central database. Not the best solution, but better and more surgical than having to do a full Unit Data Refresh, which takes 24 hours to complete and overwrites other things.
Recommendation:
Solution A may not robust enough, because that doesn't contain assurance that something else could cause the delete to be missed, such as a server service hiccup. Solution B and C are better because they provide positive feedback that the delete executed before ceasing to try to delete; however, this approach may be more complex to implement. Solution D doesn't really fix the problem, but it at least allows the users the ability to manually resolve the problem once it occurs, without having to do a Unit Data Refresh, which is a heavy blunt tool that can have other side-effects that take excessive time to deal with. I recommend solution D, but that requires a change to MLS. If changing the MLS software is not an option, then solution A or C may be the next best thing. Something needs to be done, though, because lots of people are experiencing problems with callings being out of synch between LDS.org and MLS, with no good way to workaround it reliably.
There have been several reports of a ward's calling positions list getting out of synch between MLS and LDS.org. In particular extra or redundant or obsolete callings showing up on LDS.org with no immediate way to delete them. I recently requested a Unit Data Refresh of the membership data to resolve some of these redundant callings. After going through all of that, I was able to immediately duplicate the problem again, though, by adding a new custom calling and then removing it from MLS, thus causing it to be orphaned on the list at LDS.org with no way to delete it without having to go through a full Unit Data Refresh again. This problem does appear to be readily repeatable, so should be fixable.
Here are the steps to Duplicate the Problem:
1. In MLS, go to the "Other Callings" tab under Organization.
2. On the Custom Positions subtab, click "Add New Position" and name it, say, Testing3. Then press Save.
3. On the "Other Callings" subtab, click "Add Position", select this new Testing3 position, then add a person to the calling. Then press Save.
4. Press Send/Receive Changes button
5. At this point, check LDS.org to see if the new Testing3 position shows up under Other. It may not show up right away. I did not see it right away.
6. Now go back to Custom Positions subtab. Click "Remove" for the Testing3 position. Then press OK.
7. Press Send/Receive Changes button again.
8. At this point, check LDS.org again to see if Testing3 position shows up. You may need to exit LDS.org and close the browser and open it again and sign in to LDS.org to the directory. When I did that, I now see the new Testing3 position there. But it no longer exists in MLS, so there is no longer a way to delete it from LDS.org other than doing a "Unit Data Refresh", which is time consuming and wipes out placeholders for unfilled positions in MLS.
Preliminary Analysis of Problem and Potential Solutions:
The root of the problem is that LDS.org and MLS are getting out of synch in the first place. The Unit Data Refresh is just a brute force workaround that works temporarily until the problem shows up again. The problem appears to be most likely a race condition on posting database changes from MLS to LDS.org via Send/Receive Changes. One theory is that the delete command for the calling arrives at the central database before the add command of the calling arrives there, causing the delete to be ignored, the add to be accepted, and now with no way for MLS to command the delete again. There are several possible solutions. One solution A is to ensure that any pending commands to add callings to the CDOL database are executed before any commands to delete callings. Another solution B would be for MLS to continue to try to delete the calling with each subsequent Send/Receive until there is positive feedback from the server that the delete was successfully executed. Another solution C would be for the server to keep a queue of delete comands and keep trying periodically for a day or more until the command is successfully executed, rather than just immediately drop the delete command if the object being deleted is not currently in the database. Somehow need to make it very hard for the delete command to be missed due to a race condition. Another quick user workaround solution D would be for MLS to keep track of anytime that a custom position is deleted to send a command to LDS.org at the next Send/Receive Changes to have any position with a name that matches the deleted custom position to be deleted. At least that way the user could re-create a custom position with the exact name as the orphaned one, do Send/Receive, then deleted that custom position, do another Send/Receive, which would then send another delete command for that position name for that ward organization in the central database. Not the best solution, but better and more surgical than having to do a full Unit Data Refresh, which takes 24 hours to complete and overwrites other things.
Recommendation:
Solution A may not robust enough, because that doesn't contain assurance that something else could cause the delete to be missed, such as a server service hiccup. Solution B and C are better because they provide positive feedback that the delete executed before ceasing to try to delete; however, this approach may be more complex to implement. Solution D doesn't really fix the problem, but it at least allows the users the ability to manually resolve the problem once it occurs, without having to do a Unit Data Refresh, which is a heavy blunt tool that can have other side-effects that take excessive time to deal with. I recommend solution D, but that requires a change to MLS. If changing the MLS software is not an option, then solution A or C may be the next best thing. Something needs to be done, though, because lots of people are experiencing problems with callings being out of synch between LDS.org and MLS, with no good way to workaround it reliably.