Orphaned callings at LDS.org that are no longer in MLS

Discussions around using and interfacing with the Church MLS program.
genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Orphaned callings at LDS.org that are no longer in MLS

Postby genman » Fri Jan 04, 2013 8:43 am

Description of Problem:
There have been several reports of a ward's calling positions list getting out of synch between MLS and LDS.org. In particular extra or redundant or obsolete callings showing up on LDS.org with no immediate way to delete them. I recently requested a Unit Data Refresh of the membership data to resolve some of these redundant callings. After going through all of that, I was able to immediately duplicate the problem again, though, by adding a new custom calling and then removing it from MLS, thus causing it to be orphaned on the list at LDS.org with no way to delete it without having to go through a full Unit Data Refresh again. This problem does appear to be readily repeatable, so should be fixable.

Here are the steps to Duplicate the Problem:
1. In MLS, go to the "Other Callings" tab under Organization.
2. On the Custom Positions subtab, click "Add New Position" and name it, say, Testing3. Then press Save.
3. On the "Other Callings" subtab, click "Add Position", select this new Testing3 position, then add a person to the calling. Then press Save.
4. Press Send/Receive Changes button
5. At this point, check LDS.org to see if the new Testing3 position shows up under Other. It may not show up right away. I did not see it right away.
6. Now go back to Custom Positions subtab. Click "Remove" for the Testing3 position. Then press OK.
7. Press Send/Receive Changes button again.
8. At this point, check LDS.org again to see if Testing3 position shows up. You may need to exit LDS.org and close the browser and open it again and sign in to LDS.org to the directory. When I did that, I now see the new Testing3 position there. But it no longer exists in MLS, so there is no longer a way to delete it from LDS.org other than doing a "Unit Data Refresh", which is time consuming and wipes out placeholders for unfilled positions in MLS.

Preliminary Analysis of Problem and Potential Solutions:
The root of the problem is that LDS.org and MLS are getting out of synch in the first place. The Unit Data Refresh is just a brute force workaround that works temporarily until the problem shows up again. The problem appears to be most likely a race condition on posting database changes from MLS to LDS.org via Send/Receive Changes. One theory is that the delete command for the calling arrives at the central database before the add command of the calling arrives there, causing the delete to be ignored, the add to be accepted, and now with no way for MLS to command the delete again. There are several possible solutions. One solution A is to ensure that any pending commands to add callings to the CDOL database are executed before any commands to delete callings. Another solution B would be for MLS to continue to try to delete the calling with each subsequent Send/Receive until there is positive feedback from the server that the delete was successfully executed. Another solution C would be for the server to keep a queue of delete comands and keep trying periodically for a day or more until the command is successfully executed, rather than just immediately drop the delete command if the object being deleted is not currently in the database. Somehow need to make it very hard for the delete command to be missed due to a race condition. Another quick user workaround solution D would be for MLS to keep track of anytime that a custom position is deleted to send a command to LDS.org at the next Send/Receive Changes to have any position with a name that matches the deleted custom position to be deleted. At least that way the user could re-create a custom position with the exact name as the orphaned one, do Send/Receive, then deleted that custom position, do another Send/Receive, which would then send another delete command for that position name for that ward organization in the central database. Not the best solution, but better and more surgical than having to do a full Unit Data Refresh, which takes 24 hours to complete and overwrites other things.

Recommendation:
Solution A may not robust enough, because that doesn't contain assurance that something else could cause the delete to be missed, such as a server service hiccup. Solution B and C are better because they provide positive feedback that the delete executed before ceasing to try to delete; however, this approach may be more complex to implement. Solution D doesn't really fix the problem, but it at least allows the users the ability to manually resolve the problem once it occurs, without having to do a Unit Data Refresh, which is a heavy blunt tool that can have other side-effects that take excessive time to deal with. I recommend solution D, but that requires a change to MLS. If changing the MLS software is not an option, then solution A or C may be the next best thing. Something needs to be done, though, because lots of people are experiencing problems with callings being out of synch between LDS.org and MLS, with no good way to workaround it reliably.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby genman » Fri Jan 04, 2013 9:12 am

I also submitted this problem report via the Feedback link on the bottom right of the lds.org home page.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby genman » Fri Jan 04, 2013 9:42 am

Is there a way to cross-post this to another forum also? This seems to overlap between the following sub-forums:

Clerk and Technology Support > MLS Support, Help, and Feedback
LDS.org Website > Leader and Clerk Resources
LDS.org Website > Directory
LDS.org Website > Beta Testing > Beta Directory

I posted it here under MLS Support because I think that a supporting change is needed to MLS to support resolving this interface problem between MLS and LDS.org. It may be more difficult and less reliable to try to resolve this problem without at least a small supporting MLS change.

User avatar
aebrown
Community Administrator
Posts: 14693
Joined: Tue Nov 27, 2007 8:48 pm
Location: Sandy, Utah

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby aebrown » Fri Jan 04, 2013 9:51 am

genman wrote:Is there a way to cross-post this to another forum also?

You shouldn't post the whole content of this topic or its original post in another forum, but if there is an existing topic somewhere else that deals with an issue like this, you can always make a brief post in that topic that links to this topic.

User avatar
Mikerowaved
Community Moderators
Posts: 3132
Joined: Sun Dec 23, 2007 12:56 am
Location: Layton, UT

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby Mikerowaved » Fri Jan 04, 2013 11:02 am

Thanks very much for the time you spent recreating and documenting this bug. I passed on the link for your report to Ryan Jones, who is one of the CDOL developers that frequents this forum. His team has been working on solving some of these oddities and I'm sure will benefit from your findings.
So we can better help you, please edit your Profile to include your general location.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Recommended Solution E

Postby genman » Fri Jan 04, 2013 11:59 am

Mikerowaved wrote:I passed on the link for your report to Ryan Jones, who is one of the CDOL developers

Thanks Mikerowaved, I appreciate that and the kind words.

genman wrote:I recommend solution D

On second thought, solution D is problematic. I noticed after doing the Unit Data Refresh that the custom positions list in MLS had lots of what appeared to be redundant names. We wouldn't want someone who is deleting one of those redundant names to end up deleting all callings with that name in that ward organization at LDS.org.

Maybe a solution E is the best solution at this point, which would be a change to MLS to allow the clerk to request that the organization data in MLS be used to refresh the server database. It could be put under the MLS File menu next to the "Request Unit Data Refresh" menu item. Could call it "Request Sync Local Organization Data to LDS.org". When the clerk selects that, it would set a temporary flag in MLS that would take effect upon the next Send/Receive Changes. At that time, MLS would create a file of all of the organization positions and upload it to the server. On the server side, the file could be queued up to run later when the server is less busy. When the file is unpacked, the server software would know to in-effect do a unit data refresh of the organization data based on the contents of this file that contains all of the callings. In other words, the server data would delete all callings for the unit and repopulate them with the contents of this file.

That would resolve all of the issues, both of callings in MLS that are not showing up in LDS.org, and it would resolve the orphaned obsolete callings at LDS.org that are no longer in MLS. It is also clean in that it should reliably recreate at LDS.org what the clerk has certified is correct in MLS. The only downside that I can think of is that when the clerk does that, and presses Send/Receive, it could take a day or more for the change to take effect online. But other than that latency, there is very little wasted touch labor.

So I am now recommending solution E. It should resolve all of the outstanding issues that I am aware of.

russellhltn
Community Administrator
Posts: 20762
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby russellhltn » Fri Jan 04, 2013 12:22 pm

The problem with Solution E is that in the future callings can be entered via the web and not just MLS. That's probably why we have the current system with MLS forwarding the data to CHQ to be inserted into the system.
Have you searched the Wiki?
Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Re: Recommended Solution E

Postby genman » Fri Jan 04, 2013 12:34 pm

genman wrote:It could be put under the MLS File menu next to the "Request Unit Data Refresh" menu item. Could call it "Request Sync Local Organization Data to LDS.org".

Whatever it is named, it needs to be clear that "Request Unit Data Refresh" is requesting a data flow from the server database to the local database. And that this other "Request Sync" is requesting a data flow from the local database to the server database. Maybe the following menu item names would clarify that:

File >
Request Unit Data Refresh DOWNLOAD
Request Unit Org Data Refresh UPLOAD

The window that pops up could add clarification text to make it extra clear to the Clerk which direction of data flow is being sought.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Re: Orphaned callings at LDS.org that are no longer in MLS

Postby genman » Fri Jan 04, 2013 12:36 pm

russellhltn wrote:The problem with Solution E is that in the future callings can be entered via the web and not just MLS. That's probably why we have the current system with MLS forwarding the data to CHQ to be inserted into the system.

Why is that a problem? Please elaborate.

We already have the capability to edit things at both LDS.org and MLS, such as phone numbers. No problem. If a phone number is edited at LDS.org, then the next time Send/Receive is done at MLS the latest phone number that was edited in the server database is downloaded to MLS. But if the phone number was edited at MLS, then the next time Send/Receive is done at MLS then this phone number is uploaded to the server database. Same thing for callings list when they are editable at LDS.org.

So not sure what this has to do with Solution E being a problem.

Just as with the current Request Unit Data Refresh download, when the request is sent via Send/Receive, any local changes made recently are also sent along with the Send/Receive, so that when the Unit Data Refresh is actually downloaded a few hours later, all of the data on the server is current. That's currently the way Unit Data Refresh works and is not a problem.

The Request Unit Org Data Refresh Upload would work analogously. Any changes that were made locally or on the server would be synched up when the Send/Receive Changes is executed. So any online org changes made up to the point of the Send/Received that sent the request would be included in the Refresh. The only thing that would be missed would be any org changes made between when the request was sent and when it was run in the background up to a day later. But we are already living with that same limitation with the Unit Data Refresh download that any changes made in MLS between when the request was sent and when the download happened would be overwritten. Not a big deal, though, because those Unit Data Refreshes are not done very often.

In the case of the Requested Unit Org Data Refresh Upload, could make it even better by automatically temporarily disabling the ability to edit callings data online between the time of the Org Data Refresh Request and when it actually executes on the server data for the unit.

Given that, do you still see a problem with Solution E? If so, please explain.

genman
Member
Posts: 136
Joined: Wed Feb 08, 2012 4:51 pm
Location: US

Re: Recommended Solution E

Postby genman » Fri Jan 04, 2013 1:41 pm

genman wrote:File >
Request Unit Data Refresh DOWNLOAD
Request Unit Org Data Refresh UPLOAD

This approach of adding the new Org Data Refresh Upload command also provides the infrastructure for uploading other things from local data to the server as well, such as home teaching and visiting teaching route data. My understanding is that, per the roadmap, home/visiting teaching organization are planned to be supported at LDS.org at some point. Having this Org Data Refresh Upload capability would dovetail and enable that nicely. So this is not wasted effort but would actually facilitate the implementation of future capability. Even if the long long term plans are to do-away with MLS altogether, that is going to take quite a while to get there, and having a way to phase it out gracefully in steps is needed. This would provide infrastructure for doing that. Adding new capability to LDS.org is less risky if we still have a way in the meantime to treat the Org data in MLS as the trusted master data with the mature and tested software baseline.

The clerk having a last-resort way to reset/sync the Org data at LDS.org to match what is in MLS makes it less risky to add new things to LDS.org, such as home/visiting teaching organizations. It would be nice to at least have visibility into the home/visiting teaching organization at LDS.org as a first step. This Org Data Refresh Upload feature would provide the ability to make sure it is an accurate data baseline to start with and to fix it if it ever gets out of sync.

More reasons to add a "Request Unit Org Data Refresh UPLOAD" menu item.


Return to “MLS Support, Help, and Feedback”

Who is online

Users browsing this forum: No registered users and 1 guest