Google Calendar Sync Issue (Robots.txt)

Discussions about the Calendar Tool at lds.org. Questions about the calendar on the classic site should be posted in the LUWS forum.
ramonc
Member
Posts: 50
Joined: Mon Oct 08, 2012 9:49 am

Re: Google Calendar Sync Issue (Robots.txt)

Postby ramonc » Tue Aug 26, 2014 1:07 pm

For what it is worth department workaround- I just created a new calendar in google calendar - SLC-Manual. Downloaded the .ics file from the lds.org link and then imported into my SLC-Manual. Come Saturday night I will delete my google calendar SLC-Manual, Create it again, download from lds.org again and import it again. I will just keep doing this as needed until issue is resolved, thus I am somewhat current for my uses anyway.

russellhltn
Community Administrator
Posts: 25279
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Google Calendar Sync Issue (Robots.txt)

Postby russellhltn » Tue Aug 26, 2014 1:12 pm

ramonc wrote:Come Saturday night I will delete my google calendar SLC-Manual, Create it again, download from lds.org again and import it again.

I wonder if it's necessary to delete the calendar. Have you tried re-importing into the existing calendar?

My primary use is to get the sync onto my Android device. Unlike iOS, Android does not have a way to do it natively. But there are apps that can be installed. One possible advantage - you can sync as fits your schedule rather than wait until Google gets around to it.
Have you searched the Wiki?
Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.

brucejay
New Member
Posts: 5
Joined: Sun Jul 31, 2011 3:15 pm

Re: Google Calendar Sync Issue (Robots.txt)

Postby brucejay » Thu Aug 28, 2014 8:11 pm

This is a problem on Google's end. I have had this problem before, and have it now. Google is not able to communicate with the server. However, a browser can communicate with the server and download the calendar to an ics file that can be imported into the Google calendar. The questions and comments need to be directed to Google as to the sync process.

russellhltn
Community Administrator
Posts: 25279
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Google Calendar Sync Issue (Robots.txt)

Postby russellhltn » Fri Aug 29, 2014 12:16 am

brucejay wrote:This is a problem on Google's end. I have had this problem before, and have it now. Google is not able to communicate with the server. However, a browser can communicate with the server and download the calendar to an ics file that can be imported into the Google calendar. The questions and comments need to be directed to Google as to the sync process.


It may be an issue of interpreting standards. Robots.txt is a file used by the site to direct web crawlers what is and is not to be indexed. (See Robots exclusion standard) The question is, should Google pay attention to Robots.txt when being directed by a specific user to retrieve an ICS file from a specific location for their own use? It's understandable that the church doesn't want Google to index everyone's calendar into their search engine, but there's no intent to block what Google does in behalf of a specific request from a user.

Web browsers ignore robots.txt as they are not "robots" but work as directed by the user.
Have you searched the Wiki?

Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.

rbraithwaite1995
New Member
Posts: 4
Joined: Sun Aug 24, 2014 8:46 am

Re: Google Calendar Sync Issue (Robots.txt)

Postby rbraithwaite1995 » Sun Aug 31, 2014 10:28 am

So basically nothing has been resolved on this issue?

To the person who recommended to delete the "robot.txt" from the HTML. Can you be more specific about how this is done?

User avatar
aebrown
Community Administrator
Posts: 15089
Joined: Tue Nov 27, 2007 8:48 pm
Location: Sandy, Utah

Re: Google Calendar Sync Issue (Robots.txt)

Postby aebrown » Sun Aug 31, 2014 10:48 am

rbraithwaite1995 wrote:So basically nothing has been resolved on this issue?

To the person who recommended to delete the "robot.txt" from the HTML. Can you be more specific about how this is done?

Whoever made that suggestion is a bit confused. The file robots.txt (it has an s on that "robots", by the way) is not in the HTML. It is a separate file. And since it is one file for LDS.org, you can't just remove it from the Calendar sync service without affecting more than just sync URLs.

Search engines gather the information for their search indexes by "crawling" the web using applications known as "web crawlers" or "spiders" or "robots" to find pages of content, then they put that information in indexes so that they can search for that content. The robots.txt file allows sites to tell search engines to ignore anything they find in certain parts of their site.

The robots.txt file for LDS.org can be found at https://www.lds.org/robots.txt . If you examine the file, you'll see a line that says "Disallow: /church-calendar". That means that web crawlers are supposed to ignore everything found in lds.org/church-calendar and below. So if we were only talking about web crawling, then Google would be correct in ignoring sync URLs, since they are found below lds.org/church-calendar. But reading a sync URL has nothing to do with crawling the web, so Google's Calendar application is misinterpreting the purpose of robots.txt if they read that and assume that they can't open a file from there.

russellhltn
Community Administrator
Posts: 25279
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Google Calendar Sync Issue (Robots.txt)

Postby russellhltn » Sun Aug 31, 2014 11:32 am

aebrown wrote:Google's Calendar application is misinterpreting the purpose of robots.txt if they read that and assume that they can't open a file from there.

I'm not so sure. When I look up the purpose of Robots.txt, it seems to be aimed at automated processes, not just search engine web crawlers.

When I look for complaints about the robots.txt for Google calendar - I can find them, but they are from years ago. And thanks to the Wayback Machine we can see that the there was a change to the church's robots.txt done on Aug 13. So while we can argue standards, it was the church that made the change that broke things. The fact it hasn't been changed back makes me think there's a security issue involved.
Have you searched the Wiki?

Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.

diidu
New Member
Posts: 5
Joined: Mon Sep 01, 2014 12:35 pm

Re: Google Calendar Sync Issue (Robots.txt)

Postby diidu » Mon Sep 01, 2014 12:55 pm

I have had LDS calendar/Google calendar sync issue from July or so. Today started working on it and finally recreated the link. At the first attempt of adding the new link to Google calendar I saw the mentioned robots.txt error for the first time. After that I did not see the error anymore, but no sync either. Copying the link to browser window displays an empty calendar in Firefox.

Another issue (should probably start a new thread for this, but I'm not doing it now) is that for few montyhs already I have not received any email notifications from lds calendar even though based on my settings and events added I should have.

diidu
New Member
Posts: 5
Joined: Mon Sep 01, 2014 12:35 pm

Re: Google Calendar Sync Issue (Robots.txt)

Postby diidu » Mon Sep 01, 2014 12:57 pm

>> Copying the link to browser window displays an empty calendar in Firefox.

With this I mean the link for embedded calendar.

russellhltn
Community Administrator
Posts: 25279
Joined: Sat Jan 20, 2007 2:53 pm
Location: U.S.

Re: Google Calendar Sync Issue (Robots.txt)

Postby russellhltn » Mon Sep 01, 2014 1:09 pm

diidu wrote:After that I did not see the error anymore, but no sync either. Copying the link to browser window displays an empty calendar in Firefox.

How long did you wait? There's a 9 minute throttle on the calendar. If you try to use it within the 9 minutes, you'll get a http error (although many things won't report that to you.) In most browsers, you'll end up with nothing. Creating a new URL won't reset the timer - it's tied to your account.

diidu wrote:Another issue (should probably start a new thread for this, but I'm not doing it now) is that for few montyhs already I have not received any email notifications from lds calendar even though based on my settings and events added I should have.

It's addressed in the LDS.org calendar not sending email updates thread.
Have you searched the Wiki?

Try using a Google search by adding "site:tech.lds.org/wiki" to the search criteria.


Return to “Calendar”

Who is online

Users browsing this forum: No registered users and 2 guests