Page 1 of 5

Scriptures as a Service (API for LDS Scriptures)

Posted: Sun Sep 14, 2008 6:51 am
by carljokl
I was wondering if there was interest in providing a service portal to the LDS online scriptures as some kind of web service? This would involve accessing the online scriptures in a machine friendly manner such as in the form of XML. Along with this a form of scripture query language such that you could submit a query like Moroni 10:3 - 5 and have the portal return the content of those verses in a machine parsable format.

I ask this because I am involved in a comunity in Second Life which aims to promote the Church to users of Second Life. As one of the more tech savy members of that comunity I was asked whether it would be possible to do some kind of scripting in Second Life to provide access to the scriptures to make it easier to share the gospel or have gospel discussions there.

This would be just one use of this service but I am sure a machine to machine "Scriptures as a Service" mechanism could help many projects which want to link into the scriptures but without having to store the whole scriptures themselves.

Posted: Sun Sep 14, 2008 10:44 am
by mkmurray
Carl Jokl wrote:I was wondering if there was interest in providing a service portal to the LDS online scriptures as some kind of web service? This would involve accessing the online scriptures in a machine friendly manner such as in the form of XML. Along with this a form of scripture query language such that you could submit a query like Moroni 10:3 - 5 and have the portal return the content of those verses in a machine parsable format.
We asked for this too when we started this project:

http://tech.lds.org/forum/showthread.php?t=113

It's nice to see another request for it. I would like to see more than just the scripture text also (the footnotes and everything).
Carl Jokl wrote:This would be just one use of this service but I am sure a machine to machine "Scriptures as a Service" mechanism could help many projects which want to link into the scriptures but without having to store the whole scriptures themselves.
Also storing it yourself for the purpose of then redistributing it to other people again violates copyrights. Storing it isn't the problem, but to then show it to other people besides yourself through some UI is.

Scripture API

Posted: Mon Sep 15, 2008 3:30 am
by carljokl
You mention "More than just the text"

Does a scripture API of any kind currently exist?

I was looking at one point to try and just parse the content of the LDS Online Scriptures. If the pages conform to valid xhtml then an XML parser could pull content out of them. This is however not the prefered way of doing things as it would be much preferred if the scriptures could be accessed directly from LDS.org. I don't know whether this would infringe on copywright as in essence the reason for trying to get the content from lds.org in the first place is that then we are only relaying already openly accesible content from the internet. I think a parser for the original pages would work but could break if the Church made any fundamental changes to the page structure. It is also possible to create some kind of mapping of the URL locations within LDS.org online scriptures so that a Scripture Query Language knows where to look to find the content.

I am not really yet in a position to work on this just yet because I am currently using most of my free personal development time completing a library of Java classes used for communication with LSL Scripts in Second Life. Once I have those working well I could try and resurrect that Scripture project.

Posted: Mon Sep 15, 2008 9:15 am
by mkmurray
Carl Jokl wrote:Does a scripture API of any kind currently exist?
No, unfortunately not. It would really be useful to us for the same reasons you have.
Carl Jokl wrote:I was looking at one point to try and just parse the content of the LDS Online Scriptures. If the pages conform to valid xhtml then an XML parser could pull content out of them. This is however not the prefered way of doing things as it would be much preferred if the scriptures could be accessed directly from LDS.org. I don't know whether this would infringe on copywright as in essence the reason for trying to get the content from lds.org in the first place is that then we are only relaying already openly accesible content from the internet. I think a parser for the original pages would work but could break if the Church made any fundamental changes to the page structure. It is also possible to create some kind of mapping of the URL locations within LDS.org online scriptures so that a Scripture Query Language knows where to look to find the content.
Yes, for the project I mentioned that we started, we first created a downloading mechanism. In the Rights and Use Information of the online Scriptures (http://www.lds.org/library/display/0,49 ... -1,00.html), it says you may "view, download, or print" the scripture content for "personal, noncommercial use." So we did just that. We created a desktop application that would help the user in downloading the scripture content for their own personal use within our application. We also did begin work on a parser, but you're right it's a difficult task and one where the content format could possibly change somewhat frequently.

So as for your uses, it may be more difficult to do something similar to what we did. Here are the legal statements that might get ya:
You may not post material from this site on another web site or on a computer network without our permission. You may not transmit or distribute material from this site to other sites.

I know you're not posting to another "site," but it may be interpreted that you are posting the scriptural content "on a computer network without permission."

This is where a Scripture API or web service would really help both of us. And as for me saying "more than just text," that is what I am hoping the web service could deliver (things like footnotes, etc.).

Posted: Mon Sep 15, 2008 12:49 pm
by WelchTC
Carl Jokl wrote:You mention "More than just the text"

Does a scripture API of any kind currently exist?
Not currently yet but a project to create one is in the works. Part of the issues, however, are how to control and maintain copyright on the scriptures.

Tom

Permission

Posted: Tue Sep 16, 2008 1:23 am
by carljokl
The legal note says that the content cannot be distributed without permission not that it cannot be redistributed at all. I would hope to secure permission. I don't think it is unreasonable. It is not being done for proffit.

The Second Life Scripting engine provides other obstacles too because there are lots of tight contraints in terms of memory and how much can be retrieved via HTTP. For example it has been the case that the response body from a http request can be no longer than 2048 characters. Outputting text in a chat in Second Life is constrained to 255 characters. For that reason I have already got some code in place to break down text into smaller fragments which Second Life can handle. LSL is limited too in that it has no built in XML parsing capabilities which means I would need to have server side code standing between Second Life and LDS.org in order to reformat everything into a form a LSL script could cope with.

It is a shame I wasn't around for the first LDS Tech broadcast or I could have asked about "Scriptures as a Service". Maybe I can try to for the next broadcast.

Posted: Fri Sep 19, 2008 8:10 am
by jbh001
I've noticed that the site scriptures.lds.org, plays nicely with the iPhone (so far). It would be nice to have it be a bit more user-friendly for mobile devices, but I don't have any suggestions at the moment.

Posted: Tue Sep 23, 2008 8:52 am
by WelchTC
jbh001 wrote:I've noticed that the site scriptures.lds.org, plays nicely with the iPhone (so far). It would be nice to have it be a bit more user-friendly for mobile devices, but I don't have any suggestions at the moment.
Yes, the developers did make some minor improvements so that the content is easier to read on a PDA such as the iPhone. More work can and will be done but it is not a high priority as the current priority is to publish in more languages.

Tom

Posted: Tue Sep 23, 2008 9:21 am
by carljokl
In terms of the scriptures as an API I suppose if a level of abstraction were introduced in a system which has to retrieve the scriptures from lds.org whereby it could parse the existing xhtml content as it currently stands but the details of how the parsing is being done being encapsulated from the rest of the application. This would allow for a parsing solution in the interim. When a machine parsable portal is available it would be a matter of swapping out the parser with a different parser but as far as the rest of the application the encapsulation would insulate the rest of the application from the change. As long as both parsers expose the same interface to the rest of the application and behave in the same way when using them it would just be "old parser drops out and new parser drops in".

Posted: Tue Sep 23, 2008 9:27 am
by mkmurray
Carl Jokl wrote:In terms of the scriptures as an API I suppose if a level of abstraction were introduced in a system which has to retrieve the scriptures from lds.org whereby it could parse the existing xhtml content as it currently stands but the details of how the parsing is being done being encapsulated from the rest of the application. This would allow for a parsing solution in the interim. When a machine parsable portal is available it would be a matter of swapping out the parser with a different parser but as far as the rest of the application the encapsulation would insulate the rest of the application from the change. As long as both parsers expose the same interface to the rest of the application and behave in the same way when using them it would just be "old parser drops out and new parser drops in".
I don't really think a parser is needed at all. Perhaps if we as the Community needed a temporary solution, but the Church doesn't need to write a parser of XHTML. They have the content in databases. They just need to expose it via web service to us. As Tom mentioned, there is a project planned, but it is in the stages of figuring out copyright and other legal issues.