Publishers Aim to Kill Their Business (Title slightly rewritten from the original.)
Actually, it’s a bit more complex than this. Reading between the lines, I believe that what’s going on is that these publishers want to allow search engines to crawl and index their content, but NOT allow search engines to make use of snippets of that content or aggregate the information.
Kind of a situation of wanting to have their cake and eat it too, as I see it.
In addition, the article states:
In one example of how ACAP would work, a newspaper publisher could grant search engines permission to index its site, but specify that only select ones display articles for a limited time after paying a royalty.
Although this is a poorly written sentence, it appears to say that this program would allow publishers to control what search engines were allowed to access their content when, in addition to requiring royalties be paid by those search engines.
Andy Beal describes this as publishers building a new robots.txt file, but I think there’s really a bit more to it. The publishers still see that search engines are a great way to get users to their information. They don’t want their content removed, they simply want more control over how it’s used.
The article also says:
What is required is a standardized way of describing the permissions which apply to a Web site or Web page so that it can be decoded by a dumb machine without the help of an expensive lawyer.
A robots.txt file does meet this description. However, a robots.txt file is purely a binary decision maker: either you can come in, or you can’t. This project, I suspect, is designed to build a more complex permissions system, allowing robots to visit at some times but not others, to view articles provided certain criteria are met, etc. None of this couldn’t be done by combining a robots.txt file with some sort of server-side decision making, but why not centralize the logic?
This isn’t to say that I think it’s a good idea. I’m pretty confident that there are much simpler ways to accomplish their goals – starting with effective monetization of their own sites. But it has to be acknowledged that the publishing industry has been undergoing a groundswell of change over the last decade. Access to information has changed radically during this time, and more traditional publishing companies are simply trying to keep their business models alive.