+1 Button Overrides robots.txt

Posted in
We got a word of confirmation from John Mueller, a Google employee, that Google +1 button will enable pages to being seen or getting indexed even if those pages are blocked in "robots.txt" file.
...if you put a +1 button on your pages, we assume that you're ok with us accessing the pages, even if they're blocked by a robots.txt file. In general, users don't know about robots.txt files, so they may +1 or reshare any URL if they find it interesting enough... [reference]
He also added that if we intend to block unverified users from re-sharing and accessing the private content we should use server-side authentication.. Because of this behaviour, it is important that we use proper canonicalization (again, don't use the robots.txt for canonicalization as advised). More information about +1 button here.
Comments: (0)

Speed Up The Indexing Of Your Newly Added Pages

Posted in ,
If you have just updated added a new page to your site, you can request Google to crawl it immediately without waiting for Google to discover it naturally.

Click To Enlarge
  1. On the Google Webmaster Tools page, choose the site you like.
  2. Under Diagnostics in the dashboard, choose Fetch as Googlebot.
  3. On the text box, fill-up the complete URL of the page you want to submit.
  4. On the dropdown menu, choose Web. (select cHTML (for Japanese mobile sites) or xHTML/WML for the standard mobile sites.
  5. Click Fetch. Google will fetch the URL you requested. It may take up to 10 or 15 minutes for Fetch status to be updated.
  6. Once you see a Fetch status of "Successful", click Submit to Index, and then click one of the following:
    7. Click To Enlarge
    • To submit the individual URL to Google's index, select URL and click Submit. You can submit up to 50 URLs a week in this way.
    • To submit the URL and all pages linked from it, click URL and all linked pages. You can submit up to 10 of these requests a month.
Note:
  • Fetch as Googlebot submission will not follow redirects.
  • Google doesn't guarantee that they will index every URL they crawl.
To ask Google to crawl a URL on a site you don't own, visit http://www.google.com/webmasters/tools/submit-url.
Comments: (3)