Browse aymeric's, an independent, searchable archive of readable snapshots from followed feeds, preserved in Git with aggr. https://aggr.aymericbeaumet.com/sources/kennyqin.com/ 2026-10-08T07:59:30+00:00 aymeric's aggr urn:aggr:item:2d40dc34dc6acba9c870a9219204affab4e774af 2026-10-08T07:59:30+00:00 2026-10-08T07:59:30+00:00 kennyqin.com via brn

Comments

<p>I don't remember exactly when but, earlier in the year, I decided I didn't want this website to be indexed by search engines anymore. A lot of it has to do with AI and the relentless scraping that plagues the internet now. I know that as long as my blog is accessible by the public internet, there are no real measures I can put in to stop it but at least some of it will be respected.</p> <p>I started out by setting a blanket disallow in the robots.txt.</p> <pre><code><span class="code-snippet">User-agent: * Disallow: / </span></code></pre> <p>When I realised that didn't do anything after a few weeks of waiting and checking, I set the <code>robots</code> meta tag.</p> <pre><code><span class="code-snippet">&lt;meta name=&quot;robots&quot; content=&quot;noindex, nofollow&quot; /&gt; </span></code></pre> <p>I thought that would do the trick. What I didn't realise is that you have to actually, ironically, <strong>allow</strong> the bot that will do the deindexing to visit your website so that it can see the meta tag that tells it to not index your page anymore.</p> <pre><code><span class="code-snippet">User-agent: Googlebot Allow: / User-agent: * Disallow: / </span></code></pre> <p>I also did this with several other common bots and removed them from the robots.txt whenever I found that my website had been deindexed.</p> <p>It's been at least 3 months since I did my last change to have Google deindex me and, after checking today, I'm happy to announce that this website is free from Google! 🥳</p>