It’s been a while since I’ve made changes to the backend of this site.
Given my retirement from web design, some features and functionality I’ve added to the site are no longer needed or a priority.
So I’m taking steps to streamline.
The updates I’ve made this week won’t affect the look and feel of the site, but I’m planning layout changes in the next month or so.
Delete Statistics Plugins
For the first time in 10+ years, I have no statistic plugin on this website.
I’ve deactivated and removed Koko Analytics and Jetpack Stats from the site.
In the past, I would check my statistics regularly to find out what content was most popular and what sites were referring their visitors to my site.
While my web hosting provides general site statistics, it’s not something I’ll be spending time on anymore.
Restrict Content-Scraping Bots
The second change I’ve made, and one you may want to consider for your own site, is blocking the bots used by artificial intelligence (AI) companies.
Companies are using bots to scrape content from your site to train their large language models (LLM).
Like many other website owners, I haven’t given those companies consent to scrape my content.
When I learned about LLMs earlier this year, I took steps to update my robots.txt file.
I used the free version of All in One SEO plugin to block the ChatGPT-User bot from this site.
But since then, more bots have been released.
Thanks to Jeremy Keith for his crawlers post which led me to Neil Clarke’s post explaining how to block the bots.
Clarke provides detailed instructions for blocking the bots whether you use:
- robots.txt
- .htaccess
- Firewall or Content Delivery Network (CDN)
In addition, he offers information and suggestions for folks who use:
- WordPress
- Squarespace
- Wix
Thank you, Neil. Appreciate the info. And glad you’ve updated your post in the past week with additional bots to block.
If you have additional information or steps for blocking the bots, share your info in the comments.
