erik/mastodon
1
0
Fork 0
mirror of https://github.com/mastodon/mastodon.git synced 2024-08-20 21:08:15 -07:00
Code Issues Projects Releases Wiki Activity

Update robots.txt

Browse source
This commit is contained in:
Shlee 2024-08-01 19:39:19 +09:30 committed by GitHub
parent 3f0f833a78
commit 5e041ab455
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
public/robots.txt
View file

@ -6,7 +6,7 @@ Disallow: /interact/
## AI Bots - Research from darkvisitors.com and multiple other sources.
# AI Data Scraper/Crawler/Assistant - https://platform.openai.com/docs/bots
# OpenAI Scraper/Crawler/Assistant - https://platform.openai.com/docs/bots
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot