violet/diyhrt
1
0
Fork 0
mirror of https://github.com/soapingtime/diyhrt.git synced 2026-11-02 23:44:51 +01:00
Code Issues Packages Projects Releases Wiki Activity

re-initialize and make some changes. repository got to around 2 gigabytes and needed to be more efficient

Browse Source
This commit is contained in:
soap
2023-09-29 12:56:26 -05:00
commit 234e4592fa
1441 changed files with 476627 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crawl_contact.txt
+2
View File
@@ -0,0 +1,2 @@
if you're a webmaster who wants to contact me for any reason - open an issue github.com/boytwink/diyhrt/issues and give it the webmaster label. there's also a couple ways to contact me on my github profile.
i do not and will not respect robots.txt. if you want me to not crawl a certain page or use different rate limits, read this https://wiki.archiveteam.org/index.php/Robots.txt, then ask me.