links: Adding robots.txt, security.txt and some robots meta headers to help with the crawling activity. v1 APPLIED

Peter Sanchez: 1
 Adding robots.txt, security.txt and some robots meta headers to help with the crawling activity.

 3 files changed, 22 insertions(+), 0 deletions(-)

[PATCH links] Adding robots.txt, security.txt and some robots meta headers to help with the crawling activity. Export this patch

---
 static/robots.txt        |  8 ++++++++
 static/security.txt      | 11 +++++++++++
 templates/link_list.html |  3 +++
 3 files changed, 22 insertions(+)
 create mode 100644 static/robots.txt
 create mode 100644 static/security.txt

diff --git a/static/robots.txt b/static/robots.txt
new file mode 100644
index 0000000..c4750a2
--- /dev/null
+++ b/static/robots.txt
@@ -0,0 +1,8 @@
+User-agent: *
+Disallow: /
+Allow: /$
+Allow: /blog
+Allow: /blog/
+Allow: /pricing
+Allow: /docs/
+
diff --git a/static/security.txt b/static/security.txt
new file mode 100644
index 0000000..69c3bec
--- /dev/null
+++ b/static/security.txt
@@ -0,0 +1,11 @@
+Contact: peter@netlandish.com
+Encryption (PGP/GnuPG): 90956225167A28F5A0B461A2838FFCD2312235D1
+Encryption Key URL: https://petersanchez.com/publickey.txt
+Alternative email: ~netlandish/security@lists.code.netlandish.com (Admin only,
+private mailing list. You can send encrypted email there as well using the key
+above to encrypt the contents)
+
+SPF/DMARC: Please do not submit reports related to our SPF or DMARC DNS
+entries. We will not pay bounties for these reports. Our entries are set
+correctly for our current needs.
+
diff --git a/templates/link_list.html b/templates/link_list.html
index 838a3eb..a019bbd 100644
--- a/templates/link_list.html
+++ b/templates/link_list.html
@@ -1,5 +1,8 @@
 {{template "base" .}}
 {{ define "title" }}{{ .title }}{{ end }}
+{{ define "extrahead" }}
+<meta name="robots" content="noindex, follow">
+{{ end }}
 {{ if (not .bookmarkListing) }}
 <form method="GET"
     action="{{ if .isUserDomain }}/{{ else }}{{if .isPopular}}{{reverse "core:popular_link_list"}}{{else if .isOrgLink}}{{.currURL}}{{else}}{{reverse "core:recent_link_list"}}{{end}}{{ end }}{{if .hasUnreadFilter}}?filter=unread{{else if .hasStarredFilter}}?filter=starred{{end}}"
-- 
2.47.2

Peter Sanchez <peter@netlandish.com>

14 days ago

Applied.

To git@git.code.netlandish.com:~netlandish/links
   0835e6f..35f49d6  master -> master