Browse Source

Add robots.txt for web robots

Allow all robots to access any part of the site. This may change in the
future, as the html snippets that are generated for the posts and pages
should not be directly indexed.

See: http://www.robotstxt.org/robotstxt.html

Signed-off-by: Collin J. Doering <collin.doering@rekahsoft.ca>
pull/2/head
Collin J. Doering 6 years ago
parent
commit
92acb5344e
2 changed files with 3 additions and 1 deletions
  1. +2
    -0
      robots.txt
  2. +1
    -1
      src/site.hs

+ 2
- 0
robots.txt View File

@ -0,0 +1,2 @@
User-agent: *
Allow: /

+ 1
- 1
src/site.hs View File

@ -105,7 +105,7 @@ main = do
hakyllWith myConfig $ do
-- All Versions ------------------------------------------------------------------------------------------
match ("action/**" .||. "files/**" .||. "images/**" .||. "fonts/**") $ do
match ("action/**" .||. "files/**" .||. "images/**" .||. "fonts/**" .||. "robots.txt") $ do
route idRoute
compile copyFileCompiler


Loading…
Cancel
Save