Complete re-write
Table of Contents
Over the years I have tried drupal, wordpress and more recently hugo + ox-hugo and in that category have tried a few different themes, but often came up against an issue after some time. Perhaps a hugo update had broken certain aspects of the theme or to get it to do what I wanted was time consuming so this time no messing, no fancy export requirements just plain emacs, publish and orgmode.
It has to be stated I would not be here without refering to Dan Liden's guide and github info in fact most of this is a copy of what is there but I have modified certain aspects and more recently ran it through chatGPT to only publish if a file has changed which along with a couple of other tweaks seriously increased the file size.
No preview text use #+DESCRIPTION instead
In Dan's site he uses a function to get and show a preview of the page under the heading, he does this with text encapsulated with the #+PREVIEW: construct using this function
1: ;; org-site/build-site.el
2: (defun my/get-preview (file)
3: "get preview text from a file
4:
5: Uses the function here as a starting point:
6: https://ogbe.net/blog/blogging_with_org.html"
7: (with-temp-buffer
8: (insert-file-contents file)
9: (goto-char (point-min))
10: (when (re-search-forward "^#\\+BEGIN_PREVIEW$" nil 1)
11: (goto-char (point-min))
12: (let ((beg (+ 1 (re-search-forward "^#\\+BEGIN_PREVIEW$" nil 1)))
13: (end (progn (re-search-forward "^#\\+END_PREVIEW$" nil 1)
14: (match-beginning 0))))
15: (buffer-substring beg end)))))
I decided I preferred having this not within a separate entity in the org file but using the ~#+DESCRIPTION:~ part of the headers ie
TITLE: Complete re-write
DESCRIPTION: Major refactor of website content into just an emacs orgmode entity with no hugo
KEYWORDS: ramblings orgmode emacs
DATE:
So to achieve this the function needed re-writing;
(defun my/org-publish-org-sitemap-format (entry style project)
"Format sitemap entries using #+DESCRIPTION."
(cond
((not (directory-name-p entry))
(let* ((full-path (expand-file-name entry "./content"))
(preview (or (my/org-get-description full-path)
"(No description)")))
(format "[[file:%s][(%s) %s]]\n%s"
entry
(format-time-string "%Y-%m-%d"
(org-publish-find-date entry project))
(org-publish-find-title entry project)
preview)))
((eq style 'tree)
(file-name-nondirectory
(directory-file-name entry)))
(t entry)))
It works well and I keep a simple snippet to generate the required headers when creating a new post.
Other org-publish references
chatGPT rewrite
decided to try chatGPT on the script as i wanted to only publish changed files not everything each time. It took a couple attempts working through different script iterations and prompting chatgpt with returned errors etc but it seems to have worked well.
;; BUILD SCRIPT START
;; │
;; ▼
;; Find all content/*.org
;; │
;; ▼
;; Ignore sitemap.org/drafts
;; │
;; ▼
;; Compare .org → corresponding .html
;; │
;; ┌──────────┴──────────┐
;; │ │
;; unchanged changed
;; │ │
;; ▼ ▼
;; do nothing org-publish
;; │
;; ▼
;; sitemap
;; │
;; ▼
;; RSS
The full code is below and you can see it is quite a long script
;;; org-website-publish.el --- Build script -*- lexical-binding: t; -*-
(require 'package)
;; ------------------------------------------------------------
;; Package configuration
;; ------------------------------------------------------------
(setq package-user-dir
(expand-file-name "./.packages"))
(setq package-archives
'(("gnu" . "https://elpa.gnu.org/packages/")
("melpa" . "https://melpa.org/packages/")))
;; Prefer GNU ELPA for Org and other packages available from
;; both archives.
(setq package-archive-priorities
'(("gnu" . 20)
("melpa" . 10)))
;; Do not activate a previously installed Org 9.5 package here.
;; Its startup code emits the obsolete Org-ELPA closure warning
;; before we have had a chance to install the GNU ELPA version.
(let ((package-load-list
'(all
(org nil))))
(package-initialize))
;; Refresh on every build so the Org descriptor is current and
;; definitely comes from the archives configured above, rather
;; than from stale Org-ELPA metadata cached by an older script.
(message "Refreshing GNU ELPA and MELPA package archives")
(condition-case err
(package-refresh-contents)
(error
(message "Package refresh failed: %s" err)))
(let* ((org-entry
(assq 'org package-archive-contents))
(org-desc
(cond
((and org-entry
(package-desc-p (cdr org-entry)))
(cdr org-entry))
((and org-entry
(package-desc-p (cadr org-entry)))
(cadr org-entry)))))
(if (or (not org-desc)
(not (equal
(package-desc-archive org-desc)
"gnu")))
;; A network or archive failure should not prevent the site
;; from being built. Continue with the newest Org already
;; available to this Emacs invocation.
(message
"Org was not found in GNU ELPA metadata; using the installed/bundled version")
;; A bundled Org satisfies a plain `package-installed-p' check.
;; Compare against the archive version explicitly so the newest
;; available Org package is installed when necessary.
(unless (package-installed-p
'org
(package-desc-version org-desc))
(let ((package-install-upgrade-built-in t))
(package-install org-desc)))))
;; Org was deliberately excluded from the initial activation.
;; Activate the newest installed Org now that the GNU ELPA
;; installation/upgrade step has finished.
(package-activate 'org)
(dolist (pkg '(htmlize ess ox-rss webfeeder esxml))
(unless (package-installed-p pkg)
(package-install pkg)))
;; ------------------------------------------------------------
;; Required packages
;; ------------------------------------------------------------
(require 'org)
(message "Using Org version %s" (org-version))
(require 'ox-publish)
(require 'ox-rss)
(require 'webfeeder)
(require 'esxml)
(require 'json)
(require 'seq)
(require 'subr-x)
;; ------------------------------------------------------------
;; Site constants
;; ------------------------------------------------------------
(defconst my/org-site-base-url
"https://leehalls.net")
(defconst my/org-site-root
(file-name-directory
(file-truename
(or load-file-name
default-directory))))
;; ------------------------------------------------------------
;; Utility functions
;; ------------------------------------------------------------
(defun my/org-site--format-iso8601 (time)
"Format TIME as an ISO-8601 UTC timestamp."
(when time
(format-time-string
"%Y-%m-%dT%H:%M:%SZ"
time
t)))
(defun my/org-site--collect-keyword (name)
"Return the first Org keyword named NAME from the current buffer."
(let ((val
(cdr
(assoc-string
name
(org-collect-keywords (list name))
t))))
(when val
(car val))))
(defun my/org-site--escape-html (s)
"Escape the HTML-sensitive characters in S."
(when s
(replace-regexp-in-string
"[<>&\"]"
(lambda (c)
(pcase c
("<" "<")
(">" ">")
("&" "&")
("\"" """)))
s
t
t)))
(defun my/org-site--truncate (s n)
"Truncate S to N characters."
(when s
(if (> (length s) n)
(concat (substring s 0 n) "…")
s)))
(defun my/org-site--abs-url (path)
"Return PATH as an absolute URL."
(when path
(if (string-prefix-p "http" path)
path
(concat
my/org-site-base-url
"/"
(string-remove-prefix "/" path)))))
(defun my/org-site--file-contents (file)
"Return the complete contents of FILE."
(with-temp-buffer
(insert-file-contents file)
(buffer-string)))
;; ------------------------------------------------------------
;; Additional HTML export settings
;; ------------------------------------------------------------
(setq org-html-validation-link nil
org-html-htmlize-output-type 'css
org-html-style-default
(my/org-site--file-contents
(expand-file-name
"assets/head.html"
my/org-site-root)))
;; ------------------------------------------------------------
;; DESCRIPTION-based preview
;;
;; DESCRIPTION is the single source of truth for sitemap
;; previews and page metadata.
;; ------------------------------------------------------------
(defun my/org-get-description (file)
"Return the single-line #+DESCRIPTION from FILE, or nil."
(when (and file
(file-exists-p file))
(with-temp-buffer
(insert-file-contents file)
(goto-char (point-min))
(when (re-search-forward
"^#\\+DESCRIPTION:[ \t]*\\(.*\\)$"
nil
t)
(string-trim
(match-string 1))))))
;; ------------------------------------------------------------
;; SEO metadata injection
;; ------------------------------------------------------------
(defun my/org-site--insert-head-extra (lines)
"Insert LINES as #+HTML_HEAD_EXTRA keywords."
(when lines
(save-excursion
(goto-char (point-min))
;; Move past the initial Org keyword block.
(while (looking-at "^#\\+")
(forward-line 1))
(dolist (line lines)
(insert
(format
"#+HTML_HEAD_EXTRA: %s\n"
line))))))
(defun my/org-site--build-jsonld
(title description canonical published modified keywords image)
"Build JSON-LD metadata for a page."
(let ((obj
`(("context" . "https://schema.org")
("@type" . ,(if published
"BlogPosting"
"WebPage"))
("headline" . ,title)
("description" . ,description)
("mainEntityOfPage"
. (("@type" . "WebPage")
("@id" . ,canonical)))
("author"
. (("@type" . "Person")
("name" . "Lee Halls"))))))
(when published
(push
`("datePublished" . ,published)
obj))
(when modified
(push
`("dateModified" . ,modified)
obj))
(when keywords
(push
`("keywords" . ,keywords)
obj))
(when image
(push
`("image" . ,image)
obj))
(json-encode obj)))
(defun my/org-site--add-page-metadata (backend)
"Add SEO metadata to HTML exports."
(when (org-export-derived-backend-p
backend
'html)
(let* ((info
(org-export-get-environment backend))
(source
(plist-get info :input-file))
(title
(org-element-interpret-data
(plist-get info :title)))
(desc
(or
(my/org-site--collect-keyword
"DESCRIPTION")
(and source
(my/org-get-description
source))))
(desc
(my/org-site--truncate
desc
200))
(canonical
(my/org-site--collect-keyword
"CANONICAL_URL"))
;; We deliberately guard this because date lookup
;; can involve the publishing project/cache.
;; This hook only runs during an actual export,
;; after org-publish has established its context.
(published
(my/org-site--format-iso8601
(ignore-errors
(when source
(org-publish-find-date
source
(org-publish-get-project-from-filename
source
org-publish-project-alist))))))
(meta-lines
(delq
nil
(list
(when canonical
(format
"<link rel=\"canonical\" href=\"%s\">"
(my/org-site--escape-html
canonical)))
(when desc
(format
"<meta name=\"description\" content=\"%s\">"
(my/org-site--escape-html
desc))))))
(my/org-site--insert-head-extra
meta-lines)))))
(add-hook
'org-export-before-processing-hook
#'my/org-site--add-page-metadata)
;; ------------------------------------------------------------
;; Sitemap helpers
;; ------------------------------------------------------------
(defun my/remove-docs-from-sitemap (tree)
"Remove the DOCS branch from sitemap TREE."
(cond
((and (listp tree)
(equal (car tree) "docs"))
nil)
((listp tree)
(delq
nil
(mapcar
#'my/remove-docs-from-sitemap
tree)))
(t
tree)))
(defun my/org-publish-org-sitemap (title list)
"Generate the Org sitemap."
(concat
"#+OPTIONS: toc:nil\n"
(org-list-to-subtree
(my/remove-docs-from-sitemap
list))))
(defun my/org-publish-org-sitemap-format
(entry style project)
"Format sitemap ENTRY using its #+DESCRIPTION."
(message "SITEMAP ENTRY: %S" entry)
(cond
;; ---------------------------------------------------------
;; File entry
;; ---------------------------------------------------------
((not (directory-name-p entry))
(let* ((full-path
(expand-file-name
entry
"./content"))
(preview
(or
(my/org-get-description
full-path)
"(No description)"))
;; These functions are called only while Org Publish
;; is actually generating the sitemap, so its cache
;; has already been initialised.
(date
(ignore-errors
(org-publish-find-date
entry
project)))
(title
(ignore-errors
(org-publish-find-title
entry
project))))
(format
"[[file:%s][(%s) %s]]\n%s"
entry
(if date
(format-time-string
"%Y-%m-%d"
date)
"")
(or title
(file-name-base entry))
preview)))
;; ---------------------------------------------------------
;; Directory in tree style
;; ---------------------------------------------------------
((eq style 'tree)
(file-name-nondirectory
(directory-file-name
entry)))
;; ---------------------------------------------------------
;; Everything else
;; ---------------------------------------------------------
(t
entry)))
;; ------------------------------------------------------------
;; RSS feed generation
;; ------------------------------------------------------------
(defun my/org-site-generate-rss ()
"Generate an Atom/RSS feed from published HTML files."
(message "Generating RSS feed...")
(let ((webfeeder-default-author
"Lee Halls"))
(let* ((public-dir
(expand-file-name
"public"
my/org-site-root))
;; WEBFEEDER-BUILD resolves every entry relative to
;; PUBLIC-DIR, so pass relative names such as
;; "docs/about.html", not "./public/docs/about.html".
;; Passing the latter makes it look for
;; "./public/public/docs/about.html".
(html-files
(mapcar
(lambda (file)
(file-relative-name
file
public-dir))
(seq-filter
(lambda (file)
(not
(or
;; Don't include drafts.
(string-match-p
"/posts/drafts/"
file)
;; Don't include assets.
(string-match-p
"/assets/"
file)
;; Don't include sitemap.html.
(string-suffix-p
"/sitemap.html"
file))))
(directory-files-recursively
public-dir
"\\.html$")))))
(message
"RSS: found %d HTML files"
(length html-files))
(webfeeder-build
(expand-file-name
"rss.xml"
public-dir)
public-dir
my/org-site-base-url
html-files
:title
"Ramblings of a lost one"
:description
"Blog posts and updates from Lee Halls."))))
;; ------------------------------------------------------------
;; Org Publish configuration
;; ------------------------------------------------------------
;; Keep Org's timestamp/cache data outside the published site.
;;
;; NOTE:
;; Our custom pre-flight test below does NOT call
;; org-publish-needed-p, so it does not require this cache to
;; exist before the first invocation of org-publish.
;;
;; Org Publish itself will initialise and use this directory
;; normally when publishing.
(setq org-publish-timestamp-directory
(expand-file-name
".org-timestamps/"
my/org-site-root))
(setq org-publish-project-alist
(list
(list
"my-org-site"
;; -----------------------------------------------------
;; Source
;; -----------------------------------------------------
:recursive
t
:base-directory
(expand-file-name
"content"
my/org-site-root)
;; Don't publish drafts.
:exclude
"posts/drafts/.*"
;; -----------------------------------------------------
;; Destination
;; -----------------------------------------------------
:publishing-directory
(expand-file-name
"public"
my/org-site-root)
:publishing-function
'org-html-publish-to-html
;; -----------------------------------------------------
;; HTML
;; -----------------------------------------------------
:html-preamble
(my/org-site--file-contents
(expand-file-name
"assets/html_preamble.html"
my/org-site-root))
:with-author
nil
:with-creator
t
:with-toc
t
:section-numbers
nil
:time-stamp-file
nil
;; -----------------------------------------------------
;; Sitemap
;; -----------------------------------------------------
;; Keep this value stable between runs. Changing it at
;; runtime invalidates Org's project cache and causes all
;; source files to be republished.
:auto-sitemap
t
:sitemap-filename
"sitemap.org"
:sitemap-title
"Ramblings of a lost one"
:sitemap-sort-files
'anti-chronologically
:sitemap-function
'my/org-publish-org-sitemap
:sitemap-format-entry
'my/org-publish-org-sitemap-format)))
;; ------------------------------------------------------------
;; File modification helpers
;; ------------------------------------------------------------
(defun my/org-site--file-mtime (file)
"Return FILE's modification time."
(when (and file
(file-exists-p file))
(file-attribute-modification-time
(file-attributes file))))
(defun my/org-site--output-file-for
(source-file base-dir publishing-dir)
"Return the HTML output path for SOURCE-FILE."
(let ((relative-file
(file-relative-name
source-file
base-dir)))
(expand-file-name
(concat
(file-name-sans-extension
relative-file)
".html")
publishing-dir)))
(defun my/org-site--file-needs-publishing-p
(source-file output-file)
"Return non-nil if SOURCE-FILE needs publishing.
This function intentionally does NOT use Org Publish's internal
cache. It only compares source and output modification times."
(cond
;; Source doesn't exist.
((not (file-exists-p source-file))
nil)
;; Output doesn't exist.
((not (file-exists-p output-file))
t)
;; Source is newer than output.
(t
(let ((source-time
(my/org-site--file-mtime
source-file))
(output-time
(my/org-site--file-mtime
output-file)))
(and source-time
output-time
(time-less-p
output-time
source-time))))))
;; ------------------------------------------------------------
;; Pre-flight change detection
;;
;; IMPORTANT:
;;
;; Do NOT use org-publish-needed-p here.
;;
;; org-publish-needed-p depends on Org Publish's internal
;; org-publish-cache having already been initialised.
;;
;; This function runs BEFORE org-publish, so using the cache
;; here causes:
;;
;; "org-publish-cache-file-needs-publishing called,
;; but no cache present"
;;
;; Instead, we perform a simple source/output timestamp check.
;; ------------------------------------------------------------
(defun my/org-site-files-needing-publish
(project-name)
"Return source files in PROJECT-NAME that need publishing.
This pre-flight check deliberately avoids Org Publish's
internal cache. Org Publish is responsible for its cache
during the actual publishing phase."
(let* ((project
(assoc
project-name
org-publish-project-alist))
(properties
(cdr project))
(base-dir
(expand-file-name
(plist-get
properties
:base-directory)))
(publishing-dir
(expand-file-name
(plist-get
properties
:publishing-directory)))
(exclude
(plist-get
properties
:exclude))
;; Only consider Org files.
(files
(directory-files-recursively
base-dir
"\\.org$"))
(files-to-publish
nil))
;; If the project doesn't exist, fail loudly rather than
;; silently deciding that nothing needs publishing.
(unless project
(error
"Org Publish project not found: %s"
project-name))
(dolist
(file files (nreverse files-to-publish))
;; -------------------------------------------------------
;; sitemap.org is generated by Org Publish.
;;
;; It should never cause the pre-flight check to decide
;; that the site needs rebuilding.
;; -------------------------------------------------------
(unless
(string-suffix-p
"sitemap.org"
file)
;; -----------------------------------------------------
;; Respect the project's :exclude expression.
;; -----------------------------------------------------
(unless
(and exclude
(string-match-p
exclude
file))
(let
((output-file
(my/org-site--output-file-for
file
base-dir
publishing-dir)))
(when
(my/org-site--file-needs-publishing-p
file
output-file)
(message
"Publish required: %s"
(file-relative-name
file
base-dir))
(push file
files-to-publish))))))))
;; ------------------------------------------------------------
;; Publishing
;; ------------------------------------------------------------
(defun my/org-site-publish ()
"Publish the site if source files have changed.
This is the main entry point used by the build script."
(let* ((project-name
"my-org-site")
(project
(assoc project-name
org-publish-project-alist))
(changed-files
(my/org-site-files-needing-publish
project-name)))
(if
changed-files
(progn
(message
"Changes detected in content files.")
;; Initialise Org's cache for title/date lookups used by
;; sitemap generation, but do not ask `org-publish' to
;; iterate over the whole project.
(org-publish-initialize-cache
project-name)
(let* ((sitemap-name
(or (org-publish-property
:sitemap-filename
project)
"sitemap.org"))
(sitemap-file
(expand-file-name
sitemap-name
(org-publish-property
:base-directory
project))))
;; Regenerate and publish the sitemap, then publish
;; only the exact source files found by the pre-flight
;; modification-time check.
(org-publish-sitemap
project
sitemap-name)
(dolist (file
(append changed-files
(list sitemap-file)))
(message
"Publishing selected file: %s"
(file-relative-name
file
(org-publish-property
:base-directory
project)))
(org-publish-file
file
project
t)))
;; ---------------------------------------------------
;; Generate RSS after publishing has completed.
;; ---------------------------------------------------
(my/org-site-generate-rss)
(message
"Org site publishing complete."))
;; -------------------------------------------------------
;; Nothing changed.
;; -------------------------------------------------------
(message
"No content changes detected. "
"Skipping publishing, sitemap and RSS generation."))))
;; ------------------------------------------------------------
;; Main entry point
;; ------------------------------------------------------------
(my/org-site-publish)
;;; org-website-publish.el ends here
;; BUILD SCRIPT START
;; │
;; ▼
;; Find all content/*.org
;; │
;; ▼
;; Ignore sitemap.org/drafts
;; │
;; ▼
;; Compare .org → corresponding .html
;; │
;; ┌──────────┴──────────┐
;; │ │
;; unchanged changed
;; │ │
;; ▼ ▼
;; do nothing org-publish
;; │
;; ▼
;; Org creates/uses
;; its own cache
;; │
;; ▼
;; sitemap
;; │
;; ▼
;; RSS