Building Emacs from Source on Windows (Once Upon a Time)
It was the 22nd of May (this year). I was desperate for all the improvements of Emacs 29, but the latest official release was 28.2, which I was using on my Windows desktop. I decided to compile my own binary from the source, guided by a great Reddit post from tuhdo, which turned out to be quite easy. I put the script on GitHub.
When I started the new Emacs, I encountered an error:
OutputSymbol’s value as variable is void: comp-deferred-compilation-deny-list
This is apparently due to variables being renamed, but I don’t know what the correct name is or how to fix it. There’s a similar straight.el bug from 2021. I ultimately followed a bit of advice I saw somewhere on GitHub and added this before the straight.el bootstrapping code in my configuration:
Emacs Lisp(or (boundp 'native-comp-deferred-compilation-deny-list)
(setq native-comp-deferred-compilation-deny-list '()))
That made the error vanish.
I expected this installation to automatically fetch the newest versions of all packages thanks to
straight.el pulling from Git repositories, but I was wrong. For no apparent reason, most packages
were quite out-of-date. (Come to think of it, perhaps it was fetching older versions recorded in my
lockfile, though I hadn’t explicitly used straight-thaw-versions. I’ve never really considered how
the lockfile affects the normal installation of packages.) I had to update individual packages as
they threw errors at me.
A related Windows issue is the constant Too many open files errors: I have straight.el set to load packages on demand, but restoring my Emacs desktop meant demanding and building many packages, reliably triggering said error. I had to restart without the saved session, trigger rebuilds for all the packages I could think of one by one, and then restart normally, so that there was less to do at once.
With that out of the way and Emacs functioning correctly, it’s not a night and day difference, but I can definitely see a change. Emacs 30 from Git feels snappy compared to before even though things like Git itself remain fundamentally slow on Windows.
Sadly, I’ve been unable to repeat my success after reinstalling Windows last
month. I installed MSYS2 via Scoop and ran the same steps
under MinGW64 but keep getting an error about mismatched sizes deep inside some Emacs Lisp that
seems to deal with hash tables. I can’t build older versions of the source either, or the branch for
version 29, so it doesn’t appear to have been caused by upstream changes. I tried removing all
CFLAGS and tried running under both MinGW32 and UCRT64, but all of it produced the same result. I
tried the exact ./configure line from the Reddit example and the exact ./configure line from a
different
guide
(linked in the Reddit post), none of which helped. Here’s part of the end of the output, for the
record:
OutputLoading d:/Media/src/my/compile-emacs/source/lisp/leim/leim-list.el (source)... Loading emacs-lisp/rmc... Finding pointers to doc strings... Finding pointers to doc strings...done Dumping under the name bootstrap-emacs.pdmp Dumping fingerprint: a176a0c43f8c543e43040781a96a5710a183540a0b64fefc38ef29ab83c2d55b Dump complete Byte counts: header=100 hot=15460836 discardable=206568 cold=9165600 Reloc counts: hot=1019523 discardable=5643 ANCIENT=yes make -C ../lisp compile-first EMACS="../src/bootstrap-emacs.exe" make[2]: Entering directory '/d/Media/src/my/compile-emacs/source/lisp' ELC emacs-lisp/macroexp.elc Source file `d:/Media/src/my/compile-emacs/source/lisp/emacs-lisp/bytecomp.el' newer than byte-compiled file; using older file Source file `d:/Media/src/my/compile-emacs/source/lisp/help-mode.el' newer than byte-compiled file; using older file Error: args-out-of-range (#s(comp-cstr-ctxt #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ())) 7) Source file `d:/Media/src/my/compile-emacs/source/lisp/emacs-lisp/cl-print.el' newer than byte-compiled file; using older file mapbacktrace(#f(compiled-function (evald func args flags) #<bytecode -0xd212fdc699118c2>)) debug-early-backtrace() debug-early(error (args-out-of-range #s(comp-cstr-ctxt :typeof-types #<hash-table equal 0/65 0x4cf5b0ded5> :pred-type-h #<hash-table equal 0/65 0x4cf5b0dde1> :union-typesets-mem #<hash-table equal 0/65 0x4cf5bb1849> :common-supertype-mem #<hash-table equal 0/65 0x4cf5b0c021> :subtype-p-mem #<hash-table equal 0/65 0x4cf5b0f2c5> :union-1-mem-no-range #<hash-table equal 0/65 0x4cf5b0f45d> :union-1-mem-range cl-prin1: (args-out-of-range #s(comp-cstr-ctxt #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ())) 7) )
…and several more pages until it gets to:
Output(let ((old-face-font-rescale-alist face-font-rescale-alist)) (unwind-protect (command-line) (if default-directory (setq default-directory (abbreviate-file-name default-directory))) (or auto-save-list-file-name (and auto-save-list-file-prefix (setq auto-save-list-file-name (cond ((eq system-type 'ms-dos) (make-directory (file-name-directory auto-save-list-file-prefix) t) (concat (make-temp-name (expand-file-name auto-save-list-file-prefix)) "~")) (t (expand-file-name (format "%s%d-%s~" auto-save-list-file-prefix (emacs-pid) (system-name)))))))) (if inhibit-startup-hooks nil (run-hooks 'emacs-startup-hook 'term-setup-hook)) (if (or frame-initial-frame (not (and initial-window-system (not noninteractive) (not (eq initial-window-system 'pc))))) (progn (if (and (display-multi-font-p) (not (eq face-font-rescale-alist old-face-font-rescale-alist)) (assoc (font-xlfd-name (face-attribute 'default :font)) face-font-rescale-alist #'string-match-p)) (progn (set-face-attribute 'default nil :font (font-spec)))) (if (fboundp 'frame-notice-user-settings) (frame-notice-user-settings)) (if (fboundp 'frame-set-background-mode) (frame-set-background-mode (selected-frame))))) (if (fboundp 'font-menu-add-default) (font-menu-add-default)) (if inhibit-startup-hooks nil (run-hooks 'window-setup-hook)))) (if command-line-processed (message internal--top-level-message) (setq command-line-processed t) (setq startup--xdg-config-home-emacs (let ((xdg-config-home (getenv-internal "XDG_CONFIG_HOME"))) (if xdg-config-home (concat xdg-config-home "/emacs/") startup--xdg-config-default))) (setq user-emacs-directory (startup--xdg-or-homedot startup--xdg-config-home-emacs nil)) (if (featurep 'native-compile) (progn (if (native-comp-available-p) nil (progn (setq native-comp-jit-compilation nil) (setq native-comp-enable-subr-trampolines nil))) (let ((path-env (getenv "EMACSNATIVELOADPATH"))) (if path-env (progn (let ((tail (split-string path-env path-separator))) (while tail (let ((path (car tail))) (if (string= "" path) nil (setq native-comp-eln-load-path (cons path native-comp-eln-load-path))) (setq tail (cdr tail)))))))) (setq native-comp-eln-load-path (cons (expand-file-name "eln-cache/" user-emacs-directory) native-comp-eln-load-path)))) (let ((tail load-path) (lispdir (expand-file-name "../lisp" data-directory)) dir) (while tail (setq dir (car tail)) (let ((default-directory dir)) (load (expand-file-name "subdirs.el") t t t)) (or (string-prefix-p lispdir dir) (let ((default-directory dir)) (load (expand-file-name "leim-list.el") t t t))) (setq tail (cdr tail)))) (cond ((memq system-type '(ms-dos windows-nt)) (progn (setq eol-mnemonic-unix "(Unix)") (setq eol-mnemonic-mac "(Mac)"))) (t (progn (setq eol-mnemonic-dos "(DOS)") (setq eol-mnemonic-mac "(Mac)")))) (set-locale-environment nil) (if locale-coding-system (progn (let ((coding (if (eq system-type 'windows-nt) 'utf-8 locale-coding-system))) (save-excursion (let ((tail (buffer-list))) (while tail (let ((elt (car tail))) (set-buffer elt) (if default-directory (setq default-directory (if (eq system-type 'windows-nt) (let ((defdir (decode-coding-string default-directory coding t))) (expand-file-name defdir defdir)) (decode-coding-string default-directory coding t)))) (setq tail (cdr tail)))))) (let ((tail '(load-path exec-path))) (while tail (let ((pathsym (car tail))) (let ((path (symbol-value pathsym))) (if (listp path) (set pathsym (mapcar #'(lambda (dir) (decode-coding-string dir coding t)) path)))) (setq tail (cdr tail))))) (if (featurep 'native-compile) (progn (let ((npath (symbol-value 'native-comp-eln-load-path))) (set 'native-comp-eln-load-path (mapcar #'(lambda (dir) (expand-file-name (decode-coding-string dir coding t))) npath))) (setq startup--original-eln-load-path (copy-sequence native-comp-eln-load-path)))) (let ((tail '(data-directory doc-directory exec-directory installation-directory invocation-directory invocation-name source-directory shared-game-score-directory))) (while tail (let ((filesym (car tail))) (let ((file (symbol-value filesym))) (if (stringp file) (set filesym (decode-coding-string file coding t)))) (setq tail (cdr tail)))))))) (let ((dir default-directory)) (save-current-buffer (set-buffer "*Messages*") (messages-buffer-mode) (setq default-directory (or dir (expand-file-name "~/"))))) (put 'user-full-name 'standard-value (list (default-value 'user-full-name))) (let ((pwd (getenv "PWD"))) (and pwd (or (and default-directory (condition-case nil (progn (equal (file-attributes (file-name-as-directory pwd)) (file-attributes (file-name-as-directory default-directory)))) (error nil))) (setq process-environment (delete (concat "PWD=" pwd) process-environment))))) (if (listp charset-map-path) (let ((coding (if (eq system-type 'windows-nt) 'utf-8 locale-coding-system))) (setq charset-map-path (mapcar #'(lambda (dir) (decode-coding-string dir coding t)) charset-map-path)))) (if default-directory (setq default-directory (abbreviate-file-name default-directory)) (display-warning 'initialization "Error setting default-directory")) (let ((old-face-font-rescale-alist face-font-rescale-alist)) (unwind-protect (command-line) (if default-directory (setq default-directory (abbreviate-file-name default-directory))) (or auto-save-list-file-name (and auto-save-list-file-prefix (setq auto-save-list-file-name (cond ((eq system-type 'ms-dos) (make-directory (file-name-directory auto-save-list-file-prefix) t) (concat (make-temp-name (expand-file-name auto-save-list-file-prefix)) "~")) (t (expand-file-name (format "%s%d-%s~" auto-save-list-file-prefix (emacs-pid) (system-name)))))))) (if inhibit-startup-hooks nil (run-hooks 'emacs-startup-hook 'term-setup-hook)) (if (or frame-initial-frame (not (and initial-window-system (not noninteractive) (not (eq initial-window-system 'pc))))) (progn (if (and (display-multi-font-p) (not (eq face-font-rescale-alist old-face-font-rescale-alist)) (assoc (font-xlfd-name (face-attribute 'default :font)) face-font-rescale-alist #'string-match-p)) (progn (set-face-attribute 'default nil :font (font-spec)))) (if (fboundp 'frame-notice-user-settings) (frame-notice-user-settings)) (if (fboundp 'frame-set-background-mode) (frame-set-background-mode (selected-frame))))) (if (fboundp 'font-menu-add-default) (font-menu-add-default)) (if inhibit-startup-hooks nil (run-hooks 'window-setup-hook)))) (setenv "TERM" "dumb") (let ((display (frame-parameter nil 'display))) (if (stringp display) (setq display (concat "DISPLAY=" display)) (let ((env initial-environment)) (while (and env (or (not (string-match "\\`DISPLAY=" (car env))) (progn (setq display (car env)) nil))) (setq env (cdr env))))) (if display (progn (setq process-environment (delete display process-environment)))))) normal-top-level() Loading macroexp.elc (compiled; note, source file is newer)... Args out of range: #s(comp-cstr-ctxt #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ())), 7 make[2]: *** [Makefile:323: emacs-lisp/macroexp.elc] Error 127 make[2]: Leaving directory '/d/Media/src/my/compile-emacs/source/lisp' make[1]: *** [Makefile:1015: bootstrap-emacs.pdmp] Error 2 make[1]: Leaving directory '/d/Media/src/my/compile-emacs/source/src' make: *** [Makefile:554: src] Error 2
I installed and ran garyo’s Emacs build to see if I could
make do with that instead. Quail seems to be broken, because it keeps saying it can’t load the
quail-rfc1345 package. (All I do in my personal configuration is enable the rfc1345 input method.)
I used find-library to inspect quail, which did exist in compressed form; I couldn’t find
rfc1345 in there. I don’t know whether that’s deliberate, an artefact of the compression process,
or something else. I didn’t see anything in the repository that would suggest it’s been removed on
purpose.
In the end, I reverted to using Emacs 29 from Scoop. I hope I can find the root cause of the compilation issues soon and go back to building my own binary, though.