Categories: coding

RSS - Atom - Subscribe via email

Using an Emacs Lisp macro to define quick custom Org Mode links to project files; plus URLs and search

| org, emacs, coding
  • [2024-01-12 Fri] Added embark action to copy the exported link URL.
  • [2024-01-11 Thu] Switched to using Github links since Codeberg's down.
  • [2024-01-11 Thu] Updated my-copy-link to just return the link if called from Emacs Lisp. Fix getting the properties.
  • [2024-01-08 Mon] Add tip from Omar about embark-around-action-hooks
  • [2024-01-08 Mon] Simplify code by using consult--grep-position

Summary (882 words): Emacs macros make it easy to define sets of related functions for custom Org links. This makes it easier to link to projects and export or copy the links to the files in the web-based repos. You can also use that information to consult-ripgrep across lots of projects.

I'd like to get better at writing notes while coding and at turning those notes into blog posts and videos. I want to be able to link to files in projects easily with the ability to complete, follow, and export links. For example, [[subed:subed.el]] should become subed.el, which opens the file if I'm in Emacs and exports a link if I'm publishing a post. I've been making custom link types using org-link-set-parameters. I think it's time to make a macro that defines that set of functions for me. Emacs Lisp macros are a great way to write code to write code.

(defvar my-project-web-base-list nil "Local path . web repo URLs for easy linking.")

(defmacro my-org-project-link (type file-path git-url)
  `(progn
     (defun ,(intern (format "my-org-%s-complete" type)) ()
       ,(format "Complete a file from %s." type)
       (concat ,type ":" (completing-read "File: "
                                          (projectile-project-files ,file-path))))
     (defun ,(intern (format "my-org-%s-follow" type)) (link _)
       ,(format "Open a file from %s." type)
       (find-file
        (expand-file-name
         link
         ,file-path)))
     (defun ,(intern (format "my-org-%s-export" type)) (link desc format _)
       "Export link to file."
       (setq desc (or desc link))
       (when ,git-url
         (setq link (concat ,git-url (replace-regexp-in-string "^/" "" link))))
       (pcase format
         ((or 'html '11ty) (format "<a href=\"%s\">%s</a>"
                                   link
                                   (or desc link)))
         ('md (if desc (format "[%s](%s)" desc link)
                (format "<%s>" link)))
         ('latex (format "\\href{%s}{%s}" link desc))
         ('texinfo (format "@uref{%s,%s}" link desc))
         ('ascii (format "%s (%s)" desc link))
         (_ (format "%s (%s)" desc link))))
     (with-eval-after-load 'org
       (org-link-set-parameters
        ,type
        :complete (quote ,(intern (format "my-org-%s-complete" type)))
        :export (quote ,(intern (format "my-org-%s-export" type)))
        :follow (quote ,(intern (format "my-org-%s-follow" type))))
       (cl-pushnew (cons (expand-file-name ,file-path) ,git-url)
                   my-project-web-base-list
                   :test 'equal))))

Then I can define projects this way:

(my-org-project-link "subed"
                     "~/proj/subed/subed/"
                     "https://github.com/sachac/subed/blob/main/subed/"
                     ;; "https://codeberg.org/sachac/subed/src/branch/main/subed/"
                     )
(my-org-project-link "emacsconf-el"
                     "~/proj/emacsconf/lisp/"
                     "https://git.emacsconf.org/emacsconf-el/tree/")
(my-org-project-link "subed-record"
                     "~/proj/subed-record/"
                     "https://github.com/sachac/subed-record/blob/main/"
                     ;; "https://codeberg.org/sachac/subed-record/src/branch/main/"
                     )
(my-org-project-link "compile-media"
                     "~/proj/compile-media/"
                     "https://github.com/sachac/compile-media/blob/main/"
                     ;; "https://codeberg.org/sachac/compile-media/src/branch/main/"
                     )
(my-org-project-link "ox-11ty"
                     "~/proj/ox-11ty/"
                     "https://github.com/sachac/ox-11ty/blob/master/")

And I can complete them with the usual C-c C-l (org-insert-link) process:

completing-custom-links.gif
Figure 1: Completing a custom link with org-insert-link

Sketches are handled by my Org Mode sketch links, but we can add them anyway.

(cl-pushnew (cons (expand-file-name "~/sync/sketches/") "https://sketches.sachachua.com/filename/")
            my-project-web-base-list
            :test 'equal)

I've been really liking being able to refer to various emacsconf-el files by just selecting the link type and completing the filename, so maybe it'll be easier to write about lots of other stuff if I extend that to my other projects.

Quickly search my code

Since my-project-web-base-list is a list of projects I often think about or write about, I can also make something that searches through them. That way, I don't have to care about where my code is.

(defun my-consult-ripgrep-code ()
  (interactive)
  (consult-ripgrep (mapcar 'car my-project-web-base-list)))

I can add .rgignore files in directories to tell ripgrep to ignore things like node_modules or *.json.

I also want to search my Emacs configuration at the same time, although links to my config are handled by my dotemacs link type so I'll leave the URL as nil. This is also the way I can handle other unpublished directories.

(cl-pushnew (cons (expand-file-name "~/sync/emacs/Sacha.org") nil)
            my-project-web-base-list
            :test 'equal)
(cl-pushnew (cons (expand-file-name "~/proj/static-blog/_includes") nil)
            my-project-web-base-list
            :test 'equal)
(cl-pushnew (cons (expand-file-name "~/bin") nil)
            my-project-web-base-list
            :test 'equal)

Actually, let's throw my blog posts and Org files in there as well, since I often have code snippets. If it gets to be too much, I can always have different commands search different things.

(cl-pushnew (cons (expand-file-name "~/proj/static-blog/blog/") "https://sachachua.com/blog/")
            my-project-web-base-list
            :test 'equal)
(cl-pushnew (cons (expand-file-name "~/sync/orgzly") nil)
            my-project-web-base-list
            :test 'equal)
ripgrep-code.gif
Figure 2: Using my-consult-ripgrep-code

I don't have anything bound to M-s c (code) yet, so let's try that.

(keymap-global-set "M-s c" #'my-consult-ripgrep-code)

At some point, it might be fun to get Embark set up so that I can grab a link to something right from the consult-ripgrep interface. In the meantime, I can always jump to it and get the link.

Tip from Omar: embark-around-action-hooks

[2024-01-07 Sun] I modified oantolin's suggestion from the comments to work with consult-ripgrep, since consult-ripgrep gives me consult-grep targets instead of consult-location:

(cl-defun embark-consult--at-location (&rest args &key target type run &allow-other-keys)
  "RUN action at the target location."
  (save-window-excursion
    (save-excursion
      (save-restriction
        (pcase type
          ('consult-location (consult--jump (consult--get-location target)))
          ('org-heading (org-goto-marker-or-bmk (get-text-property 0 'org-marker target)))
          ('consult-grep (consult--jump (consult--grep-position target)))
          ('file (find-file target)))
        (apply run args)))))

(cl-pushnew #'embark-consult--at-location (alist-get 'org-store-link embark-around-action-hooks))

I think I can use it with M-s c to search for the code, then C-. C-c l on the matching line, where C-c l is my regular keybinding for storing links. Thanks, Omar!

In general, I don't want to have to think about where something is on my laptop or where it's published on the Web, I just want to

View org source for this post
This is part of my Emacs configuration.

Avoiding automatic data type conversion in Microsoft Excel and Pandas

| coding, python

Automatic conversion of data types is often handy, but sometimes it can mess things up. For example, when you import a CSV into Microsoft Excel, it will helpfully convert and display dates/times in your preferred format–and it will use your configured format when exporting back to CSV, which is not cool when your original file had YYYY-MM-DD HH:MM:SS and someone's computer decided to turn it into MM/DD/YY HH:MM. To avoid this conversion and import the columns as strings, you can change the file extension to .txt instead of .csv and then change each column type that you care about, which can be a lot of clicking. I had to change things back with a regular expression along the lines of:

import re
s = "12/9/21 11:23"
match = re.match('([0-9]+)/([0-9]+)/([0-9]+)( [0-9]+:[0-9]+)', s)
date = '20%s-%s-%s%s:00' % (match.group(3).zfill(2), match.group(1).zfill(2), match.group(2).zfill(2), match.group(4))
print(date)

The pandas library for Python also likes to do this kind of data type conversion for data types and for NaN values. In this particular situation, I wanted it to leave columns alone and leave the nan string in my input alone. Otherwise, to_csv would replace nan with the blank string, which could mess up a different script that used this data as input. This is the code to do it:

import pandas as pd
df = pd.read_csv('filename.csv', encoding='utf-8', dtype=str, na_filter=False)

I'm probably going to run into this again sometime, so I wanted to make sure I put my notes somewhere I can find them later.

Started learning how to interactively debug Javascript in Emacs with Indium

| 11ty, emacs, coding

I noticed something strange in my static blog: my blogging category page didn't list my post on statically generating my blog with Eleventy. Now it does, of course, since I fixed it. But it didn't, and that was weird. I tried using console.log to debug it, but it was annoying to try to figure out the right thing to print out in a long list of nested objects. Besides, console.log debugging is so… last century.

Since these tips for debugging in 11ty mentioned interactively debugging things in VS Code, I decided it was a good time to learn how to use Indium, a Javascript development environment for Emacs.

(use-package indium :hook ((js2-mode . indium-interaction-mode)))

After some trial and error, this was the .indium.json file that allowed me to use M-x indium-launch to start the Eleventy process.

{
  "configurations": [
    {
      "name": "11ty",
      "type": "node",
      "program": "node",
      "args": "./node_modules/.bin/eleventy"
    }
  ]
}

I originally had "inspect-brk": true in it as well, following the suggested configuration, but I found it easier to just set breakpoints in my files using indium-add-breakpoint (C-c b b, a keybinding set up by indium-interaction-mode in my js2-mode-hook).

Conditional breakpoints didn't seem to work, so I just put my logic in an if and set my breakpoint in there.

  categories.forEach((item) => {
    if (item.slug == 'blogging') {
      let post = data.collections._posts.find(o => o.inputPath.match(/statically-generating-my-blog-with-eleventy/));
      console.log(post);
    }
    ...
  }

When I set my breakpoint on the let post... line and ran M-x indium-launch, I got an interactive debugger at that breakpoint. I could also switch to the REPL console and type stuff. Yay!

As it turned out, the post I wanted wasn't showing up in the list of posts. It was because I had used eleventyConfig.setTemplateFormats and forgotten to include md for Markdown files. Once I figured out what was going on, it was easy to fix. This is what the debugger looks like. It adds values to the ends of lines, and you can evaluate things.

Screenshot_20210816_002331.png

I'm looking forward to learning more about using Indium to debug scripts running in Node or Chrome. Slowly, slowly having some focused time to sharpen the saw!

If you use Emacs for Javascript development and you're curious about Indium, you can check out the documentation.