Excerpts from a conversation with John Wiegley (johnw) and Adam Porter (alphapapa) about personal information management

| productivity, org, pkm

Adam Porter (alphapapa) reached out to John Wiegley (johnw) to ask about his current Org Mode workflow. John figured he'd experiment with a braindumping/brainstorming conversation about Org Mode in the hopes of getting more thoughts out of his head and into articles or blog posts. Instead of waiting until someone finally gets the time to polish it into something beautifully concise and insightful, they decided to let me share snippets of the transcript in case that sparks other ideas. Enjoy!

John on meetings as a CTO and using org-review

Today I was playing a lot with org-review. I'm just trying to really incorporate a strong review process because one of the things I started doing recently is that this [Fireflies AI]​ note taker that's running in the background. Now, it produces terrible transcripts, but it produces great summaries. And at the bottom of every summary, there's a list of all the action items that everyone talked about associated with the names.

So I now have some automation, that will all I have to do is download the word document and then I have a whole process in the background that uses Pandoc to convert it to Org Mode. Then I have Elisp code that automatically will suck it into the file that I dedicate to that particular meeting.

It will auto-convert all of the action items into Org-mode tasks where it's either a TODO if it's for me, or if it's a task for somebody else, tagged with their name.

Then, when I have a one-on-one with a person in the future, I now have a one-on-one template that populates that file, and part of the template is under the agenda heading. It uses an a dynamic block that I've written: a new type of dynamic block that can pull from any agenda file. And what it does is it [takes] from all of those meetings, all of the action items that are still open that are tagged with their name.

This has been actually really, really effective. Now, I don't jump into a one-on-one being like, "Well, I didn't prepare so I don't know what to talk about." I've usually got like 10 to 30 items to go through with them to just see. Did you follow up? Did you complete this? Do we need to talk about this more?

I want to incorporate org-review. Scheduling is not sufficient for me to see my tasks. What I need is something that is like scheduling, but isn't scheduling. That's where org-review comes in. I have a report that says: show me everything that has never been reviewed or everything that is up for review.

Then I have a whole Org key space within agenda for pushing the next review date to a selected date or a fixed quantity of time. So if I hit r r, it'll prompt for the date that I want to see that again. But if I hit r w, it'll just push it out a week.

Every day I try to spend 15 minutes looking at the review list of all the tasks that are subject for review. I don't force myself to get through the whole list. I count it as success if I get through 20 of the tasks. Because earlier I had 730 of them, right? I was just chewing on them day by day.

But now I'm building this into the Org agenda population, because in the dynamic block match query, I can actually say: only populate this agenda with the tasks that are tagged for them that are up for review. That way, if we're in the one-on-one and they say, "Oh I'm working on that but I won't get to it for a month," I'll say, "Let's review that in a month." Then next week's one-on-one won't show that tasks. I don't have to do that mental filtering each time.

This is something I've been now using for a few weeks. I have to say I'm still streamlining, I'm still getting all the inertia out of the system by automation as much as possible, but it's helping me stay on top of a lot of tasks.

I'm surprised by how many action items every single meeting generates. It's like, it's like between 5 and 12 per meeting. And I have 3 to 7 meetings a day, so you can imagine that we're generating up to a hundred action items a week.

In the past, I think a lot of it was just subject to the whims of people's memory. They'll say, "I'm going to do that," and then… Did they remember to do that? Nobody's following up. Three months later, somewhere, they'll go like, "Oh yeah we talked about that, didn't we?"

So I'm trying to now stem the the tide of lost ideas. [My current approach] combines dynamic blocks with org-roam templates to make new files for every meeting and it combines org-review to narrow down the candidate agendas each time appropriately, and it combines a custom command to show me a list of all tasks that are currently needing review.

Reviewing isn't just about, "Is the thing done?" It's also, "Did I tag it with the right names? Did I delegate? Did I associate an effort quantity to it?" (I'm using efforts now as a way to quickly flag whether a day has become unrealistically over-full.)

I only started using column view very, very recently. I've never used it before. But now that I'm using effort strings, it does have some nice features to it: the ability to see your properties laid out in a table.

Back to table of contents

John on making meaningful distinctions (semantic or operational)

Today's agenda has 133 items on it. I need ways to narrow that agenda down.

I've used a lot of different tasks management philosophies. We're always looking for more efficiency, and we're looking for more personal adaptation to what works for us. I've gone from system to system. What I'm starting to realize is that the real value in all of these systems is that they're different enough from whatever you're using today, that they will force you to think about the system you're making for yourself, that is their value.

That's why I think there should always be a huge variety of such systems and people should always be exploring them. I don't believe any one one system can work for everybody, but we all need to be reflecting on the systems that we use. Somebody else showing you, "Hey, I do it this way" is a really nice way to juxtapose whatever system you're using.

I discovered through reading Karl Voit's articles that there are three principal information activities: searching, filtering, and browsing.

  • Hierarchies assist with browsing.
  • Tagging assist with filtering and keywords.
  • Metadata assist with searching.

Those are the three general ways that we approach our data.

We have to do work to draw distinctions between that data. The whole reason that we're drawing distinctions between that data is to narrow our focus to what is important.

I have over 30,000 tasks in my Org Mode overall. 23,000 of them are TODOs. Several thousand of them are still currently open. I'm never gonna see them all. Even if I wanted to, I'm never gonna see them all. I don't know what to search for. I don't know what the query should be. I have to use tagging and scheduling and categorization and everything. I believe that that is the work of a knowledge worker is to introduce these distinctions. That takes time and it takes effort.

What's really important is to draw meaningful distinctions. Make distinctions that matter.

I could tag things with like the next time I go to Walmart, so that I could do a filtered query to show me all things that I might want to do at Walmart, but is that worth the effort or is just tagging it as an errand enough? Because that list will get within the size range that I can now eyeball them all and mentally filter out the ones that I need for Walmart.

What makes a meaningful distinction? I believe there are two things that make a distinction meaningful. One is semantic, and one is operational.

A semantic distinction is a distinction that changes the meaning of the task. If I have a task that says "Set up Zoom account", if that's in my personal Org Mode, that has one level of priority and one level of focused demand. If it's in my work list, that has a totally different importance and a totally different focused demand. It changes the nature of the task from one that is low urgency (maybe a curiosity) to high urgency that might impact many people or affect how I can get my work done. That distinction is meaningful or semantic. It changes the meaning of the task.

An operational distinction changes how I interact with the task. [For example, if I tag a phone call, I can] group all of my phone calls during a certain time of the day. That changes my nature of interaction with the task. I'm doing it at a different time of day or doing it in conjunction with other tasks. That helps narrow my focus during that section of time that I have available for making calls. It's an operational distinction. if it's changing how you interact with the task.

You're succeeding at all of this if on any given day and any given time, what's in front of your eyes is what should be in front of your eyes. That's what all of this is about. If an operational distinction is not aiding you in that effort, it's not worth doing. It's not meaningful enough to go above the bar.

Back to table of contents

John on examples of distinctions that weren't personally worth it

I'm trying to narrow and optimize down to the minimum distinctions necessary to remain effective. If I can ever get rid of a distinction, I'm happy to do it.

I used to have projects and have categories, or what PARA method calls areas. Projects are different from areas and that they have a definition of completion and they have a deadline, but that's the only distinction. I realized that distinction doesn't do me any good because if it has a deadline, that's the distinction, right?

Calling it an area or calling it a project… I can just have projects without deadlines and then that's good enough. I have a query that shows me all projects whose deadlines are coming up within the next month, and then I'm aware of what I need to be aware of. I don't need to make the distinction between the ones that have and don't have deadlines. I just need to assign a deadline so the deadline was sufficient discrimination. I didn't need the classification difference between area and project.

And then [PARA's] distinction between projects, areas, and archives. I realize that there's only one operational benefit of an archive, and it's to speed things up by excluding archives from the Org ID database or from the org-roam-dbsync. That's it. That's the only reason I would ever exclude archives, because I want to search in archives. org-agenda-custom-commands is already only looking at open tasks. In a way, it's by implication archiving anything that's done in terms of its meaning.

This is all just an example of me looking at the para method and realizing that none of their distinctions really meant something for me.

What was meaningful was:

  • Does it have a deadline?
  • Is it bounded or not bounded?
  • Do I want to included in the processing of items?
  • [Is it a habit?]
Back to table of contents

John on habits

I did decide to draw the distinction of habits. I want them to look and feel different because I'm trying to become more habit-heavy.

I read this really brilliant book called Atomic Habits that I think has changed my life more than any other. I've read a lot of really good time management books but this book far and away has made the biggest impact on my life. One of its philosophical points that it makes that is so profound is that goal-oriented thinking is less successful in the long run than behavior-oriented thinking or habit- or system-oriented thinking. Instead of having a goal to clean your office, have a habit to remove some piece of clutter from your office like each time you stand up to go get a snack. You seek habits that in the aggregate will achieve the goals you seek to do.

I'm trying now to shift a lot of things in my to-do lists that were goals. I'm trying to identify the habits that will create systems of behavior that will naturally lead to those goals. I want habits to be first class citizens, and I want to be aware of the habits I'm creating.

I think the other thing that Atomic Habits did is it changed my conception of what a habit is. Before, I thought of a habit as "using the exercise bike" or something like that, which always made it a big enough task that I would keep pushing it off. Then I would realize I'd pushed it off for six months and then I would unschedule it and give up on it because it was just it would just be glaring at me with a look of doom from my agenda list.

What's important is the consistency, not the impact of any one particular accomplishing of that habit. It's a habit. If I do it daily, it's doesn't matter how much of it I do. So even if it just means I get on the bike and I spin the pedals for three minutes, literally, that's successful completion.

Any time you have a new habit, one of the activities in mastering that habit is to keep contracting the difficulty of the habit down, down. You've got to make it so stupidly small and simple to do, that you do it just for the fun of marking it done in the agenda, right?

I have a habit to review my vocabulary lists for languages that I'm learning. I'm okay with one word. As long as I ran the app and I studied one word, that's success.

What you find happening is that you'll do the one word, and now because you're there, because you're in the flow of it, you're like, "I'll do two. You know, I'm already here. What's the big difficulty in doing two?"

So you make the success bar super low. You're trying to almost trick yourself into getting into the flow of whatever that activity is.

[org-habit org-ql list] So I have all of these habits here, and every single habit on this list is super easy to do. Five minutes is all that it would take, or even one minute for most of them. I use different little icons to group them. It also keeps the title of the habit really small. I found that when the titles were really long. I didn't like reading it all the time. It just was a wall of text. When it's these one word plus an icon, it just kind of jumps out.

Back to table of contents

Adam on the Hammy timer and momentum

I took that to a bit of an extreme sort of with my my package remote called Hammy, for hamster. It's for timers and ideas, kind of like being a hamster on a hamster wheel.

Anyway, one of the timers is called flywheel mode. The idea is: just do a little bit. Like, if I'm just having a mental block, I can't stand working on that test today, I'm going to do five minutes. I can spend five minutes doing whatever. Next time, we do 10 minutes in 15. Pretty soon, I'm doing 45 minutes at a stretch. Maybe when I sit down to do 5, I'll actually do 15. I'm just slowly building up that mental momentum. I'll allow myself to quit after 5 minutes, but I end up doing 20.

Back to table of contents

John on momentum and consistency

Momentum is key. There's a flip side to this whole concept of the value of iterative improvement. The opposite remains also true.

Consistent good is your best ally, and inconsistent bad is also your ally. It's when the reverse is true that you have inconsistent good and consistent bad, that's what leads you into roads of doom.

That never occurred to me before. I would always be one of those people who would set myself up with a goal, like, I want to lose 20 pounds. I would struggle to achieve it. I would be dismayed because of how hard it was to get there, and then you'd have a day when you're like, you get off the wagon and you're like, The game is lost. And then and then you can't get back on again. Whereas now it's like that wagon, it's not so easy to get off of. I have to really make a concerted effort to be consistently bad in order to make things horrible again.

I almost want to change org-habit to have a different kind of visualization, because streaks are not motivators for me. Streaks punish you for losing one day out of 200, right? I don't want a graph that shows me streaks. I want a graph that shows me consistency. If I have 200 days and I've missed five of them, I'm super consistent. Maybe I could do this with colors. Just show a bar with that color, and don't show individual asterisks to show when I did it or when I didn't do it, because I find streaks anti-motivating.

[Discussion about other ways to display habits]

Back to table of contents

John on Life Balance by Llamagraphics

The whole principle around Life Balance [by Llamagraphics]​ was: you take all of your tasks, you categorize them, you associate difficulty to them and priority and everything else. Then it tries to use heuristics to determine if your life is being balanced, [and it percolates certain tasks to the top of your list].

If the system's doing a good job, then your agenda list should always be A-Z pretty much the best order in which you ought to do things. It didn't just do category-based balance, it also did difficulty-based balance. You should only be doing super hard stuff once in a while. You do a hard thing, then you do lots of easy things, then you do a hard thing.

Now, I'm wondering… This idea of momentum is very similar to the idea of balance. "Have established momentum with a system of behavior" is similar to "Have an established balance with all of the tasks that I do related to different activities." Is there a data architecture that would allow me to do both of these things.

The whole idea of making the habits be colors and then sorting them according to the spectrum is literally just to achieve balance among how much attention I'm paying to different habits.

[Discussion about dynamic prioritization]

Back to table of contents

Adam on the structure of his TODO view

My fundamental system right now is there's like two org-ql views. There's the view of tasks that are scheduled for today or have a deadline of today, and then there's a view of tasks that I've decided that they need to be done, but I haven't decided when to do them yet.

[Top list]: I just pick the next task off the list or reschedule if it's not important enough now. But then when that's empty, if it ever gets that way, it's the second view. I decide, okay, there's something I need to do. I can do that on Tuesday. Then it disappears until I need to think about it again.

This separates deciding what to do from when to do. Then I can just switch into my own manager mode for a moment, and then switch into "just put your head down and do the work mode."

[More details]

The top view is basically tasks that have a deadline, that are relevant to now (either deadline today or in the past), or it's an item that I've scheduled to work on today or in the past.

The view below, that is items that have no planning date. I need to give them one, or maybe they can just sit in that list of projects that have no next task. I use a project heading to [note] something that needs to be subdivided if I don't have a next task for it, then that'll show up there to remind me to give it one. Once it has a next task, [that] task would appear instead of the project heading until I schedule it. Anything I've forgotten to schedule yet will show up in that list.

Below that I just have a small window that shows me things. I've completed or clocked in the past week.

And then, another small window shows me anything that's a project status so I can get an overview.

In the work file itself, I have a number of links to org-ql views, like "Show me all my top level projects," "Show me tasks I need to talk to my boss about" or somebody else.

Back to table of contents

John on Org and data consistency

Org Mode is really a database, right? It's a database of of highly structured data that has a lot of associated metadata.

The value of that data requires a certain level of consistency which is work that we have to do. In the same way we do work drawing distinctions, we need to do work to keep that data consistent. Am I using this [property]? Am I using this tag to mean the right thing or whatever? Karl Voit says that one of the most valuable things if you're going to use tagging to organize your data is a constrained tag vocabulary. Make a fixed list. Then it's an error if you tag something and it's not in that list, because you either need to expand the list or you need to choose a better tag. That's really valuable.

Even though I use org-lint on all my org files, I found serious data errors. [The newline before an initial star had been lost], and then Org wouldn't see the entry. I never knew that it wasn't even being a participant in any of my queries. I just didn't know stuff like that.

I created a whole bunch of Haskell libraries that allow me to parse Org Mode data. It's a very opinionated parser. It's a very strict parser. It will not parse data files that do not have the exact shape and text and taxonomy that I want.

I wrote a linting module that basically encodes every single rule that I have ever wanted to apply to my data. Like, in the title of an Org Mode heading. I don't want two spaces. I don't want extra excess white space. That should be a rule right?

[Multiple examples, including when a file had TODO entries but didn't have a TODO filetag.]

My linter makes sure that this rule is consistently maintained. Being able to have an aggressive, thorough, universal consistency throughout all of my org data has really put my mind at ease. I can't break my data because I just won't be able to commit the broken data into git. I find myself adding new linting rules on a weekly basis. The more that I add, the more value my data has, because the more regular it is, the more normal, the more searchable.

Back to table of contents

My takeaways

People:

View org source for this post

How do I want to get better at learning out loud? Part 1 of 4: Starting

| sharing, blogging, writing

Nudged by Thierry Stoehr's toot about my 23rd blogiversary, I've been thinking about how much I've learned thanks to blogging, and how I can get even better at learning out loud. I'm curious about what this could become over the next twenty years, when I'm in my sixties.

The first part of the text from the sketch is duplicated and expanded in the list below. There are a lot of different aspects I want to get better at, so I'm not going to try to work on all of them in one go, but it's fun mapping out so much room for growth.

Besides, maybe one of these aspects will resonate with you as either something you're learning or something you've figured out something about, and then you'll get in touch, and then we'll both learn more. Wouldn't that be cool?

I'm experimenting with getting stuff out in smaller chunks, so this is part 1 of 4: Starting.

Noticing

I think of this as seeing the opportunity for learning, which I sometimes miss out on because I take things for granted or I don't connect the dots. I can get better at this by slowing down and by borrowing other people's questions. I've been giving myself more time to write and draw these days. It feels a little weird ignoring the other tasks on my TODO lists that are more clearly defined or that are related to other people's requests, but I like the way this feels.

Imagining

It's easy for me to come up with all sorts of ideas for things I want to tweak about Emacs. I can get better at this by reading more about what other people are doing and what other capabilities are there.

I can also get better at exploring ideas for non-Emacs topics, like ways to respond to parenting situations and things I can do support the causes I care about. I can expand my toolbox by reading books and blog posts, and depending on the topic, I can also listen to podcasts and videos.

Bumping into things

It's useful to bump into things I might not think of looking for. One way to do that would be to save various manuals on my e-ink notebook and phone so that I can read them during quiet moments.

I've added some randomness to shuffle old blog posts and tasks, although browsing through this tends to be low-priority. (I never get to the bottom of my reading/thinking list!)

"On this day" might be interesting too. This is more for fun and serendipity. I used to have it on my blog, and it should be pretty easy to reimplement using 11ty.

Learning from others

I'd like to spend more time thinking about and building on other people's ideas, maybe starting with Emacs and then branching out to other topics.

I also want to get back to reading people's blogs through an RSS reader so that I can get a slightly wider view of people's interests and learn more about non-Emacs things. I've added Feeder to my phone.

Taking notes

I capture a lot of snippets in my Org Mode inbox. I'd like to get better at adding some more context and quick thoughts when I create a note so that it's easier to pick up the idea later on. I do most of this capturing on my phone, so I'm getting the hang of slowing down and adding some more notes.

I also want to get better at actually reviewing and refining those notes. My inbox tends to grow and grow, especially when I get interrupted by an interesting idea. I have some writing/editing time while I keep A+ company during virtual school, so it'll be fun revisiting the notes I stashed.

Collecting

This is about putting a bunch of related notes together, which I usually do by refiling them. It's probably also related to clustering, which I'll get to in the next post about thinking.

I do most of this collecting on my computer, so I can write a few Emacs functions to make it easier. For example, I have some code to do the opposite of refiling so that while I'm looking at a topic, I can pull in a subtree from somewhere else.

Some buckets collect thoughts for blog posts, some for projects, some just for areas I'm interested in. I feel like I tend to lose track of the buckets that I'm collecting thoughts into–the list of slightly-less-active thoughts, as the active thoughts are easily findable. Maybe this is okay.

Expanding

I want to get better at going from a microblog post/toot or a quick index-card-type sketch to a longer blog post or sketchnote.

Actually, since my current workflow focuses mostly on blog posts, I think this part is more about contracting: picking out a small thought that I can share right now instead of waiting until I write the rest of it. This idea might also include picking a medium-sized chunk and making it the first post in a series, and the current post is an experiment in doing so. I'm a little hesitant to do so because my brain tends to wander off towards the end of a series, but it might be worth an experiment.

I'll also need some code to make it easier to add links between things in a series, which could be manual (handy for other non-series links too) or possibly something handled on the 11ty side (like How to build a blog series with 11ty/Eleventy).

I can also experiment with spreading posts out by scheduling them as Org tasks. I've theoretically added support for holding back future-dated posts in my 11ty config, but I think managing that on the Org side might be easier for now.

Boosting

This is linked to learning from others. Boosting what other people have said or thought can be a quick and easy way of learning out loud and enlarging the conversation. I can do more of that through Mastodon toots, rolling them up into my blog. I often add snippets to my config based on the things I come across in Emacs News.

I also like the commentary on blogs like Irreal and would like to grow into things like that.

Next up: thinking, making, and sharing

Over the next little while, I'm looking forward to fleshing out the next sections. Let's see if breaking things up into posts works…

  • Thinking: trying; reflecting; developing thoughts; shifting; reviewing; clustering; building on; searching (local, web, exact, approximate); questioning; reframing
  • Making: organizing; writing; editing; drawing (sketchnotes, doodles); diagramming; plotting; charts; coding; showing
  • Sharing: designing; linking; mapping thoughts; joining the conversation
View org source for this post

Wednesday weblog: Toots ending 2024-11-06

| review, roundup
  • embark-org-insert-link-to - 2024-11-02T15:49:09.420Z

    Ooh, it looks like `consult-org-heading` already lets me use embark-act with the shortcut `j` to call `embark-org-insert-link-to`. It doesn't feel like a "j" shortcut, though, so I'll just bind it to `L` for "link" instead: `(keymap-set embark-org-heading-map "L" #'embark-org-insert-link-to)`

  • EmacsConf progress: intros - 2024-11-01T18:32:21.973Z

    #emacsconf progress: I've recorded intros for all the talks so that speakers can review them, and some of the talk videos have started coming in. I'll ask the speakers for feedback after the video upload target date. Still slightly stressed about the prospect of replacing or re-setting-up BigBlueButton; corwin has taken over the stressing out about it at the moment.

  • Planet Emacslife CSS tweaks - 2024-11-01T13:14:52.582Z

    More tweaks to https://planet.emacslife.com: now using flex layout, so it should be a bit more responsive to screen size; sticky headers on large screens so that you can see which post it is when you scroll down. Does it make sense to do sticky headers on small screens? I don't want long titles taking up too much screen space on phones…

  • Emacs commits in Planet Emacslife? - 2024-11-01T11:14:53.651Z

    How do people feel about automatically including #Emacs commits affecting etc/NEWS and Org Mode's ORG-NEWS in https://planet.emacslife.com ? Handy for staying up to date? Too much, since it's easy to subscribe separately?

  • Thinking about navigating a large archive - 2024-10-30T14:48:16.127Z

    I use the 11ty static site generator to publish my blog as plain HTML pages. I have a lot of posts in some categories, like Emacs. I want to generate some of the pages for easier browsing, but I'm not sure it makes sense to generate all of them. Right now I generate 5 pages of posts and then a page that links to all of them. (Ex: https://sachachua.com/blog/category/emacs). It occurred to me that it might not be obvious that there are more than 5 pages of posts (since we're more used to dynamic systems that paginate as much as needed). I wonder how I can make that clearer - oh, maybe I can add the number of posts.

    There's probably stuff I can do to make the All Posts easier to explore, too. I've started making topic pages. I'm also curious about implementing the stacked-cards navigation you see in digital gardens like https://notes.andymatuschak.org/ .

    Ideas? Pointers to other statically generated blogs with large categories who've figured some of this out?

  • Living in the shallows - 2024-10-30T00:24:48.668Z

    Juxtaposing "The Shallows: What the Internet Is Doing to Our Brains" (Nicholas Carr), "Deep Work" (Cal Newport), and "Four Thousand Weeks: Time Management for Mortals" (Oliver Burkeman), I find myself leaning more towards Burkeman's acceptance of limits and lack of control. I'd rather figure out how to embrace these shallows than to write off large portions of my life: parenting a young child with all the attendant interruptions (which I am learning to welcome) and preparing for eventual old age.

    One of the things I've learned while doing Emacs News is that even things I can do in the shallows can be useful. Organizing information and passing it along does not require deep reflection or a quiet mind.

    I can read in short bursts here and there, take notes, and share them.

    Most of my Emacs tweaks are short, but they accumulate.

    Now I am learning to write small thoughts. They are not amazing insights, but they are enough for me, and sometimes they resonate with other people.

    Besides, even when I had full autonomy during my experiment with semi-retirement, it's not like I did amazingly deep stuff either. So that's all cool and I don't have to kid myself or feel like I'm missing out. Instead, I can enjoy this time in the shallows, when doing the dishes or tidying up is pretty much on the same level as many other things on my list of things I could do (probably more useful than most things, even). I can let myself be interruptible, and I can play with the fragments of my attention.

    Follow-up post: Embracing the shallows

  • The Imagination Muscle - 2024-11-02T21:22:36.486Z

    AoM podcast on The Imagination Muscle:

    My notes:

    • Observation is the start of imagination.
    • Commonplace book, sketches
    • Observational closure
    • Reading many books at the same time, connecting commonplace books - this makes me think of dancer curiosity
    • History of coffee shops in London, clusters

    I wonder how I can use sketches and/or microblog posts and my Org files as commonplace books…

  • Doodling in blog posts - 2024-11-04T21:17:03.208Z

    I had fun breaking up the monotony of text with doodles: https://sachachua.com/blog/2024/11/a-year-with-my-cargo-bike/ Also, yay #biking!

View org source for this post

Interactively recolor a sketch

I wanted to be able to change the colours used in a sketch, all from Emacs. For this, I can reuse my Python script for analyzing colours and changing them and just add some Emacs Lisp to pick colours from Emacs.

2024-11-05-14-15-55.svg
Figure 1: Selecting the colour to replace
2024-11-05-14-16-04.svg
Figure 2: Selecting the new colour
(defvar my-recolor-command "/home/sacha/bin/recolor.py")

(defun my-image-colors-by-frequency (file)
  "Return the colors in FILE."
  (with-temp-buffer
    (call-process my-recolor-command nil t nil (expand-file-name file))
    (goto-char (point-min))
    (delete-region (point-min) (1+ (line-end-position)))
    (mapcar (lambda (o) (concat "#" (car (split-string o "[ \t]"))))
            (split-string (string-trim (buffer-string)) "\n"))))

(defun my-completing-read-color (prompt list)
  "Display PROMPT and select a color from LIST."
  (completing-read
   (or prompt "Color: ")
   (mapcar (lambda (o)
             (faces--string-with-color o o))
           list)))

(defun my-image-recolor-interactively (file)
  (interactive (list (read-file-name "File: " (concat my-sketches-directory "/") nil t
                                     nil
                                     (lambda (file) (string-match "\\.png\\'" file)))))
  (save-window-excursion
    (find-file file)

    ;; Identify the colors by frequency
    (let (choice done)
      (while (not done)
        (let* ((by-freq (my-image-colors-by-frequency file))
               (old-color (my-completing-read-color "Old color: " by-freq))
               (new-color (read-color "New color: " t))
               (temp-file (make-temp-file "recolor" nil (concat "." (file-name-extension file))))
               color-map)
          (when (string-match "#\\(..\\)..\\(..\\)..\\(..\\).." new-color)
            (setq new-color (concat (match-string 1 new-color)
                                    (match-string 2 new-color)
                                    (match-string 3 new-color))))
          (setq color-map (replace-regexp-in-string "#" "" (concat old-color "," new-color)))
          (call-process my-recolor-command nil nil nil
                        (expand-file-name file)
                        "--colors"
                        color-map
                        "--output" temp-file)
          (find-file temp-file)
          (pcase (read-char-choice "(y)es, (m)ore, (r)edo, (c)ancel: " "yrc")
            (?y
             (kill-buffer)
             (rename-file temp-file file t)
             (setq done t))
            (?m
             (kill-buffer)
             (rename-file temp-file file t))
            (?r
             (kill-buffer)
             (delete-file temp-file))
            (?c
             (kill-buffer)
             (delete-file temp-file)
             (setq done t))))))))

It would be nice to update the preview as I selected things in the completion, which I might be able to do by making a consult–read command for it. It would be extra cool if I could use this webkit-based color picker. Maybe someday!

View org source for this post

Emacs: Extract part of an image to another file

| emacs

It turns out that image-mode allows you to open an image and then crop it with i c (image-crop), all within Emacs. I want to select a region and then write it to a different file. I think the ability to select a portion of an image by drawing/moving a rectangle is generally useful, so let's start by defining a function for that. The heavy lifting is done by image-crop--crop-image-1, which tracks the mouse and listens for events.

;; Based on image-crop.
(defun my-image-select-rect (op)
  "Select a region of the current buffer's image.

`q':   Exit without changing anything.
`RET': Select this region.
`m':   Make mouse movements move the rectangle instead of altering the
       rectangle shape.
`s':   Same as `m', but make the rectangle into a square first."
  (unless (image-type-available-p 'svg)
    (error "SVG support is needed to crop and cut images"))
  (let ((image (image--get-image)))
    (unless (imagep image)
      (user-error "No image under point"))
    (when (overlays-at (point))
      (user-error "Can't edit images that have overlays"))
    ;; We replace the image under point with an SVG image that looks
    ;; just like that image.  That allows us to draw lines over it.
    ;; At the end, we replace that SVG with a cropped version of the
    ;; original image.
    (let* ((data (cl-getf (cdr image) :data))
           (type (cond
                  ((cl-getf (cdr image) :format)
                   (format "%s" (cl-getf (cdr image) :format)))
                  (data
                   (image-crop--content-type data))))
           (image-scaling-factor 1)
           (orig-point (point))
           (size (image-size image t))
           (svg (svg-create (car size) (cdr size)
                            :xmlns:xlink "http://www.w3.org/1999/xlink"
                            :stroke-width 5))
           ;; We want to get the original text that's covered by the
           ;; image so that we can restore it.
           (image-start
            (save-excursion
              (let ((match (text-property-search-backward 'display image)))
                (if match
                    (prop-match-end match)
                  (point-min)))))
           (image-end
            (save-excursion
              (let ((match (text-property-search-forward 'display image)))
                (if match
                    (prop-match-beginning match)
                  (point-max)))))
           (text (buffer-substring image-start image-end))
           (inhibit-read-only t)
           orig-data svg-end)
      (with-temp-buffer
        (set-buffer-multibyte nil)
        (if (null data)
            (insert-file-contents-literally (cl-getf (cdr image) :file))
          (insert data))
        (let ((image-crop-exif-rotate nil))
          (image-crop--possibly-rotate-buffer image))
        (setq orig-data (buffer-string))
        (setq type (image-crop--content-type orig-data))
        (image-crop--process image-crop-resize-command
                             `((?w . 600)
                               (?f . ,(cadr (split-string type "/")))))
        (setq data (buffer-string)))
      (svg-embed svg data type t
                 :width (car size)
                 :height (cdr size))
      (with-temp-buffer
        (svg-insert-image svg)
        (switch-to-buffer (current-buffer))
        (setq svg-end (point))
        ;; Area
        (let ((area
               (condition-case _
                   (save-excursion
                     (forward-line 1)
                     (image-crop--crop-image-1
                      svg op))
                 (quit nil))))
          (when area
            ;;  scale to original
            (let* ((image-scaling-factor 1)
                   (osize (image-size (create-image orig-data nil t) t))
                   (factor (/ (float (car osize)) (car size)))
                   ;; width x height + left + top
                   (width (abs (truncate (* factor (- (cl-getf area :right)
                                                      (cl-getf area :left))))))
                   (height (abs (truncate (* factor (- (cl-getf area :bottom)
                                                       (cl-getf area :top))))))
                   (left (truncate (* factor (min (cl-getf area :left)
                                                  (cl-getf area :right)))))
                   (top (truncate (* factor (min (cl-getf area :top)
                                                 (cl-getf area :bottom))))))
              (list :left left :top top
                    :width width :height height
                    :right (+ left width)
                    :bottom (+ top height)))))))))

Then we can use it to select part of an image, and then use ImageMagick to extract that part of the image:

(defun my-image-write-region ()
  "Copy a section of the image under point to a different file.
This command presents the image with a rectangular area superimposed
on it, and allows moving and resizing the area to define which
part of it to crop.

While moving/resizing the cropping area, the following key bindings
are available:

`q':   Exit without changing anything.
`RET': Save the image.
`m':   Make mouse movements move the rectangle instead of altering the
       rectangle shape.
`s':   Same as `m', but make the rectangle into a square first."
  (interactive)
  (when-let* ((orig-data (buffer-string))
              (area (my-image-select-rect "write"))
              (inhibit-read-only t)
              (type (image-crop--content-type orig-data))
              (left (plist-get area :left))
              (top (plist-get area :top))
              (width (plist-get area :width))
              (height (plist-get area :height)))
    (with-temp-file (read-file-name "File: ")
      (set-buffer-multibyte nil)
      (insert orig-data)
      (image-crop--process image-crop-crop-command
                           `((?l . ,left)
                             (?t . ,top)
                             (?w . ,width)
                             (?h . ,height)
                             (?f . ,(cadr (split-string type "/"))))))))

i w seems like a sensible shortcut for writing a region of an image.

(with-eval-after-load 'image
  (keymap-set image-mode-map "i w" #'my-image-write-region))
This is part of my Emacs configuration.
View org source for this post

A year with my cargo bike

| decision, life

Summary: Life with a cargo bike has been working out really well for our family.

Stroller

I used to walk for an hour to get to some of A+'s playdates, pushing her in the Thule bike trailer / stroller that she still fit into. I liked bringing popsicles during the summer so that A+ could share them with her friends, so I often balanced a small cooler on top of the stroller and walked as briskly as I could. The popsicles were usually still reasonably cold by the time I got to the park. We'd spend a few hours playing there, and then there would be another hour's walk back. A+ usually napped on the way, so it was a chance for me to listen to podcasts.

Car

Sometimes we biked to the playdate instead. That was much faster in terms of getting there, even with a popsicle break halfway through. Those popsicles were only for us, since I couldn't bring a cooler on my bike. Also, A+ was usually too tired to bike back, or it was too dark for her to be safe biking on the busy streets between the park and our house, so we often waited in the mall parking lot for W- to pick up A+ and her bike in his car. Then I biked back by myself.

Dumped

We'd been considering cargo bikes for a while, and eventually things lined up to make it possible. It was a carefully-considered decision. I did a bunch of test rides using different models of cargo bikes. My height (or lack of it) ruled out many of the models designed for taller people. A+ was quite vocal about her preference for the suspension on the R&M Load cargo bikes, and she liked the view from the front-loaders more than the longtails. I rented the Load 75 and the Load 60 to try them out, accidentally tipping over onto the side an embarrassing number of times; A+ was safely buckled in but very grumpy about it.

When we confirmed that a cargo bike fit into our life, I bought a Riese & Müller Load 75 from Curbside Cycle. We picked the Load 75 over the Load 60 because the rain cover was nicer and the extra room could give us more years of use as A+ grows.

Loaded up

I love it. Biking is my favourite way to get around. There's just something so cheerful about it. A+ and I sing as we go around town. We smile at dogs in sweaters. She takes pictures of trees. Sometimes there are cargo bikes in front of us as we wait at the traffic light, and we wave and nod.

We got the Bakkie bag, too. It's designed to tow a kid's bike. That way, A+ can bike wherever she wants. When she gets tired, she can hop into the cargo bike and I can buckle her bike into the Bakkie bag, towing it all the way home. We've been able to go on more bike adventures by ourselves and together with W- because we don't have to worry about exceeding A+'s range.

Hot chocolate

Since we could get to the playground in 15 minutes instead of 60, it was a lot easier to bring snacks to share. We pretty much kept the playground kids well-supplied with free popsicles (and the occasional much-coveted ice cream treat) all summer, and the ice packs came in handy for treating the occasional bumps too. We even brought disposable cups and insulated bottles of hot water for making hot chocolate and instant apple cider in the colder months.

Potting mix

Aside from taking A+ to a wider range of places, we've also used it to bring several bags of potting mix or a propane tank home from the hardware store, carry other bulky items, and take lots of stuff to the community environment days for recycling/donation.

We are very lucky to have cargo biking as an option. When people ask me how much it is, I ruefully tell them, "Well, it's less than a second car." We weren't actually choosing between this and a second car; even though W- rarely uses his car these days, I'm too anxious to drive. My brain gets a little squirrelly and is prone to attentional hiccups. I don't want a moment of distraction to result in someone's death or serious injury. I'm still on alert when I bike, but it feels a lot more like something I can handle. And biking is so fast and convenient. I don't have to nudge A+ out of a playdate so that we can make it out before the subway gets packed like sardines, or shepherd A+ back home from the subway station ("I'm tiiiired.").

I got the bike in November 2023. Here's how much I biked over the past year:

Month KM
Nov 208
Dec 157
Jan 69
Feb 78
Mar 176
Apr 82
May 106
Jun 143
Jul 135
Aug 96
Sep 212
Oct 120
2024-11-04T14:04:00.159647 image/svg+xml Matplotlib v3.6.3, https://matplotlib.org/
Figure 1: Graph of kilometres by month

I was pleasantly surprised that even during the cold months (and A+'s reluctance to go outside if it was very cold or slushy), and even during the schoolweek, we still managed to get out on the bike.

2024-11-04T13:43:31.432403 image/svg+xml Matplotlib v3.6.3, https://matplotlib.org/
Figure 2: Kilometres by date

I got data from the ebike-connect site using Spookfox using the code below.

Javascript code for extracting distances and times
[...document.querySelectorAll('.activities__ride-menu')].map((o) => {
  return {
    date: o.querySelector('.activities__menu-details > span').textContent,
    distance: o.querySelector('.activities__menu-distance-text').textContent.trim(),
    time: o.querySelector('.activities__menu-details > span:nth-child(2) > span:nth-child(2)').textContent,
  }
});
Emacs Lisp to group distance by month
(let ((by-month (seq-group-by
  (lambda (row)
    (let ((date (plist-get row :date)))
      (when (string-match "[0-9][0-9]\\.\\([0-9][0-9]\\)\\.\\([0-9][0-9]\\) [0-9][0-9]:[0-9][0-9]"
                          date)
        (format "20%s-%s-01"
                (match-string 2 date)
                (match-string 1 date)))))
  trips)))
  (append
   '(("Month" "Distance")
     hline)
   (mapcar
    (lambda (row)
      (list (format-time-string "%b" (date-to-time (car row)))
            (format
             "%d"
             (round (apply '+
                           (mapcar (lambda (entry) (string-to-number (plist-get entry :distance)))
                                   (cdr row)))))))
    (reverse (seq-filter (lambda (o) (string< (car o) "2024-11")) by-month)))))
Emacs Lisp to group distance by date
(let ((by-day (seq-group-by
  (lambda (row)
    (let ((date (plist-get row :date)))
      (when (string-match "\\([0-9][0-9]\\)\\.\\([0-9][0-9]\\)\\.\\([0-9][0-9]\\) [0-9][0-9]:[0-9][0-9]"
                          date)
        (format "20%s-%s-%s"
                (match-string 3 date)
                (match-string 2 date)
                (match-string 1 date)))))
  trips)))
  (json-encode (mapcar
   (lambda (row)
     (cons (car row)
           (format
            "%d"
            (round (apply '+
                          (mapcar (lambda (entry) (string-to-number (plist-get entry :distance)))
                                  (cdr row)))))))
   (reverse (seq-filter (lambda (o) (string< (car o) "2024-11")) by-day)))))
Python code for making a bar graph of distance by month
import pandas as pd
import datetime
import matplotlib.pyplot as plt
import seaborn as sns
import json

data = trips
df = pd.DataFrame(data, columns=["Month", "Distance"])
df.set_index('Month')
df['Distance'] = df['Distance'].astype(float)
plt.figure(figsize=(8, 6), dpi=100)
sns.barplot(data=df, y='Distance', x='Month')
plt.savefig('biking-distance-by-month.svg')
Python code for making a heatmap
import pandas as pd
import datetime
import matplotlib.pyplot as plt
import seaborn as sns
import json

start_date = datetime.datetime(2023, 11, 1)
end_date = datetime.datetime(2024, 11, 1)
dates = pd.date_range(start=start_date, end=end_date, freq='D')
data = json.loads(trips)
df = pd.DataFrame.from_dict(data, orient='index')
df.index = pd.to_datetime(df.index)
df[0] = df[0].astype(float)
# Create calendar heatmap
plt.figure(figsize=(16, 3), dpi=100)
pivoted = df.pivot_table(index=df.index.day_name(), columns=df.index.strftime('%Y-%W'))
all_days = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']
pivoted.index = pd.Categorical(pivoted.index, all_days, ordered=True)
pivoted = pivoted.sort_index()
heatmap = sns.heatmap(
    pivoted,
    cmap="crest",
    linewidths=0.5,
    linecolor='white'
)

# Set the x-axis tick labels to show only the months
month_labels = df.index.strftime('%b').unique()
month_ticks = [i * 4 for i in range(len(month_labels))]
plt.xticks(
    month_ticks,
    month_labels,
    rotation=90
)
tick_positions = [i + 0.5 for i in range(len(all_days))]
plt.yticks(tick_positions, all_days)
plt.title('Distance on dates')
plt.xlabel('November 2023 - November 2024')
plt.ylabel('')
plt.xticks(rotation=90)
plt.savefig('biking-by-day.svg')

I like our cargo bike a lot. I hope to ride it for many years to come.

View org source for this post

2024-11-04 Emacs news

| emacs, emacs-news

Links from reddit.com/r/emacs, r/orgmode, r/spacemacs, r/planetemacs, Mastodon #emacs, Hacker News, lobste.rs, programming.dev, lemmy.world, lemmy.ml, communick.news, planet.emacslife.com, YouTube, the Emacs NEWS file, Emacs Calendar, and emacs-devel. Thanks to Andrés Ramírez for emacs-devel links. Do you have an Emacs-related link or announcement? Please e-mail me at sacha@sachachua.com. Thank you!

View org source for this post