Comparison-shopping with Org Mode
| emacs, orgI don't like shopping. We're lucky to be able to choose, but I get overwhelmed with all the choices. I'm trying to get the hang of it, though, since I'll need to shop for lots of things for A- over the years. One of the things that's stressful is comparing choices between different webpages, especially if I want to get A-'s opinion on something. Between the challenge of remembering things as we flip between pages and the temptations of other products she sees along the way… Ugh.
I think there are web browser extensions for shopping, but I prefer to
work within Org Mode so that I can capture links from my phone's web
browser, refile entries into different categories, organize them with
keyboard shortcuts, and tweak things the way I like. So if I have
subheadings with the NAME, PRICE, IMAGE, and URL properties, I
can make a table that looks like this:
Figure 1: Comparison-shopping
using code that looks like this:
#+begin_src emacs-lisp :eval yes :exports results :wrap EXPORT html (my-org-format-shopping-subtree) #+end_src
and I can view the table by exporting the subtree with HTML using
org-export-dispatch (C-c C-e C-s h o). When I add new items, I can
use C-u C-c C-e to reexport the subtree without navigating up to the
root.
Here's the very rough code I use for that:
(defun my-get-shopping-details () (goto-char (point-min)) (let (data) (cond ((re-search-forward " data-section-data >" nil t) (setq data (json-read)) (let-alist data (list (cons 'name .product.title) (cons 'brand .product.vendor) (cons 'description .product.description) (cons 'image (concat "https:" .product.featured_image)) (cons 'price (/ .product.price 100.0))))) ((and (re-search-forward "<script type=\"application/ld\\+json\">" nil t) (null (re-search-forward "Fabric Fabric" nil t))) ; Carter's, Columbia? (setq data (json-read)) (if (vectorp data) (setq data (elt data 0))) (if (assoc-default '@graph data) (setq data (assoc-default '@graph data))) (if (vectorp data) (setq data (elt data 0))) (let-alist data (list (cons 'name .name) (cons 'url (or .url .@id)) (cons 'brand .brand.name) (cons 'description .description) (cons 'rating .aggregateRating.ratingValue) (cons 'ratingCount .aggregateRating.reviewCount) (cons 'image (if (stringp .image) .image (elt .image 0))) (cons 'price (assoc-default 'price (if (arrayp .offers) (elt .offers 0) .offers)))))) ((re-search-forward "amazon.ca" nil t) (goto-char (point-min)) (re-search-forward "^$") (let ((doc (libxml-parse-html-region (point) (point-max)))) (list (cons 'name (dom-text (dom-by-tag doc 'title))) (cons 'description (dom-texts (dom-by-id doc "productDescription"))) (cons 'image (dom-attr (dom-by-tag (dom-by-id doc "imgTagWrapperId") 'img) 'src)) (cons 'price (dom-texts (dom-by-id doc "priceblock_ourprice")))))) (t (goto-char (point-min)) (re-search-forward "^$") (let* ((doc (libxml-parse-html-region (point) (point-max))) (result `((name . ,(string-trim (dom-text (dom-by-tag doc "title")))) (description . ,(string-trim (dom-text (dom-by-tag doc "title"))))) )) (mapc (lambda (property) (let ((node (dom-search doc (lambda (o) (delq nil (mapcar (lambda (p) (or (string= (dom-attr o 'property) p) (string-match p (or (dom-attr o 'class) "")))) (cdr property))))))) (when node (add-to-list 'result (cons (car property) (or (dom-attr node 'content) (string-trim (dom-text node)))))))) '((name "og:title" "pdp-product-title") (brand "og:brand") (url "og:url") (image "og:image") (description "og:description") (price "og:price:amount" "product:price:amount" "pdp-price-label"))) result) )))) (defun my-org-insert-shopping-details () (interactive) (org-insert-heading) (save-excursion (yank)) (my-org-update-shopping-details) (when (org-entry-get (point) "NAME") (org-edit-headline (org-entry-get (point) "NAME"))) (org-end-of-subtree)) (defun my-org-update-shopping-details () (interactive) (when (re-search-forward org-link-any-re (save-excursion (org-end-of-subtree)) t) (let* ((link (org-element-property :raw-link (org-element-context))) data) (if (string-match "theshoecompany\\|dsw" link) (progn (browse-url link) (org-entry-put (point) "URL" link) (unless (org-entry-get (point) "IMAGE") (org-entry-put (point) "IMAGE" (read-string "Image: "))) (unless (org-entry-get (point) "PRICE") (org-entry-put (point) "PRICE" (read-string "Price: ")))) (setq data (with-current-buffer (url-retrieve-synchronously link) (my-get-shopping-details))) (when data (let-alist data (org-entry-put (point) "NAME" .name) (org-entry-put (point) "URL" link) (org-entry-put (point) "BRAND" .brand) (org-entry-put (point) "DESCRIPTION" (replace-regexp-in-string "'" "'" (replace-regexp-in-string "\n" " " (or .description "")))) (org-entry-put (point) "IMAGE" .image) (org-entry-put (point) "PRICE" (cond ((stringp .price) .price) ((numberp .price) (format "%.2f" .price)) (t ""))) (if .rating (org-entry-put (point) "RATING" (if (stringp .rating) .rating (format "%.1f" .rating)))) (if .ratingCount (org-entry-put (point) "RATING_COUNT" (if (stringp .ratingCount) .ratingCount (number-to-string .ratingCount)))) )))))) (defun my-org-format-shopping-subtree () (concat "<style>body { max-width: 100% !important } #content { max-width: 100% !important } .item img { max-height: 100px; }</style><div style=\"display: flex; flex-wrap: wrap; align-items: flex-start\">" (string-join (save-excursion (org-map-entries (lambda () (if (org-entry-get (point) "URL") (format "<div class=item style=\"width: 200px\"><div><a href=\"%s\"><img src=\"%s\" height=100></a></div> <div>%s</div> <div><a href=\"%s\">%s</a></div> <div>%s</div> <div>%s</div></div>" (org-entry-get (point) "URL") (org-entry-get (point) "IMAGE") (org-entry-get (point) "PRICE") (org-entry-get (point) "URL") (url-domain (url-generic-parse-url (org-entry-get (point) "URL"))) (org-entry-get (point) "NAME") (or (org-entry-get (point) "NOTES") "")) "")) nil (if (org-before-first-heading-p) nil 'tree))) "") "</div>"))
At some point, it would be nice to keep track of how I feel about different return policies, and to add more rules for automatically extracting information from different websites. (org-chef might be a good model.) In the meantime, this makes it a little less stressful to look for stuff.