Merge branch 'dev' into vhs

data/base_16.md

@@ -1,6 +1,6 @@
 ---
 title: "Base 16"
-tags: [ "Data" ]
+tags: [ "data" ]
 ---
 
 ```bash

data/calcurse.md

@@ -5,5 +5,10 @@ tags: [ "data", "calendar", "daylight savings" ]
 
 ## Setup
 
-The UK government keeps an ics file with clock, [here](https://www.gov.uk/when-do-the-clocks-change/united-kingdom.ics).
+The UK government keeps an `ics` file with clock.
+
+```sh
+wget https://www.gov.uk/when-do-the-clocks-change/united-kingdom.ics
+calcurse -i united-kingdom.ics
+```
 

data/email.md

@@ -9,10 +9,10 @@ Install `opensmtpd` (or similar), then `ncat` or `nc` or `netcat` (this mysterio
 
 Start the `opensmtpd` service, then use netcat to speak with the mail-daemon:
 
-
-```
+```sh
 nc localhost 25
 ```
+
 The computer should respond with code `220`, which means 'I am listening'.
 
 > 220 hex ESMTP OpenSMTPD
@@ -40,7 +40,7 @@ Tell the mail daemon who you are in this format.
 
 Then tell it who you're sending to.
 
-```
+```sh
 RCPT TO: <www@dmz.rs>
 ```
 
@@ -67,7 +67,7 @@ You will find the email under `/var/spool` or `/var/mail` or similar.
 
 If unsure, just take a part of your email, like `FRAGMENT="turn off server please"`, then `grep` for it:
 
- ```bash
+ ```sh
 sudo grep -r $FRAGMENT /var/spool/*
  ```
 

data/exiftool.md

@@ -1,17 +1,17 @@
 ---
 title: "exiftool"
-tags: [ "Metadata" ]
+tags: [ "metadata", "exifdata" ]
 ---
 
-Find metadata.
+Find metadata:
 
-```bash
-exiftool image.jpg
+```sh
+exiftool "$file".jpg
 ```
 
-Find info on all images in current directory.
+Find info on all `.png` images in current directory.
 
-```bash
+```sh
 exiftool -ext .png .
 ```
 
@@ -19,22 +19,14 @@ You can make this recurring with the -r switch.
 
 And overwrite all metadata:
 
-```bash
+```sh
 exiftool -all= -overwrite_original -ext jpg .
 ```
 (NB: This does not work on pdf data. See [here](pdf_erasure.md) for erasing all pdf data)
 
 Or just GPS data:
 
-```bash
+```sh
 exiftool -gps:all= *.jpg
 ```
 
-You can also use the imagemagick tool:
-
-```bash
-identify -verbose
-```
-
-
-

data/git/git-lfs.md

@@ -1,24 +1,72 @@
 ---
 title: "git-lfs"
-tags: [ "data" ]
+tags: [ "data", "git" ]
 ---
 
-Install, and add with
+Git Large File Storage ('LFS') needs to change your `~/.gitconfig` to check out those binary files:
 
-```bash
+```sh
+cat ~/.gitconfig
 git lfs install
+cat ~/.gitconfig
 ```
 
-Then track some filetype with:
+Then track some filetypes with:
 
-```bash
-git lfs track "\*.ttf"
+```sh
+cd $git_repository
+ext=ttf
+git lfs track "*.$ext"
 ```
 
 Or a directory with:
 
-```bash
+```sh
 git lfs track "images/"
 ```
 
-All changes require adding `.gitattributes`.
+Track the changes to `.gitattributes`:
+
+```sh
+git status
+git add .gitattributes
+git commit -m "add $ext to lfs"
+```
+
+## Bash Completion
+
+If bash completion does not work, you'll have to add it:
+
+```sh
+git lfs completion bash |  sudo tee /usr/share/bash-completion/completions/git-lfs 
+```
+
+## Trouble Shooting
+
+You have some file "$FILE".png, which has some problem.
+
+Check the filetype:
+
+```sh
+file "$FILE".png
+```
+
+This should say the type is 'image'.
+If it says the type is 'text', then this file is really just a reminder to `git-lfs` to check out that file.
+
+Check `git-lfs` is expecting that file:
+
+```sh
+git lfs status
+git lfs ls-files
+```
+
+...then try these commands, and check the filetype again:
+
+```sh
+git lfs fetch --all
+git lfs fsck
+git lfs checkout
+git lfs status
+```
+

data/git/subtree.md

@@ -1,5 +1,5 @@
 ---
-title: "git"
+title: "git subtree"
 tags: [ "data", "git", "subtree" ]
 ---
 

data/gpg.md

@@ -1,6 +1,6 @@
 ---
 title: "gpg"
-tags: [ "data", "GPG" ]
+tags: [ "data", "gpg" ]
 ---
 
 - [Setup](gpg/basics.md)

data/gpg/extras.md

@@ -1,5 +1,5 @@
 ---
-title: "gpg"
+title: "gpg with vim"
 tags: [ "vim", "data", "GPG" ]
 requires: [ "GPG Basics", "vim basics" ]
 ---

data/groff.md

@@ -1,6 +1,6 @@
 ---
 title: "groff"
-tags: [ "Data" ]
+tags: [ "data", "logic" ]
 ---
 # Basic Documents
 

data/khard.md

@@ -4,57 +4,49 @@ tags: [ "Data" ]
 ---
 Get the basic config:
 
-```bash
+```sh
 mkdir ~/.config/khard
 ```
 
-```bash
+```sh
 cp /usr/share/doc/khard/examples/khard/khard.conf.example ~/.config/khard.conf
 ```
 
 Short list
 
-```bash
+```sh
 khard list
 ```
 
 Longer list
 
-```bash
+```sh
 khard show
 ```
 
 Show from addressbook 'work'
 
-```bash
+```sh
 khard list -a work
 ```
 
 Make a new contact in address book 'family'
 
-```bash
+```sh
 khard new -a family
 ```
 
-```bash
+```sh
 khard edit grampa
 ```
 
-```bash
+```sh
 khard remove bob
 ```
 
 Move contact 'nina' from 'work' to 'home' address book.
 
-```bash
+```sh
 khard move -a home nina -A work
 ```
 
-## Advanced
-
-Merge:
-
-```bash
-khard merge [-a source_abook] [-u uid|search terms [search terms ...]] [-A target_abook] [-U target_uid|-t target_search_terms]
-```
-

data/pass.md

@@ -1,42 +1,42 @@
 ---
 title: "pass"
 tags: [ "data" ]
+requires: "GPG Basics"
 ---
-[Video instructions](https://www.hooktube.com/watch?v=hlRQTj1D9LA)
 
 Setup [gpg](./gpg.md) keys.
 
 Show your gpg secret it:
 
-```bash
+```sh
 gpg --list-secret-keys
 ```
 
 Then use the id number under `sec` to make a pass repo:
 
-```bash
+```sh
 KEY="$(gpg --list-secret-keys  | grep -m 1 -A1 '^sec' | tail -n 1)"
 ```
 
-```bash
+```sh
 pass init $KEY
 ```
 
 To add a basic password, e.g. for `$WEBSITE`:
 
-```bash
+```sh
 pass $WEBSITE
 ```
 
 To insert a multiline password, e.g. with a login name:
 
-```bash
+```sh
 pass add -m $WEBSITE
 ```
 
 Remove a password:
 
-```bash
+```sh
 pass rm $WEBSITE
 ```
 

data/pdf-to-txt.md

@@ -2,7 +2,8 @@
 title: "pdf to txt"
 tags: [ "data", "pdf", "ocr" ]
 ---
-How to translate pdf book images to text (results are very poor, and will need lots of corrections).
+
+How to translate pdfs to text (results are very poor, and will need lots of corrections).
 
 ## Dependencies
 
@@ -17,8 +18,8 @@ pdftoppm -png *file*.pdf test
 ```
 
 ```bash
-for x in \*png; do
-    tesseract -l eng  "$x" - >> *out*.txt
+for x in *png; do
+    tesseract -l eng  "$x" - >> out.txt
 done
 ```
 

data/recfiles.md

@@ -44,6 +44,9 @@ recset -f "$new_field" --delete $database
 
 - [Extended example](recfiles/extended.md)
 - [Playing with board games data](recfiles/Board_Games.md)
+- [Playing with IP addresses](recfiles/IP_ASN.md)
+- [Manage LaTeX Bibliographies](recfiles/bibliography.md)
+- [Fixes](recfiles/recfixes.md)
 
 # Resources
 

data/recfiles/IP_ASN.md

@@ -0,0 +1,16 @@
+---
+title: "IP Addresses with Recfiles"
+tags: [ "data", "recfiles", "games" ]
+requires: "Recfiles"
+---
+
+## Download the Database
+
+Download the csv data, and separate the ipv4 data from the ipv6.
+
+```sh
+curl -Lo ips.zip 'https://www.kaggle.com/api/v1/datasets/download/ipinfo/ipinfo-country-asn'
+unzip -p ips.zip country_asn.csv | csv2rec | recsel -e "start_ip ~ '\.'"  > ipv4.rec
+unzip -p ips.zip country_asn.csv | csv2rec | recsel -e "start_ip ~ '::'"  > ipv6.rec
+```
+

data/recfiles/bibliography.md

@@ -0,0 +1,121 @@
+---
+title: "Recfile Bibliography for TeX"
+tags: [ "data", "database", "recfiles", "tex" ]
+requires: [ "Recfiles", "TeX", "Makefile" ]
+---
+
+Store your bibliography in a `recfile` database, then extract any part with `make`.
+
+For example, you could store books like this in `bibliography.rec`:
+
+```recfile
+%rec: book
+%key: slug
+
+slug: thinkingexperience
+author: H. H. Price
+title: Thinking and Experience
+year: 1953
+publisher: Harvard University Press, Cambridge
+
+slug: inventingrightwrong
+author: John Leslie Mackie
+title: Inventing Right and Wrong
+year: 1997
+publisher: Penguin Books, England
+
+```
+
+Run `make book` to extract `book.bib`, ready for LaTeX to use:
+
+```bib
+@book{thinkingexperience,
+   author = {H. H. Price},
+   title = {Thinking and Experience},
+   year = {1953},
+   publisher = {Harvard University Press, Cambridge},
+}
+
+@book{inventingrightwrong,
+   author = {John Leslie Mackie},
+   title = {Inventing Right and Wrong},
+   year = {1997},
+   publisher = {Penguin Books, England},
+}
+```
+
+The `makefile` syntax is just a few lines (though admittedly employs some garbled shell-crud):
+
+```make
+bibs != grep -Po '%rec: \K.*' bibliography.rec
+bibfiles = $(patsubst %, %.bib, $(bibs))
+
+$(bibfiles): %.bib: bibliography.rec
+	recsel $< -t $(basename $@) |\
+		sed 's/slug: \(.*\)/@$(basename $@){\1,/g' |\
+		sed 's/^\(\b.*\b\): \(.*\)/   \1 = {\2},/gI' |\
+		sed 's/^$$/}\n/g' > $@
+		echo '}' >> $@
+```
+
+Here's a longer `bibliography.rec` file, which can also produce `article.bib`:
+
+```recfile
+%rec: book
+%key: slug
+%type: year int
+%constraint: year > -2000
+%sort: year month
+
+slug: thinkingexperience
+author: H. H. Price
+title: Thinking and Experience
+year: 1953
+publisher: Harvard University Press, Cambridge
+
+slug: inventingrightwrong
+author: John Leslie Mackie
+title: Inventing Right and Wrong
+year: 1997
+publisher: Penguin Books, England
+
+slug: metaphysicscontemporaryintro
+author: Michael J. Loux
+title: Metaphysics: A Contemporary Introduction
+year: 1998
+publisher: Routledge, London
+
+slug: pluralityworlds
+author: David Lewis
+title: On the Plurality of Worlds
+publisher: Blackwell Publishing, Oxford
+year: 2001
+
+%rec: article
+%key: slug
+%sort: year month
+
+slug: genuinerealisttheory
+author: John Divers
+title: A Genuine Realist Theory of Advanced Modalizing
+year: 1999
+pages: 217–240
+month: april
+journaltitle: Mind
+uri: https://academic.oup.com/mind/article-abstract/108/430/217/975258?redirectedFrom=fulltext
+volume: 108
+publisher: Harvard University Press, Cambridge
+
+slug: twokindsmentalrealism
+author: Tam\'{a}s Demeter
+title: Two Kinds of Mental Realism
+year: 2009
+pages: 40:59-71
+uri: https://www.researchgate.net/profile/Tamas_Demeter2/publication/41554923_Two_Kinds_of_Mental_Realism/links/0deec53247f5a4ae21000000.pdf
+month: august
+journaltitle: Journal for General Philosophy of Science
+volume: 30
+publisher: Harvard University Press, Cambridge
+
+```
+

data/recfiles/recfixes.md

@@ -0,0 +1,33 @@
+---
+title: "Recfixes"
+tags: [ "data", "recfiles" ]
+requires: "Recfiles"
+---
+
+Sometimes `recsel` chokes on a large query, and you need to break the query into chunks with a pipe.
+
+This Kickstarter file has 374,853 records.
+Here's the chonky query:
+
+```sh
+recsel kick.rec -e "Category = 'Games'" -p "Subcategory,Avg(Goal)" -G Subcategory
+```
+
+It breaks down like this:
+
+| Chunk                         | Meaning                                       |
+|:-----------------------------:|:---------------------------------------------:|
+| `recsel kick.rec`             | Select records from `kick.rec`                |
+| `-e "Category = 'Games'"`     | Select only records where Category = 'Games'  |
+| `-p "Subcategory,Avg(Goal)"`  | Print the Subcategory and average goal        |
+| `-G "Subcategory"`            | Group by subcategory                          |
+
+Two ways to break the query apart:
+
+```sh
+recsel kick.rec -e "Category = 'Games'" | recsel -p "Subcategory,Avg(Goal)" -G "Subcategory"
+
+recsel kick.rec -e "Category = 'Games'" > games.rec
+recsel games.rec -p "Subcategory" -G "Subcategory"
+```
+