Rsync support!
Posted
Modified
Comments
0
What? Rsync for what?
I’ve had a request from Beaches (he hangs out in our IRC channel and has been on one or two of the shitshows) to be able to download all episodes at once.
While you can certainly script a python scraper using urllib and lxml or something to parse our RSS feeds, that certainly shouldn’t be necessary!
So I’ve opened a public rsync share of the episode media/signatures and exported versions of the shownotes. Note that it’s capped at 4 active connections at a time- if this becomes a more popular method, I can raise this cap. For the time being, if it conks out, just try again in a bit. :) We also host a mirror of the e-zine PoC||GTFO.
$ rsync rsync://rsync.sysadministrivia.com/
┌─────────────────────────────────────────────────────────────────────┐
│ ____ _ _ ____ ____ ___ _ _ _ _ _ _ ____ ___ ____ _ _ _ _ ____ │
│ [__ \_/ [__ |__| | \ |\/| | |\ | | [__ | |__/ | | | | |__| │
│ ___] | ___] | | |__/ | | | | \| | ___] | | \ | \/ | | | │
│ │
└─────────────────────────────────────────────────────────────────────┘
______ _______ ___ _ _____
| ___ / ___\ \ / | \ | / __ \
| |_/ \ `--. \ V /| \| | / \/
| / `--. \ \ / | . ` | |
| |\ \/\__/ / | | | |\ | \__/\
\_| \_\____/ \_/ \_| \_/\____/
.....
_ ___ ___ __ __ __
/_\ / __\/ __\/__/ _/ _\
//_\\ / / / / /_\ \ \\ \
/ _ / /__/ /__//__ _\ _\ \
\_/ \_\____\____\__/ \__\__/
___ __ _ __ _____ __ ___
/ _ \/__\ /_\ /\ \ /__ \/__\/ \
/ /_\/ \////_\\ / \/ / / /\/_\ / /\ /
/ /_\/ _ / _ / /\ / / / //__/ /_//
\____\/ \_\_/ \_\_\ \/ \/ \__/___,'
--
https://sysadministrivia.com/news/RSYNC
─────────────────────────────────────────────────────────────────────
archive Audio, GPG signatures, and extras for all episodes
notes Static versions of all episode notes (updated nightly)
pocorgtfo A mirror of PoC||GTFO (https://www.pocogtfo.com/)
$ rsync --no-motd rsync://rsync.sysadministrivia.com/archive drwxr-xr-x 4,096 2017/01/30 23:57:43 . drwxr-xr-x 4,096 2017/01/25 20:21:21 S0 drwxr-xr-x 4,096 2017/01/25 20:24:55 S1 drwxr-xr-x 4,096 2017/02/25 18:35:57 S2 drwxrwxr-x 4,096 2017/02/02 08:28:13 extras $ rsync --no-motd rsync://sysadministrivia.com/notes drwxr-xr-x 4,096 2017/03/05 00:59:03 . drwxr-xr-x 4,096 2017/03/05 00:59:28 html drwxr-xr-x 4,096 2017/03/05 00:59:28 markdown drwxr-xr-x 4,096 2017/03/05 00:59:28 pdf
(NOTE: you can use rsync with the --no-motd to skip the banner, but it won’t list the rsync modules – “shares” – if you do!)
How do I use it?
Episodes
If, for example, you’d want to download all of S1 (note that this includes the episodes in mp3 AND ogg format, as well as the GPG signatures), and save them locally to /srv/http/sysadmin_mirror, you’d do:
rsync -a rsync://rsync.sysadministrivia.com/archive/S1 /srv/http/sysadmin_mirror/.or
rsync -a rsync.sysadministrivia.com::archive/S1 /srv/http/sysadmin_mirror/.Shownotes
I also archive static copies of the shownotes in various formats- PDF, HTML, and Markdown. Note that they may be a little mangled from the live versions. They are freshly regenerated every night (morning?) at midnight UTC.
If you want all notes but only in Markdown format and saved to /srv/http/sysadmin_mirror, you’d do:
rsync -a rsync://rsync.sysadmininistrivia.com/notes/markdown /srv/http/sysadmin_mirror/.or
rsync -a rsync.sysadmininistrivia.com::notes/markdown /srv/http/sysadmin_mirror/.PoC||GTFO
If you want to simply grab all issues and download them to /srv/http/pocgtfo_mirror,
rsync -a rsync://rsync.sysadministrivia.com/pocgtfo/*.pdf /srv/http/pocgtfo_mirror/.or
rsync -a rsync.sysadministrivia.com::pocgtfo/*.pdf /srv/http/pocgtfo_mirror/.
Comments
There are currently no comments on this article.
Comment...