- 7
- 111 194
spudlyo
เข้าร่วมเมื่อ 9 ก.ย. 2006
Advanced Linux Tutorials
Literate Documentation with Emacs and Org Mode
All the materials for this talk can be found at: gitlab.com/spudlyo/orgdemo2
When writing about programming or other technical subjects, you’re often weaving blocks of source code, program output, and raw data in with your prose. These supplementary materials are usually copied and pasted into your document from other sources, which can be difficult and tedious to keep up-to-date as things change. Inconsistencies and errors can easily creep in when you “hard-code” dynamic information like program output into your writing.
Wouldn’t it be great if the tool you used for writing knew how to run code in a variety of programming languages, collect and format output, and let you refer symbolically to all this dynamically generated content in your prose? In this talk I’ll demonstrate how to use GNU Emacs’ Org mode to create technical documents that do just that. We’ll explore the features of Babel, Org mode’s literate programming add-on, that makes it convenient to edit, evaluate, and manage embedded code, output, and data all from inside GNU Emacs.
We'll also show how these literate documents can be exported to LaTeX and ultimately PDF format to create professional looking output that looks stunning when printed or viewed.
Note: This is an updated version of the talk I gave at SeaGL 2019 that features new techniques and workflow improvements I've developed over the intervening years.
When writing about programming or other technical subjects, you’re often weaving blocks of source code, program output, and raw data in with your prose. These supplementary materials are usually copied and pasted into your document from other sources, which can be difficult and tedious to keep up-to-date as things change. Inconsistencies and errors can easily creep in when you “hard-code” dynamic information like program output into your writing.
Wouldn’t it be great if the tool you used for writing knew how to run code in a variety of programming languages, collect and format output, and let you refer symbolically to all this dynamically generated content in your prose? In this talk I’ll demonstrate how to use GNU Emacs’ Org mode to create technical documents that do just that. We’ll explore the features of Babel, Org mode’s literate programming add-on, that makes it convenient to edit, evaluate, and manage embedded code, output, and data all from inside GNU Emacs.
We'll also show how these literate documents can be exported to LaTeX and ultimately PDF format to create professional looking output that looks stunning when printed or viewed.
Note: This is an updated version of the talk I gave at SeaGL 2019 that features new techniques and workflow improvements I've developed over the intervening years.
มุมมอง: 4 009
วีดีโอ
Hello World for Songwriting
มุมมอง 4533 ปีที่แล้ว
This is a recording of a talk I gave at work for a lecture series designed to give folks a new hobby to engage with during the pandemic when everyone is stuck at home. Since this video is really long, I've put some chapter markers in so you can jump around to sections that might be of interest. 00:00 Introduction 00:43 Time: Bars & Beats 05:22 Our Song's Key 07:59 The Arrangement 12:00 Adding D...
Features of a Modern Terminal Emulator
มุมมอง 7K4 ปีที่แล้ว
This is a prerecorded talk of a presentation I gave on November 13th 2020 at the SeaGL conference. I hope you like it!
Consistent Technical Documents Using Emacs and Org Mode
มุมมอง 59K5 ปีที่แล้ว
Files used in the demo: gitlab.com/spudlyo/orgdemo My dotfiles: gitlab.com/spudlyo/dotfiles Read The Org: github.com/fniessen/org-html-themes Read The Docs: sphinx-rtd-theme.readthedocs.io/en/latest/installing.html Git from the Bits Up: th-cam.com/video/MYP56QJpDr4/w-d-xo.html
Spying on Apache and PHP with Strace
มุมมอง 24K13 ปีที่แล้ว
This video shows how to use strace to determine why an apache worker loads a page slowly. Commands shown in this video: telnet localhost 80 lsof -p `pidof telnet` netstat -nap | grep pid strace -o /tmp/output -f -s4096 -r -p pid cat /tmp/output | cut -c12-16 | sort -rn | head
Getting More out of Top
มุมมอง 16K13 ปีที่แล้ว
How to configure top(1) to show you interesting information. Some of the keys used in this video: z Turn on color H Show threads d Set the delay 1 Show SMP CPU info i Show only non-idle processes/threads f Add fields to your top display F Pick which field to sort on Handy things I missed: W Write out your config to a .toprc file The watch command I used was: watch -d cat /proc/interrupts
You're very good at this. I hope you continue
Very good
00:39 #+title: - in-buffer setting 01:13 what is org mode? 01:44 formatting syntax 02:18 org-export-dispatch (C-c C-e) as html 02:28 #+setupfile: - in-buffer setting 03:29 what is org babel? 03:35 #+begin_src and #+end_src - org babel code block 03:52 org-ctrl-c-ctrl-c (C-c C-c) execute org babel code block 04:28 :dir - code block header argument 04:48 #+property: - in-buffer setting; sets file-level header arguments 05:07 #+include: - in-buffer setting 05:23 # -- mode: -- - local variables in files 05:30 #+name: - code block directive 05:38 :exports - code block header argument 06:04 #+call: - in-buffer setting 06:22 why redirect stderr to stdout? 07:15 :results output - code block header argument 08:59 call_hello() - inline call to code block (prefix with "call_" and append the #+name: of the code block) 09:20 call_hello[:results raw]() - header arguments in inline call () 09:41 :noweb - code block header argument 10:36 :var - code block header argument 11:09 call_short_hash(line=hashobject) - inline call with argument 13:42 :noweb - call with argument 15:49 terminal vs gui - font size 16:35 org-edit-special (C-c ') 16:54 company-mode - emacs third-party package 17:28 :results - code block header argument 18:52 custom org publish function 21:06 org-publish 22:20 final thoughts on confluence editor
Hi Mike, well, that was mind blowing! Quick q if you don't mind: How do you switch between those VMs so quickly?
That’s the combined powers of SSH and Tmux!
Would have been helpful if you would have said which command you used to generate the HTML file @ 2:24.
The command is org-export-dispatch (C-c C-e)
Awesome demo, thanks! What's the font? Where can I download the background image from?
That's great, again! Is there any chance you could discuss a bit more around export/dispatch customisation? Ie, having this great document, we'd like probably to publish it on our server (ftp?) where some css-es already are present. Or, perhaps, we have our WordPress blog and we'd like to publish this technical document as a post? Or publish it in confluence? Many thanks in advance. K.
Many thanks Mike! Another great video about showing what pinacle of automation is possible at our fingertips (literally) and how much time we could save with pure, simple and ethernal tools like emacs, grep, LaTeX and other "stone age" non M$ Windows software. In your next video 😀may we ask you to discuss approach like this: A tech guy/lass has to write an article/documentation. In his/her task the data from the web has to be consumed, the graphs, tables have to be created, and along writing this article a few presenation slides (with the company marketing stuff like logo, bacground etc.) need to be created to "sell" the article in the meeting. It seems natural that during the article writing we could put some org-mode and LaTeX directives around fragments of the article, and using the dispatcher/exporter we could have and the article and the slides from one org file. Pretty popular scenario, isn't it?
Having both slides and a document being sourced from the same org file seems like a great idea! I'll have to try that out.
Just watched a 4 years old video of yours and then found this one. I just love the unassuming professionalism that your videos emit. I remember when I first installed Emacs from source sometimes around 1990 on an SCO UNIX (SCO didn't have a bad name at the time, it was just incredibly expensive - was lucky to get a copy from my part time employer then). SCO was not supported by the configure scripts, at least the development suite wasn't. After diving into this shell script you mentioned I learned that it's easier to first get a gcc. The bootstrap gcc didn't work with SCO's C compiler either, so I had to find a prebuild binary, which was incredibly hard at the time. The whole process took about a week, until I got emacs (as well as most of the GNU ecosystem, gmake, bison, flex, gm4, gawk, ...) to start the first time. Considering what I had to do, that was actually pretty fast. It kinda closes a loop, seeing how amazingly you employ emacs to create beautiful stuff. Org mode is amazing. That in turn reminds me on my attempts to use tangle and weave. Surprising how all the old tools and concepts seem to enjoy a revival in recent years. Make an old fart like myself feel modern and fancy :-)
Great talk and work! Thanks very much for sharing your experience with us!
Ok, this was genius. Thank you!
Great video! I've been working with tangled documents since '95 or so, and still find a lot of good advice here. Will certainly bookmark this one.
Great presentation, One question though: I followed you code samples and event downloaded the gitlab repo but pdf export does not actually work: Maybe you are using something other than the default org-latex-pdf-process command - - ie. latexmk -f -pdf -%latex -interaction=nonstopmode -output-directory=%o %f - ?
Incredibly slick! I'm doing my best to learn and practice the literate programming approach, and your two videos have been amazingly helpful demos. Right now I'm trying to go through Advent of Code with elisp + a literate programming style.
Really enjoyed the new tips presented. I’m saving this to refer to later, like your other video.
that theme are you using? In config file I have found this (load-theme 'modus-vivendi) but it's not that theme.
That's Zenburn with a darkened background.
extremely well prepared video tutorial - amazing job!
This is easily the best video on literate programming with Org Mode on the internet.
👍👍
awesome. thank you sir!
Thanks for the great video!
you can specify the file to cut instead of using cat and redirecting the output (and by using tail instead of head I don't have to type that one extra 'r': cut -c12-16 /tmp/output | sort -n | tail very cool demo though - I'd no idea about the -s argument that strace has - thank you!
Glad you liked it! It's amazing that folks are still getting something out of a 10 year old video. I continue to employ the "useless use of a cat" paradigm simply because I like pipes, and I find the explicit cat satisfying.
@@elspuddo How do you deep dive into concepts? What's your process in understanding stuff?
@@masterchief1520 It's mostly a mix of reading and then trying to put what I've read into practice. I understand things a lot better after I try to use that knowledge for something practical. Writing about a topic in order to explain it to someone else is also super helpful for gaining a better understanding. This helps reveal things that you might have glossed over and only have a superficial understanding of.
I went ahead and downloaded your video for backup 🙂
Damn cool topic and range of information covered, might have taken me years to get around it and then see the bigger picture but this what proper knowledge sharing feels like. Thanks a lot🤘
Thanks. I've been using terminals sice 1994 (VT220 ...) and I still live in terminals (and multiplexers like tmux). Still a cool presentation, thanks!
where is the complete video?? link
Awesome!
I like the video even before I watched it!
Are there conventions within the community regarding where to put those icons within the private use area?
Hi Mike, I am slowly following your tuto, I am now @13:50 and I was wondering if you could explain what the =file - = does. I looked at the file utility help page but I don't think it mentions the =file - = , thanks!
A lot of UNIX utilities that take a filename as an argument will accept a dash ("-") to mean "read from STDIN instead". That's why it says "/dev/stdin:" at the start of the output. Hope this helps!
@@elspuddo thank you ( I thought I had answered your kind explanation last year but I had not, sorry). This video is still a great reference for me, amazing job Mike
Killed it! Thing about Emacs and Org is there is always something better and new to learn. Especially from content published few years back, and just when you think there is nothing more to learn about these two....lol. Thanks
@9:58 Hi Mike, I followed along but when I try to echo the result of invoking the =hello= function (with "<<hello()>>") I am not getting the proper result, I am only getting the string "<<hello()>>". When within the code.inc file I can invoke the function hello and access the right message. Any idea? I tried to add the header-args ":noweb yes" but that did not help either. Cheers,
Check out the function "org-babel-lob-ingest" which tells org to add the function in that file to the "Library of Babel". If you look at my config, I have a function called "my-maybe-lob-ingest" which I run as an after-save-hook for Org mode. This makes it so when I save code.inc it automatically adds it to the LOB so you can call hello() from your main document. You'll notice @8:57 when I save it says "2 src blocks added to Library of Babel", that's my after-save-hook.
@@elspuddo gotcha, I will use the LOB from now on thank you. Yesterday I finally understood that I could also from <<code.inc:hello()>> and I was quite proud of myself haha, but yes LOB is easier. Thank you for the feedback. I have left issue on your github I was not sure which way you prefer using when giving feedback. Whichever way you prefer, cheers!
@@MMABeijing Very cool, I didn't know you could do that!
Really nice presentation. Good work!
Hi Mike, I am trying to follow that tutorial, however my orgmode setup is different from yours and your results do not match what I get. Ex: @4:02 when you execute your codeblock within your results there is a nested begin_example, I don't have that. WHen you export in html your results get exported and not mine. Can you point me in the right direction? thank you
Hey, sorry for the confusion on this. In my Emacs configuration I have manipulated the "org-babel-default-header-args" to change the default header args supplied to every code block. You can manually get the same results as me by doing: #+BEGIN_SRC shell :results output verbatim replace echo "My current working directory is: $(dirs +0)" ls -l #+END_SRC Hope this helps!
@@elspuddo It's not working for me :) Right now I m only using fniessen set up file to export to html. I tried your code but it won't output the name of my current file (it does it in orgmode, but not on html). Same thing with the output the codeblock: it is not showing within the html page.
update: might be a bug, but in my case the only way to displaying the output of the block is to make sure there is at least a text character between the babel codeblock and the output. Weird:)
@@elspuddo I really want to follow along your instructional, do you see any pb with the code below? #+NAME: hashobject #+begin_src sh :exports both :results output verbatim replace echo -n '<<hello()>>' | git hash-object --stdin -w #+end_src #+RESULTS: hashobject : 7435a4d4bbd4c88971f2b1929491abd8ca9da3bb ------------------ I don't understand why I am not able to export to html, am I missing something out?
Great talk, thanks!
My brain melted. Not very good brain
@14:41 "Looks good though" Quite the understatement!
Love it !! Best summary I've seen ;)
I live in the terminal my self. I really really like to do anything I can in the terminal from web browsing, email, reddit, discord, games like nethack to my everyday workflow of programing. I enjoy making things in the terminal over a GUI as well do to the limitations it brings make for a good challenge as a programmer. I haven't been much into Emacs my self but more of a Vim dude. I think both are good though, way better then the SUPER Clunky IDE's. Thanks for sharing your knowledge! The One thing i wish that the terminal supported with ESC Code's was a way to change the characters width and height. You can do this on windows with the help of the windows.h file but i wished it was a slandered. Would change terminal gaming a lot!
use let instead of setq in the my-publish function that would affect only locally
Dude, awesome job. Keep the good stuff coming...
Ok I want to pick you brains on your video editing :) what did you use to create the sound of the keyboard (unless you actually recorded your keyboard, but somehow I doubt it), also could you tell me what you used to speed up the input at certain moments of the video? thank you forehand!
I recorded the on-screen portions of the video separately from the narration, but I did have my microphone active (and my breath held) while I did it, so it picked up the sounds of my keyboard. I later recorded the narration separately and then edited it into the video timeline.
@@elspuddo did you use something special to have the fast-typing effect, or did you simply sped it up with your editing software? (btw which sw do you use for video edition?). But yeah you have a very interesting delivery, I am subbing to your channel just in case you come back to making more videos :), and yesterday night for the first time I tried to use erc/irc as you mentionned in the comment that's where people can find you. I have yet to register my nick properly, I am very new to that, but for what it 's worth you opened one person's eyes to irc, so thank you for that as well:)
The fast-typing comes from my keyboard, a Kinesis Advantage 2, which has a macro record/playback capability. So when it's going fast, I'm hitting my macro playback key. I use ScreenFlow on macOS for all my screen capture and video editing. I may try out an all Linux workflow, as I'm interested in trying out Kdenlive. The #emacs channel on Freenode is pretty great, there are very knowledgeable people there!
Haha, I waded into the comments specifically to find anyone talking about the keyboard sound. I was pretty sure I recognized it just by the sound, and indeed I did… I have the Kinesis Advantage 2 as well. It's not just the click of the switches, but the difference in tone on the thumb pads, I think.
@@elspuddo belated thank you for the recording tip, I managed to not notice it until today and really appreciate the information. :pray:
thank you YOutube, thank you Sir. You ability to story-tell and the quality of your video/sound made viewing that video as enjoyable as watching a short movie. You Sir had me on the edge of my seat and I learnt a lot as well. You are my new reference in terms of emacs. I am just gonna go ahead and sub/like, I am sure the rest of your channel rocks. Again, thank you !
I'm glad you liked it! These days I mostly make videos of the presentations I give at the yearly SeaGL conference, but I hope to stop being lazy and to make more at other times of year too :)
Didn't know that "modern" terminal emulators were so capable. Very useful.
256 colors is enough for basically anything, tbh, truecolor is mostly overkill... unicode is incredible, as long as you're not dealing with magic parts of it very sad sixel and such aren't supported tho, but there's drawille ;P abusing braille to show graphs and kinda show images (contrast can easily be seen if you try out "mapscii" and switch between drawille and normal box drawing), the "st" terminal has a patch to make braille characters fill up the cells (and one to make box drawing chars touch exactly, with no spaces, way more aesthetic), so it looks a lot better, you could also do that with a custom font
I would've loved to see you go through reGIS.
Wonderful talk and very entertaining too. If I am allowed some for of negative criticism, I would have loved to see the Tektronix terminal mentioned. Xterm has support for it still (4010). Most people hate it, since it is a very obsolete technology, but it is pretty cool none the less. And one can do some interesting plots/cad with it. Anyhow, very very nice one
I actually had a whole section on the Tektronix 4010, and some examples of using it with gnuplot, but sadly it had to be cut for time. I also think it's pretty cool.
@@elspuddo you could've made the youtube version extended, with a marker that "these parts were cut from the talk", or such ;P
@@elspuddo If you are willing to make that part for youtube, I am willing to watch it :) There is not much visual (videos) media from the Tek terminal. I love playing with the gnuplot samples, they are indeed very cool. If they were drawn faster, they could be usabe in today's world :)
Great video! Funny, I was just writing some 24-bit/truecolor to "xterm" and 16c mapping utility code in Node.js today as I ran across this!
20 years in the terminal - and still learned sth. Great as all talk of yours (the emacs lit programming one DOSed me for a month, now I'm in the church).
what does "DOSed me " mean? does it mean it was so overwhelming that it left you unable to do react?
I've been using Emacs and org for almost a year now and I've watched a lot of videos and tutorials. This is among the best ones around for this topic. Thank you a lot, I was looking forward to it.