Welcome!

Welcome to Kirsle.net! This is the personal website of Noah Petherbridge, and it's where my various software projects and web blog lives.

Use Go as a Shell Scripting Language

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Tuesday, November 29 2016 @ 01:43:14 PM

A very long time ago, I stumbled upon this article "Use Java for Everything". While I disagree that you should use Java for everything (or any programming language, for that matter), the author mentions that he wrote a wrapper script that lets him use Java for shell scripts (ones where you execute the Java source file directly, without the "write, compile, run" steps).

I wanted to do something similar for Go, because I had a very simple Go program I wanted to be able to throw into my .dotfiles repo and run without needing to do too many things first: a simple static HTTP server.

In Python, there's a standard module named SimpleHTTPServer, where if all you want to do is quickly share some files from your computer, you could open a terminal and run:

$ python -m SimpleHTTPServer

...and it would make your current working directory available over HTTP at http://localhost:8000/.

The problem though is that the Python SimpleHTTPServer is single-threaded so it's very limited in its use case. For example if a user is downloading a large file, all other HTTP requests are blocked until that file completes. Somebody wrote a ThreadedSimpleHTTPServer, but that requires an extra pip install to get, and Python isn't very well suited at being an HTTP server in the first place.

Anyway, I wrote my own wrapper script for Go programs that I named gosh (for "go shell"). It lets me write my own version of SimpleHTTPServer using Go, without needing to compile it first. And since Go was designed to run HTTP servers, it does the job very well and with high concurrency. Here is my SimpleHTTPServer implementation. The only difference between this and a normal Go program is the shebang header at the top of the file, which tells it to use my gosh wrapper:

#!/usr/bin/env gosh
package main

// SimpleHTTPServer is a simple Go static file server, similar to the Python
// module of the same name, but which supports high concurrency and all the
// other niceties that you get from Go out of the box.
//
// It runs via my `gosh` wrapper for treating simple Go programs as shell
// scripts. See my `gosh` script, or just remove the shebang line at the top
// of this file to `go build` your own version.

import (
    "flag"
    "fmt"
    "log"
    "net/http"
)

// LogMiddleware logs all HTTP requests.
func LogMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        res := &ResponseWriter{w, 200}
        next.ServeHTTP(res, r)
        log.Printf("%s %d %s %s\n",
            r.RemoteAddr,
            res.Status,
            r.Method,
            r.RequestURI,
        )
    })
}

// ResponseWriter is my own wrapper around http.ResponseWriter that lets me
// capture its status code, for logging purposes.
type ResponseWriter struct {
    http.ResponseWriter
    Status int
}

// WriteHeader wraps http.WriteHeader to also capture the status code.
func (w *ResponseWriter) WriteHeader(code int) {
    w.ResponseWriter.WriteHeader(code)
    w.Status = code
}

func main() {
    // Command line flag: the port number to listen on.
    port := flag.Int("port", 8000, "The port number to listen on.")
    flag.Parse()

    fmt.Printf("Serving at http://0.0.0.0:%d/\n", *port)
    err := http.ListenAndServe(
        fmt.Sprintf(":%d", *port),
        LogMiddleware(http.FileServer(http.Dir("."))),
    )
    panic(err)
}

So with this I could put the SimpleHTTPServer script on my $PATH and just run it.

Here is the actual source of the gosh wrapper. I wrote it in Python, because I wanted to put it in my dotfiles and I don't want to check in compiled binary artifacts into my git repo. It's a very simple wrapper script that copies your Go source code into a temporary file (so it can remove the shebang header, and rename it to a .go file, to make it go runable without syntax errors), and runs it passing along any remaining command line arguments.

#!/usr/bin/env python

"""gosh: use Golang as a shell scripting language.

This is written in Python so that I don't have to commit any binaries to my
dotfiles repo, and my Go shell scripts can also remain in source form.

Usage: write a Go program with this shebang comment on top immediately before
the `package main` statement:

    #!/usr/bin/env gosh
    package main

And make it executable and run it like any shell script.
"""

import codecs
import os
import subprocess
import sys
import tempfile

def main():
    if len(sys.argv) == 1:
        die("Usage: gosh <file.go>")

    # Get the Go source file from the command line.
    source = sys.argv[1]
    argv   = sys.argv[2:]
    if not os.path.isfile(source):
        die("{}: not a file".format(source))

    # Make a temp file that lacks the shebang line of the input file.
    with codecs.open(source, "r", "utf-8") as fh:
        # Get the shebang off and sanity check it.
        shebang = fh.readline()
        if not "gosh" in shebang:
            die("{}: doesn't appear to be a Go script".format(source))

        # Write it to a temp file, sans shebang.
        temp = tempfile.NamedTemporaryFile(delete=False, suffix=".go")
        temp.write(fh.read().strip())
        temp.close()

        # Call it.
        subprocess.call(["go", "run", temp.name] + argv)

        # Clean up.
        os.unlink(temp.name)

def die(message):
    print(message)
    sys.exit(1)

if __name__ == "__main__":
    main()

The source codes to both of these things are available on my dotfiles repo:

Simulation Hypothesis

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Tuesday, November 08 2016 @ 12:28:52 AM

Besides computers and technology, something else I'm really nerdy about is science (like physics, astronomy and quantum mechanics), and something really fascinating that I admittedly don't understand is quantum physics. But I'm not going to talk too much about that on this post; instead this post will consist of more philosophical and theoretical musings related to it and what it might all mean. Some of it is my own; some is inspired by others.

First, a couple interesting experiments in quantum physics are the Double-slit Experiment, and its crazy cousin, the Delayed Choice Quantum Eraser experiment. I've read a fair bit about these and watched a handful of YouTube videos, but this video seems to sum them up in the most simple way I've seen; the quantum eraser one is fundamentally complicated to explain but that video does a fair job I think.

Delayed Choice Quantum Eraser Explained - YouTube
Delayed Choice Quantum Eraser Explained - YouTube

Another interesting topic is the Simulation Hypothesis, which contends that the universe could all be a simulation of some sort (think The Matrix). This theory goes by a few different names (Simulation Theory, Simulated Reality, etc.); I read this article recently, and you can Search YouTube if you're curious about it; pretty much all the videos I've seen have been pretty fascinating, but I'll just jump into the philosophical parts now. ;)

Life is but a dream...

Disclaimer: non-sciencey things will follow, and I'm not a scientist; this is all just my thoughts on how the weirdness of quantum physics could actually be consolidated into a potential explanation, and may even sound borderline "religious" (but I'm otherwise atheist).

An interesting facet of quantum physics is the effect that a conscious observer has on reality. When we're not looking, it seems that everything in reality exists as waves of potential that only collapse into discrete matter when being observed. So as an example, the Moon is extremely likely to be where it's at in the sky, but there's also an infinitely small chance it's a billion lightyears away, but it's probably there and every time we look it does appear there. If nobody's looking (or measuring with devices we can check back on later), does the Moon really exist? Theoretical quantum physicist Amit Goswami has a famous quote, "Consciousness does matter. Matter is secondary. Consciousness is primary. Brain does not do consciousness, consciousness does the brain."

Picture of text, "Consciousness does matter. Matter is secondary. Consciousness is primary. Brain does not do consciousness, consciousness does the brain."

What dreams may come

So, start with the plot of the book or movie What Dreams May Come. In the movie, Robin Williams' character dies and goes on to discover that Heaven is actually just a dream state, big enough for everybody to have their own private universes if they want to, or where like-minded people can collaborate on a shared space, and some people choose to be reincarnated back on Earth and live another life. That story alone gets you most of the way to my theory about how the universe might work.

In What Dreams May Come, it seems that the Earth is the central part of the universe and the dream world revolves around it (people who committed suicide on Earth go into a hell-like nightmare state in the afterlife; and in the real ending of the story, Annie had to reincarnate into a tough life on Earth to serve her punishment for it; you can see the alternate ending for the movie on YouTube).

My personal theory is that the Earth is not a central part of reality, and it's another dream world like all the others.

If you assume that the real state of the universe is dreams, where everyone could create their own place and fly around and have unlimited creative freedom (basically like "God Mode" in a videogame), you'd figure it'd get boring after a while. After several thousand years of playing Creative Mode you might like a challenge to spice things up.

So, a lot of like-minded individuals got together to create a game called "Earth" to entertain themselves. When you play Earth, you temporarily lose all your memories of the 'real world', are born into Earth and you're bounded by the laws of physics that everybody agreed on. Similarly to how multiple people could share a dream world by having a common vision, we all share the Earth world by agreeing to the rules of the game. We don't have "dream powers" to create matter out of nothing or defy the laws of physics, and we can't fly like Superman, because we made the rules of the game what they are.

But when you look at the Earth universe at a really low level, like at the quantum level, you see all this weirdness. It's a little bit like a dream: it creates itself as you explore it, and nothing is set in stone until it's been observed. When the Hubble telescope recorded the Hubble Deep Field image by looking at a tiny, tiny portion of the night sky and it discovered billions and billions of galaxies in just that tiny spec: did those galaxies already exist before we looked, or did they materialize when we looked? As if we were in a dream and opened a door into a new room and it materialized on-demand, so we feel like we're discovering the world rather than creating it consciously.

Simulation Theory, Again

One of the biggest questions that arises when you consider we might be living in The Matrix is, "What is the computer like that's running our simulation? What is its universe like? Is its universe also a simulation?" But I think the dream theory could answer that: there is no computer. All that exists is consciousness, and consciousness created this universe, and outside of this universe there's just more universes (dream worlds created by individual partitions of this consciousness). You don't have to worry if there's an infinite number of simulations going up and down the stack; Earth and the rest of the creative universe just exist side by side.

For that matter, if Earth was just a game created by bored minds, there could even be multiple such games. When you die, you return to the rest of the universe and could either sign up for reincarnation on Earth, spend some time in "Creative Mode" enjoying your own heaven, or sign up for a different game to entertain yourself with.

Fiction?

All of the above is probably not scientifically backed up, but it's pretty interesting and I like to think about it. I'm an atheist, so I don't believe in the heaven or hell that most religions talk about; when I die, I'll either just stop existing, or if I'm lucky I'll discover that things are similar to what I talked about above. At the very least, I don't think a heaven or hell as described by religions exists; if anything, Hell might just be a nightmare state in the afterlife, because if you messed up badly enough on Earth that you'd be deserving of Hell, your consciousness is probably at a very dark place on Earth already and it would make sense you'd punish yourself with a long nightmare afterward until you could eventually recover. Could take a very long time, though.

The author of What Dreams May Come says that the only fiction in the story are the stories of the individual characters involved, but that the overall idea of the way the universe works aren't his ideas and could be considered by some to be a religion of sorts. It seems a lot of people have come to similar conclusions, and I've further contemplated on these and this is a fascinating topic to discuss with my friends and I figured it'd be good to write this down on my blog.

I have more I could say on the matter, but this is a big enough wall of text for now. ;)

Music Management

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Thursday, July 28 2016 @ 08:37:46 PM

Manually managing a music collection of MP3 files on disk is such a pain in the ass that I felt like blogging about it.

First, you have cloud music services like Google Play Music which can't detect duplicates properly. The Google Music Manager will just keep re-re-re-redownloading my library from the cloud and creating duplicate files on disk over and over, with file names ending in (1).mp3, (2).mp3, etc., like it knows exactly what it's doing. Can't it just take a SHA-1 sum first and realize the MP3 I already have is exactly the same as the one they have on their side?

And then you have stupid media player programs that try to modify and reorganize your music library on your behalf. Even open source media players for Linux do this. What, do they think they're iTunes? I don't want them to touch my file system. So they end up creating duplicate songs, because they decided to update my MP3 and in the process they also decided to rename it, to be as compatible as possible with what, MS-DOS!? Only instead of just renaming the original, they copy it to a new name. And then the media player's library shows duplicate copies of the song.

On Linux I used to prefer to use XMMS as my media player. It's a Winamp 1.x clone and it has no opinions at all about your filesystem. But it was written with GTK+ 1.x and it looks ugly as sin on modern Linux desktops, and other forms of bit rot cause it to intermittently not work anymore on Fedora so I had to move on to other media players. I used to use Banshee but it seems broken atm on Fedora 24, so I'm using Clementine which really wants to touch my files and has a lot of feature bloat that I don't really need...

Nowadays I end up just tarring up my whole music folder and keeping it far, far away from where any media player will find it. So then I know I can always nuke my entire music collection and re-extract it from the tarball and be back to a known good state.

When I buy new music from Google Play, I nuke its entire downloads folder, have it download my entire library again from the cloud because it can't do anything more complicated than that, and then go in and pluck out the MP3's I just bought to put them in the tarball where they won't be touched again by any software that wants to mess with them.

I only have 192 songs in my collection so far. I can't imagine being one of those people with 20,000 songs, as problems like this would just compound themselves so much. 20,000 songs with 30,000 duplicate MP3's on disk?

And this is all to say nothing about missing/spotty ID3 tags on the files. That's the #1 reason I nuked my old collection of randomly downloaded music and started buying them back from Google and Amazon, with the assumption that the MP3s obtained from them will have full and accurate sets of metadata attached.

And why won't Clementine just shuffle my entire library instead of making me create a playlist that includes the entire library first? ffs.

XMMS apparently works again on Fedora 24

Looks like XMMS works for the moment, so I'm just gonna use that for now.

Also, Spotify is not an option. I prefer to own my own files.

Categories:

[ 1 comment | Add comment | Permalink ]

A review of Linux desktop environments

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, June 17 2016 @ 12:11:21 PM

The next version of Fedora (24) is coming out soon, so I decided a couple weeks ago that I'd take a tour of all the different desktop environments and see if I like any of them enough to switch from Xfce. My original desktop environment of choice was GNOME 2, and I had jumped ship to Xfce after GNOME 3 was released because I was no fan of the tablet-focused, feature-stripped interface of the new desktop and GNOME 2.32 was, in my opinion, the pinnacle of the desktop metaphor for Linux.

At the time I switched, Xfce was on version 4.8 and was a little bit clunky and somewhat of a step back from what GNOME 2 had to offer; however, I somewhat liked the Xfce panel better than the gnome-panel when it came to customization. In the GNOME 2 panel, you could drag and drop applets wherever you want them, but they'd sometimes "move around" if the panel or other applets changed size (for example, by increasing your screen resolution or by the Notification Area showing fewer tray icons than it did when you put your panel together). In Xfce, the placement of each applet was an explicit configuration, i.e. you'd need to insert a Spacer Applet set to expand to fill as much space as possible if you wanted left-aligned and right-aligned applets on the same panel.

I loved the GNOME 2 two-panel layout, so I always configured my Xfce desktops to match:

Xfce two-panel layout

This blog post won't be very screenshot heavy, but I link to the websites for all the desktop environments if you haven't seen what they look like, so you can check them out.

Here's how my desktop environment tour went.

Read more...

I've updated my blog!

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, June 03 2016 @ 03:30:02 PM

...and I don't mean by writing this post. ;)

I run Kirsle.net on a custom Python CMS I wrote called Rophako and I just added a bunch of improvements to the blog platform in it.

I've been wanting to start working on some larger blog articles, things that could take me a few days to write, and my blog had no support for saving drafts. You had to write and publish your blog posts in one sitting. The only alternative was to make the posts "private," where only the site admin(s) can see them, but that would still cause them to "go live" and be listed along with normal public posts and was a messy workaround.

So now my blogging platform has proper support for saving drafts; they stay invisible (even for admins) on all of the normal blog pages, and can only be seen in bulk on a dedicated "drafts" page. While I was putting all that together, I added similar support for private posts, giving them their own index too where you can see only those posts rather than have to go hunting for them.

For visibility purposes, draft and private posts get little tags next to their author/date lines so an admin can tell at a glance what they're looking at. Normal end users won't see those, though. ;)

I've also finally implemented a feature I wanted to from the beginning: sticky posts. The blog database schema has always had a "sticky" field, but it was always set to false and the Create/Edit Post page gave no way to change that value. The back-end code did have logic to bucket sticky posts higher on the index list than normal ones, but there was no (easy) way to make a post sticky to begin with.

Sticky posts will also get a little "[Sticky]" tag next to their author/date lines, and those ones can be seen by normal end users.

I also fixed a handful of smaller bugs/problems. I've removed the ability to set a custom date/time when editing a blog post. This caused confusion between your web browser's local time and the time zone of the server. So instead, the server always picks the latest UTC Unix time on new entries. When editing an existing entry, there's a checkbox option to reset the time to latest when saving, if you want to bump an old post to the top.

Categories:

[ 0 comments | Add comment | Permalink ]

Kirsle
Channels
Creativity
Software
Web Tools
Subdomains
Miscellany
Links


Fan Club