Welcome!

Welcome to Kirsle.net! This is the personal website of Noah Petherbridge, and it's where my various software projects and web blog lives.

Fedora 21 on the 2015 Macbook Air

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Saturday, May 02 2015 @ 08:06:37 PM

Today I picked up a Macbook Air (13", early 2015 model) because I wanted a new laptop, as my old laptop (the Samsung Series 5) has a horrible battery life, where it barely lasts over an hour and gives up early (powering down at 40% and not coming back up until I plug it in). This is also my first Apple computer. I'm the furthest thing from an Apple fanboy, but the choices I was throwing around in my head were between an Apple computer and a Lenovo Thinkpad.

I was given a Thinkpad as my work laptop, and it's by far the most impressive PC laptop I've ever used; it can drive three displays and run lots of concurrent tasks and has an insane battery life. Every PC laptop I've owned in the past have sucked in comparison. I hear people compare Apple computers to Thinkpads, so that's why the choice came down to one of these, and I didn't want another Thinkpad sitting around the house. ;)

Months before getting a Macbook I was looking into what kind of effort it takes to install Linux on a Macbook. There's a lot of information out there, and most of it suggests that the best way to go is to install a boot manager like rEFIt (or rEFInd, since rEFIt isn't maintained anymore). I saw some pages about not using rEFIt and installing Grub directly, which were from a Debian and Arch Linux perspective, and it sounded really complicated.

It seems that nowadays, with a user friendly Linux distribution like Fedora, a lot of this works much more flawlessly than the dozens of tutorials online would suggest. I just made a Fedora LiveUSB in the usual way (as if installing on a normal PC), rebooted the Macbook while holding the Option key, so that I was able to select the USB to boot from.

When installing Fedora to disk, the process was very much the same as doing it on a normal PC. I let Fedora automatically create the partition layout, and it created partitions and mount points for /, /boot and /home like usual, but it also created a partition and mount point for /boot/efi (for installing itself as the default bootloader in the EFI firmware on the Macbook). After installation was completed, I rebooted and the grub boot screen comes up immediately, with options to boot into Fedora.

One weird thing is, the grub screen apparently sees something related to Mac OS X (there were two entries, like "Mac OS X 32-bit" and "Mac OS X 64-bit", but both options would give error messages when picked).

If I want to boot into OS X, I hold down the Option key on boot and pick the Macintosh HD from the EFI boot menu. Otherwise, if the Macbook boots normally it goes into the grub menu and then Fedora. So, the whole thing is very similar to a typical PC dual-boot setup (with Windows and Linux), just with one extra step to get into OS X.

RiveScript Rewritten in CoffeeScript

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Wednesday, April 22 2015 @ 07:47:51 PM

I've spent the last few months porting over the JavaScript version of my chatbot scripting language, RiveScript, into CoffeeScript. I mainly did this because I didn't like to maintain raw JavaScript code (especially after I ran a linter on it and had to rearrange my variable declarations because JavaScript's variable scoping rules are stupid), but I also took the opportunity to restructure the code and make it more maintainable in the future.

For some historical background, the first RiveScript implementation was written in Perl, and I designed it to be one big monolithic file (RiveScript.pm), because then I could tell noobs that they can update to a new version of RiveScript in their bots by just dropping in a new file to replace the old one, and avoid overwhelming them with the complexity of doing a CPAN installation (especially if they're generally new to Perl as well, and all they really wanna do is just run a chatbot.)

Source File Restructuring

The Python and JavaScript ports are more-or-less direct ports of the Perl version: I literally read the Perl source from top to bottom and translated it into each respective language. So, there was rivescript.js which was a huge 2,900 line long wall of JavaScript and one of my goals in the refactor was to spread logic out into multiple files, so if I have a bug to fix in the reply matching code, I don't have to scroll through pages and pages of loading/parsing code to get to the relevant part.

The new file structure is like this:

  • rivescript.js - The user-facing API, has all the same public functions as the old one
  • parser.js - A self-contained module that loads RiveScript code into an "abstract syntax tree" (a JSON serializable blob that represents ALL of the parsed RiveScript code in a program-friendly format).
  • sorting.js - The rat's nest that is the implementation behind sortReplies() is contained here.
  • inheritance.js - Functions related to topic inheritance/includes are here.
  • brain.js - The code that actually gets a reply is here.
  • utils.js - All those miscellaneous internal utility functions, like quotemeta() and such.
  • lang/javascript.js - The implementation for JavaScript object macros in your RiveScript code.
  • lang/coffee.js - This is new - you can use CoffeeScript in your object macros (it's not enabled by default, you'll probably have to snipe lang/coffee.js and include it in your own project for now, but the built-in shell.coffee uses it out-of-the-box).

Logic Refactoring

Another thing that was messy in the Perl, Python and JS versions of RiveScript was how it lays out its internal data structures in memory. If you'd ever run the equivalent of Data::Dumper you'd see reply and trigger data appearing in multiple places depending on whether it had a %Previous tag on it or not.

It looked something like this:

$RiveScript = {
    "topics" => {
        # Most reply data is under here, BUT NOT triggers with %Previous!
        "random" => { # (topic names)
            "hello bot" => { # (trigger texts)
                "reply" => { # (replies to the trigger)
                    0 => "Hello, human!",
                    1 => "Hi there!",
                },
                "condition" => { # (conditions)
                    0 => "<get name> != undefined => Hello, <get name>!",
                },
                "redirect" => undef, # (if there's an @redirect)
            },
            # ...
        },
    },
    "thats" => {
        # This is like 'topics' but ONLY for triggers with %Previous
        "random" => { # (topic names like before)
            "who is there" => { # (the %Previous text)
                "*" => { # (trigger text)
                    # then things were like the above
                    "reply" => { 0 => "<sentence> who?" },
                    "condition" => {},
                    "redirect" => undef,
                }
            }
        }
    },

    # And then sorting! Again the 'normal' replies are completely
    # segregated from those with %Previous
    "sorted" => {
        "random" => [ # (topic names)
            "hello bot", # (triggers, sorted)
            "*",
        ],
    },
    "sortsthat" => {
        # This one is simply a trigger list for ones that have %Previous
        "random" => [ # (topic names)
            "*",
        ]
    }
    "sortedthat" => {
        # This one is, in case one %Previous has more than one answer,
        # if we ONLY had the above sort we'd overwrite the first answer
        # with the second.. this one keeps track of all "replies" with
        # the same %Previous
        "random" => {
            "who is there" => [
                "*",
            ]
        }
    },
}

As you can see, it was a mess. There were a ton of different places where reply data was kept, and triggers had to be sorted many different ways for various edge cases. Additionally, keeping track of which topics inherit or include others was kept in a separate data structure from the topics themselves!

In the new refactor of RiveScript-js I eliminated as much duplication as possible. Now, the entirety of the reply base exists under the topics key, and instead of using lots of nested dictionaries, e.g. ->{topic}->{trigger}->{reply} which had the issue of one trigger overwriting the data for another if they both happened to have the same text (e.g. when you have two answers to the same %previous question), the ordering of the triggers is preserved. The data structure ends up looking like this:

{
    "topics": { // main reply data
      "random": { // (topic name)
        "includes": {}, // included topics
        "inherits": {}, // inherited topics
        "triggers": [ // array of triggers
          {
            "trigger": "hello bot",
            "reply": [], // array of replies
            "condition": [], // array of conditions
            "redirect": "",  // @ redirect command
            "previous": null, // % previous command
          },
          ...
        ]
      }
    }
}

Additionally, whenever the code refers to reply data (for example, in the sort buffers and the %previous tree), it refers to a specific index in the singular topic structure for the reply data. This has the other side benefit that, while getting a reply for the user, when a matching trigger is found it already has the pointer to that trigger's responses right away. It doesn't have to look it up from the central topic structure like before (and since the replies are kept on an array, this would be impractical now anyway).

Keeping the replies on an array also naturally takes care of the issue with multiple responses to the same %previous without needing a third sort buffer (we still do need a separate sort buffer for %previous replies themselves, though).

Other Implementations

The Perl and Python versions probably won't get updated anytime too soon to fit this new structure, but any future ports of RiveScript to other languages that I work on will be based off this new CoffeeScript version. I have some vague plans right now to port RiveScript over to Google's Go language, and I wanted to get the refactor out of the way first so I have a new "template" to reference when writing a new port.

So, Why CoffeeScript?

To elaborate more on why I rewrote it in CoffeeScript instead of just doing this restructure in Node-style JavaScript:

  • A while back I started JS-linting the JavaScript codebase and fixing all of the mistakes it pointed out.
  • JavaScript has really weird variable scoping issues, where all variables declared within a function, no matter where it's declared, gets implicitly "hoisted" to the top of the function as though you had declared it up there instead. Sane programming languages have lexical scoping where variables declared inside of a { block } of code (including loop variables, i.e. for (my $i = 0; ...)) die with that block's closing brace, for example Perl does it this way.
  • In numerous places in my code, I would have a long function that does a loop over "something" in more than one place, always doing for (var i = 0; ...). Declaring var i multiple times in the same function is the error, so I had to take all the reused variable names and make this ugly, var i, iend, j, jend, match; at the tops of functions.
  • CoffeeScript has a clean syntax compared to JS (it looks quite like Python), and it automagically handles your variable declarations. You simply use a variable (like Python), and CoffeeScript handles when and where to declare it in the JavaScript output.

As for all the haters that say things like, "ECMAScript 6 makes CoffeeScript obsolete because it adds classes and the arrow operator and everything else into the core JavaScript language": I see no reason at all that CoffeeScript can't one day compile into ECMAScript 6 the way that it does into ES5 right now, so it's not like CoffeeScript is a dying language that will no longer be maintained in the future. In a worst case scenario, I could program a CoffeeScript to ES6 compiler myself if nobody else will. It's not as if I have no experience writing scripting language parsers. ;)

And with GitHub backing it (they use CoffeeScript to program my favorite text editor, Atom), the language isn't going anywhere anytime soon.

Linux Desktop Remote Code Execution via File URI

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, March 27 2015 @ 09:04:08 PM
I've discovered a sort of "remote code execution" vulnerability that affects all Linux desktops, particularly Fedora and Ubuntu but most likely all desktop Linux distributions could be affected, except for maybe Arch or Gentoo with extremely customized installations.

First and foremost: this requires the victim to click not one, but two random links sent to them over Pidgin (or any other program that does URL auto-linking the way Pidgin does). So it's not exactly the most severe vulnerability, but I found it interesting nonetheless.

Read more...

Minecraft Map: Swampcore

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, March 06 2015 @ 05:12:00 PM

I've created a downloadable Minecraft map that implements "Swampcore" (older blog post about Swampcore). It's a superflat swamp biome preset with a 24/7 thunderstorm, making for an extremely hostile map where you have to scratch and claw your way into having so much as a simple dirt shack to call "home".

It runs on vanilla Minecraft (no mods or anything needed) and was created on Minecraft v1.8.3, but will probably work on later versions too.

Superflat Preset

This is the preset code that was used to generate the map:

3;9*minecraft:bedrock,minecraft:dirt,minecraft:grass;6;biome_1,decoration,lake,lava_lake

From the bottom up, you have:

  • 9 layers of bedrock
  • 1 layer of dirt
  • 1 layer of grass

World Features

The superflat swamp includes lakes and lava lakes, oak trees, vines, sugar canes, mushrooms and tall grass. The water lakes can sometimes spawn with sand or clay, and the lava lakes spawn with stone around them. The stone can occasionally be an ore block instead (such as iron or redstone). Diamonds are able to spawn around the lava lakes as well (world surface is at Y=11) but are extraordinarily rare.

There is a 24/7 thunderstorm which keeps the sky dark and prevents zombies and skeletons from burning up during the day, and even allows for hostile mobs to spawn during the day. Players will frequently get killed while just trying to build a cheap dirt shack to live in.

The only stone on the map is around lava lakes, so players will need to loot a lava lake in order to build a furnace or stone tools. The main source of coal will be from burning wood logs in a furnace, as coal ore will be extremely rare. Zombies can rarely drop iron ingots when killed, or players can try to find iron ore around lava lakes.

The nether portal can be built and activated, but since diamonds are practically nonexistent players would have to build a mould for a portal and pour lava and water in using buckets to slowly build the frame. Since gravel is extremely hard to find (impossible?), no Flint & Steel, so players would have to use lava with wood planks to set a fire near the portal and wait for it to spread into the frame and activate it (pretty tedious).

With the presence of sugar canes and access to the nether, splash potions of weakness and golden apples can be crafted and a zombie villager could be healed, so players can build their own NPC village. With villagers players could trade for diamond gear and lots of other items that are otherwise impossible to get on this world.

In short, it's entirely possible to get to the late game on this world (the main thing missing is access to the End Dimension, since no strongholds exist, but that's probably for the better in a multiplayer server anyway). It's just very, very difficult. Even when you manage to secure a safe perimeter, venturing outside to get any more resources or loot more lava lakes remains just as dangerous as on day one.

Read more...

RiveScript.com Makeover

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Sunday, January 25 2015 @ 09:11:03 PM

I've just spent pretty much the whole day redoing the website for RiveScript.com, and I think it looks pretty nice.

Screenshot

RiveScript.com was the final website on my server that was still running on my legacy PerlSiikir CMS, and it's been on my to-do list for a while to get it migrated over to my new Python CMS, Rophako like what Kirsle.net is currently running on. The old Perl code was clunky and ugly and memory-leaky, and now I'll be at ease if I ever need to migrate to a new web server, as my Python web apps are extremely quick to get up-and-running, whereas it was an hours-long ordeal to get PerlSiikir to run.

So, the bulk of the work actually needed for RiveScript.com was purely front-end. I revamped the whole web design to use Twitter Bootstrap and make it look all hip and edgy like how all the other small software project websites are these days.

Besides the programming language on the back-end, I had other reasons for why I wanted to simplify RiveScript.com: I don't have the motivation or energy to do as much with that site as I did previously.

I used to run a YaBB Forum on RiveScript.com, but it wasn't extremely active and it was getting hit by too many spam bots, so several months ago I shut that down and linked to the RiveScript forum at Chatbots.org.

More recently I had programmed a chatbot hosting service that was on RiveScript.com, but that wasn't very popular either. I know nobody was using it because it had been broken for months and I hadn't heard any complaints. ;) A couple months ago I sunsetted that feature by turning off new site registrations and removing some references to the feature. And now that's officially gone! If you actually had a bot hosted there, contact me and I can get you your bot's reply files back.

So, the new site is simple and minimalistic and is just about the RiveScript language itself. It was an ordeal rewriting all of the pages from scratch (well, most of them) but now that it's done, the site should be very low-maintenance for me.

One of the most fun parts of it was that I ported over my "Try RiveScript Online" page to use JavaScript and run in the browser, whereas the old version had a Perl back-end (because a JavaScript version of RiveScript didn't exist at the time), so that's even one less thing for me to maintain and make sure it doesn't break. :)

And, the front-end pages for the new site are also open source, FWIW.

Kirsle
Channels
Creativity
Software
Web Tools
Subdomains
Miscellany
Links


Fan Club