Welcome!

Welcome to Kirsle.net! This is the personal website of Noah Petherbridge, and it's where my various software projects and web blog lives.

Internet Protocol version 6

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Monday, July 20 2015 @ 01:53:26 PM

Last weekend I was pleasantly surprised to discover that Time Warner Cable already supports IPv6 at my apartment.

They shipped me a newer cable modem/WiFi router combo device earlier this year as part of their plan to upgrade everyone's Internet speeds in Los Angeles. I didn't realize that this modem also supported IPv6, and it wasn't enabled by default.

I was playing Splatoon on my WiiU and it was having Internet connectivity problems, making multiplayer impossible. I had this issue before with Smash Bros. 4, and I couldn't figure out a work-around short of putting my WiiU into the DMZ of my router (which I generally didn't like the idea of). This meant ALL inbound connections from the Internet would bypass the router and go straight to the WiiU. It did solve the multiplayer problem, but I didn't like it and it also meant I couldn't port-forward anything to my desktop PC.

Since Splatoon basically requires online access (the single player mode is pretty limited, and you can't access any of the shops without leveling up, and you can only level up by playing online...) I was more motivated to figure it out. (Long story short, I had to disable UPnP support -- the opposite of what you'd expect; normally enabling UPnP is the fix. Weird)

I stumbled upon some IPv6 related options though, like to enable the DHCP server for IPv6. It was disabled by default so I turned it on to see what would happen. My PC was then given a publicly routable IPv6 address from my router. :) I went to a Test your IPv6 site and confirmed that all of a sudden, my Internet is IPv6 ready!

Also, I went to the What Is My IP site and instead of saying my usual IPv4 address it told me my IP was a big long string of numbers.

I was then curious about inbound routing to my IP, because one of the big features of IPv6 is that NAT is no longer required, and that every device is able to have its own publicly routeable address. I didn't have immediate access to any off-site IPv6 devices though to test it which made things more difficult.

But then I found out I could enable IPv6 on my DigitalOcean VPS. I'd heard of them rolling out IPv6 support a while back, but I thought it was limited to certain data centers or available only to newly created VPS's. It turns out I just had to click a big "Enable" button and then configure the addresses on my server (to set them to the addresses that DigitalOcean says belong to me).

So... Kirsle.net now has a DNS AAAA record pointing to my IPv6 address on my VPS.

kirsle.net.    819    IN    A       192.241.206.46
kirsle.net.    1799   IN    AAAA    2604:a880:1:20::46:5001

Inbound Routing

Inbound routing for my personal PC still wasn't working, though. I set up a basic nginx server and tried hitting it from my web server but it didn't work. I could ping my home IPv6 address, but couldn't do much else.

I booted up my laptop and confirmed that it got its own IPv6 address distinct from my desktop PC, and that my laptop could connect to my desktop over its IPv6 address, so that ruled out a software firewall as being the problem.

I logged into my router again to poke around (the router is an ARRIS DOCSIS 3.0 Touchstone Residential Gateway, model DG1670A). I tried turning off its firewall. This worked sort of -- IPv6 addresses were now inbound routable, but IPv4 port forwarding no longer worked. Not an option for me. I turned the firewall back on and poked around some more.

There was a section called "Client IPv6 Filters" and the description on its page said "The Router can be configured to restrict access to the Internet, e-mail or other network services."

This sounded like it was designed for blocking outbound connections, like if I wanted to blacklist a website or something from being connected to from my network. But I clicked the Add button and saw this modal:

Add Client IP Filter screenshot

The Action/Direction options were "Allow+Incoming" or "Deny+Outgoing". I left it on the Allow option, copy/pasted my desktop's IPv6 address into both address boxes and clicked the confirm button. Now my web server was able to curl my desktop PC and get the nginx default web page. Success!

nginx on IPv6

When getting the IPv6 support set up in nginx on my web server I ran into some interesting problems.

My server hosts a ton of different sites, and normally every single server directive would have its own listen 80; indicating that the server is a candidate for handling requests to IPv4 port 80 on the server.

This doesn't work the same on IPv6.

If I had more than one server set to listen [::]:80 ipv6only=true (not sure if that last part was necessary), nginx would complain about it and fail to start.

I think what needs to happen is that if I want IPv6 support enabled for multiple domains at the same time, I have to individually assign an IPv6 address to each one. With IPv6, you basically have free reign over 64 bits of your address space to assign however you want. For example, DigitalOcean says my IPv6 prefix is 2604:a880:1:20 and the next 64 bits after that are all mine. On my home network with Time Warner Cable, I'm similarly given a fixed 64-bit prefix and the rest can be assigned by me (either via DHCP or statically).

So it's probably possible to tack on a whole bunch of static IPs of my choosing onto the ethernet device of my server, and then bind to those specific addresses in nginx, one for each site. I'll cross that bridge when I get to it.

Fedora 21 on the 2015 Macbook Air

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Saturday, May 02 2015 @ 01:06:37 PM

Today I picked up a Macbook Air (13", early 2015 model) because I wanted a new laptop, as my old laptop (the Samsung Series 5) has a horrible battery life, where it barely lasts over an hour and gives up early (powering down at 40% and not coming back up until I plug it in). This is also my first Apple computer. I'm the furthest thing from an Apple fanboy, but the choices I was throwing around in my head were between an Apple computer and a Lenovo Thinkpad.

I was given a Thinkpad as my work laptop, and it's by far the most impressive PC laptop I've ever used; it can drive three displays and run lots of concurrent tasks and has an insane battery life. Every PC laptop I've owned in the past have sucked in comparison. I hear people compare Apple computers to Thinkpads, so that's why the choice came down to one of these, and I didn't want another Thinkpad sitting around the house. ;)

Months before getting a Macbook I was looking into what kind of effort it takes to install Linux on a Macbook. There's a lot of information out there, and most of it suggests that the best way to go is to install a boot manager like rEFIt (or rEFInd, since rEFIt isn't maintained anymore). I saw some pages about not using rEFIt and installing Grub directly, which were from a Debian and Arch Linux perspective, and it sounded really complicated.

It seems that nowadays, with a user friendly Linux distribution like Fedora, a lot of this works much more flawlessly than the dozens of tutorials online would suggest. I just made a Fedora LiveUSB in the usual way (as if installing on a normal PC), rebooted the Macbook while holding the Option key, so that I was able to select the USB to boot from.

When installing Fedora to disk, the process was very much the same as doing it on a normal PC. I let Fedora automatically create the partition layout, and it created partitions and mount points for /, /boot and /home like usual, but it also created a partition and mount point for /boot/efi (for installing itself as the default bootloader in the EFI firmware on the Macbook). After installation was completed, I rebooted and the grub boot screen comes up immediately, with options to boot into Fedora.

One weird thing is, the grub screen apparently sees something related to Mac OS X (there were two entries, like "Mac OS X 32-bit" and "Mac OS X 64-bit", but both options would give error messages when picked).

If I want to boot into OS X, I hold down the Option key on boot and pick the Macintosh HD from the EFI boot menu. Otherwise, if the Macbook boots normally it goes into the grub menu and then Fedora. So, the whole thing is very similar to a typical PC dual-boot setup (with Windows and Linux), just with one extra step to get into OS X.

Update: I'm keeping a wiki page with miscellaneous setup notes and tips here: Fedora on Macbook

RiveScript Rewritten in CoffeeScript

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Wednesday, April 22 2015 @ 12:47:51 PM

I've spent the last few months porting over the JavaScript version of my chatbot scripting language, RiveScript, into CoffeeScript. I mainly did this because I didn't like to maintain raw JavaScript code (especially after I ran a linter on it and had to rearrange my variable declarations because JavaScript's variable scoping rules are stupid), but I also took the opportunity to restructure the code and make it more maintainable in the future.

For some historical background, the first RiveScript implementation was written in Perl, and I designed it to be one big monolithic file (RiveScript.pm), because then I could tell noobs that they can update to a new version of RiveScript in their bots by just dropping in a new file to replace the old one, and avoid overwhelming them with the complexity of doing a CPAN installation (especially if they're generally new to Perl as well, and all they really wanna do is just run a chatbot.)

Source File Restructuring

The Python and JavaScript ports are more-or-less direct ports of the Perl version: I literally read the Perl source from top to bottom and translated it into each respective language. So, there was rivescript.js which was a huge 2,900 line long wall of JavaScript and one of my goals in the refactor was to spread logic out into multiple files, so if I have a bug to fix in the reply matching code, I don't have to scroll through pages and pages of loading/parsing code to get to the relevant part.

The new file structure is like this:

  • rivescript.js - The user-facing API, has all the same public functions as the old one
  • parser.js - A self-contained module that loads RiveScript code into an "abstract syntax tree" (a JSON serializable blob that represents ALL of the parsed RiveScript code in a program-friendly format).
  • sorting.js - The rat's nest that is the implementation behind sortReplies() is contained here.
  • inheritance.js - Functions related to topic inheritance/includes are here.
  • brain.js - The code that actually gets a reply is here.
  • utils.js - All those miscellaneous internal utility functions, like quotemeta() and such.
  • lang/javascript.js - The implementation for JavaScript object macros in your RiveScript code.
  • lang/coffee.js - This is new - you can use CoffeeScript in your object macros (it's not enabled by default, you'll probably have to snipe lang/coffee.js and include it in your own project for now, but the built-in shell.coffee uses it out-of-the-box).

Logic Refactoring

Another thing that was messy in the Perl, Python and JS versions of RiveScript was how it lays out its internal data structures in memory. If you'd ever run the equivalent of Data::Dumper you'd see reply and trigger data appearing in multiple places depending on whether it had a %Previous tag on it or not.

It looked something like this:

$RiveScript = {
    "topics" => {
        # Most reply data is under here, BUT NOT triggers with %Previous!
        "random" => { # (topic names)
            "hello bot" => { # (trigger texts)
                "reply" => { # (replies to the trigger)
                    0 => "Hello, human!",
                    1 => "Hi there!",
                },
                "condition" => { # (conditions)
                    0 => "<get name> != undefined => Hello, <get name>!",
                },
                "redirect" => undef, # (if there's an @redirect)
            },
            # ...
        },
    },
    "thats" => {
        # This is like 'topics' but ONLY for triggers with %Previous
        "random" => { # (topic names like before)
            "who is there" => { # (the %Previous text)
                "*" => { # (trigger text)
                    # then things were like the above
                    "reply" => { 0 => "<sentence> who?" },
                    "condition" => {},
                    "redirect" => undef,
                }
            }
        }
    },

    # And then sorting! Again the 'normal' replies are completely
    # segregated from those with %Previous
    "sorted" => {
        "random" => [ # (topic names)
            "hello bot", # (triggers, sorted)
            "*",
        ],
    },
    "sortsthat" => {
        # This one is simply a trigger list for ones that have %Previous
        "random" => [ # (topic names)
            "*",
        ]
    }
    "sortedthat" => {
        # This one is, in case one %Previous has more than one answer,
        # if we ONLY had the above sort we'd overwrite the first answer
        # with the second.. this one keeps track of all "replies" with
        # the same %Previous
        "random" => {
            "who is there" => [
                "*",
            ]
        }
    },
}

As you can see, it was a mess. There were a ton of different places where reply data was kept, and triggers had to be sorted many different ways for various edge cases. Additionally, keeping track of which topics inherit or include others was kept in a separate data structure from the topics themselves!

In the new refactor of RiveScript-js I eliminated as much duplication as possible. Now, the entirety of the reply base exists under the topics key, and instead of using lots of nested dictionaries, e.g. ->{topic}->{trigger}->{reply} which had the issue of one trigger overwriting the data for another if they both happened to have the same text (e.g. when you have two answers to the same %previous question), the ordering of the triggers is preserved. The data structure ends up looking like this:

{
    "topics": { // main reply data
      "random": { // (topic name)
        "includes": {}, // included topics
        "inherits": {}, // inherited topics
        "triggers": [ // array of triggers
          {
            "trigger": "hello bot",
            "reply": [], // array of replies
            "condition": [], // array of conditions
            "redirect": "",  // @ redirect command
            "previous": null, // % previous command
          },
          ...
        ]
      }
    }
}

Additionally, whenever the code refers to reply data (for example, in the sort buffers and the %previous tree), it refers to a specific index in the singular topic structure for the reply data. This has the other side benefit that, while getting a reply for the user, when a matching trigger is found it already has the pointer to that trigger's responses right away. It doesn't have to look it up from the central topic structure like before (and since the replies are kept on an array, this would be impractical now anyway).

Keeping the replies on an array also naturally takes care of the issue with multiple responses to the same %previous without needing a third sort buffer (we still do need a separate sort buffer for %previous replies themselves, though).

Other Implementations

The Perl and Python versions probably won't get updated anytime too soon to fit this new structure, but any future ports of RiveScript to other languages that I work on will be based off this new CoffeeScript version. I have some vague plans right now to port RiveScript over to Google's Go language, and I wanted to get the refactor out of the way first so I have a new "template" to reference when writing a new port.

So, Why CoffeeScript?

To elaborate more on why I rewrote it in CoffeeScript instead of just doing this restructure in Node-style JavaScript:

  • A while back I started JS-linting the JavaScript codebase and fixing all of the mistakes it pointed out.
  • JavaScript has really weird variable scoping issues, where all variables declared within a function, no matter where it's declared, gets implicitly "hoisted" to the top of the function as though you had declared it up there instead. Sane programming languages have lexical scoping where variables declared inside of a { block } of code (including loop variables, i.e. for (my $i = 0; ...)) die with that block's closing brace, for example Perl does it this way.
  • In numerous places in my code, I would have a long function that does a loop over "something" in more than one place, always doing for (var i = 0; ...). Declaring var i multiple times in the same function is the error, so I had to take all the reused variable names and make this ugly, var i, iend, j, jend, match; at the tops of functions.
  • CoffeeScript has a clean syntax compared to JS (it looks quite like Python), and it automagically handles your variable declarations. You simply use a variable (like Python), and CoffeeScript handles when and where to declare it in the JavaScript output.

As for all the haters that say things like, "ECMAScript 6 makes CoffeeScript obsolete because it adds classes and the arrow operator and everything else into the core JavaScript language": I see no reason at all that CoffeeScript can't one day compile into ECMAScript 6 the way that it does into ES5 right now, so it's not like CoffeeScript is a dying language that will no longer be maintained in the future. In a worst case scenario, I could program a CoffeeScript to ES6 compiler myself if nobody else will. It's not as if I have no experience writing scripting language parsers. ;)

And with GitHub backing it (they use CoffeeScript to program my favorite text editor, Atom), the language isn't going anywhere anytime soon.

Linux Desktop Remote Code Execution via File URI

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, March 27 2015 @ 02:04:08 PM
I've discovered a sort of "remote code execution" vulnerability that affects all Linux desktops, particularly Fedora and Ubuntu but most likely all desktop Linux distributions could be affected, except for maybe Arch or Gentoo with extremely customized installations.

First and foremost: this requires the victim to click not one, but two random links sent to them over Pidgin (or any other program that does URL auto-linking the way Pidgin does). So it's not exactly the most severe vulnerability, but I found it interesting nonetheless.

Read more...

Minecraft Map: Swampcore

Noah Petherbridge
kirsle
Posted by Noah Petherbridge on Friday, March 06 2015 @ 09:12:00 AM

I've created a downloadable Minecraft map that implements "Swampcore" (older blog post about Swampcore). It's a superflat swamp biome preset with a 24/7 thunderstorm, making for an extremely hostile map where you have to scratch and claw your way into having so much as a simple dirt shack to call "home".

It runs on vanilla Minecraft (no mods or anything needed) and was created on Minecraft v1.8.3, but will probably work on later versions too.

Superflat Preset

This is the preset code that was used to generate the map:

3;9*minecraft:bedrock,minecraft:dirt,minecraft:grass;6;biome_1,decoration,lake,lava_lake

From the bottom up, you have:

  • 9 layers of bedrock
  • 1 layer of dirt
  • 1 layer of grass

World Features

The superflat swamp includes lakes and lava lakes, oak trees, vines, sugar canes, mushrooms and tall grass. The water lakes can sometimes spawn with sand or clay, and the lava lakes spawn with stone around them. The stone can occasionally be an ore block instead (such as iron or redstone). Diamonds are able to spawn around the lava lakes as well (world surface is at Y=11) but are extraordinarily rare.

There is a 24/7 thunderstorm which keeps the sky dark and prevents zombies and skeletons from burning up during the day, and even allows for hostile mobs to spawn during the day. Players will frequently get killed while just trying to build a cheap dirt shack to live in.

The only stone on the map is around lava lakes, so players will need to loot a lava lake in order to build a furnace or stone tools. The main source of coal will be from burning wood logs in a furnace, as coal ore will be extremely rare. Zombies can rarely drop iron ingots when killed, or players can try to find iron ore around lava lakes.

The nether portal can be built and activated, but since diamonds are practically nonexistent players would have to build a mould for a portal and pour lava and water in using buckets to slowly build the frame. Since gravel is extremely hard to find (impossible?), no Flint & Steel, so players would have to use lava with wood planks to set a fire near the portal and wait for it to spread into the frame and activate it (pretty tedious).

With the presence of sugar canes and access to the nether, splash potions of weakness and golden apples can be crafted and a zombie villager could be healed, so players can build their own NPC village. With villagers players could trade for diamond gear and lots of other items that are otherwise impossible to get on this world.

In short, it's entirely possible to get to the late game on this world (the main thing missing is access to the End Dimension, since no strongholds exist, but that's probably for the better in a multiplayer server anyway). It's just very, very difficult. Even when you manage to secure a safe perimeter, venturing outside to get any more resources or loot more lava lakes remains just as dangerous as on day one.

Read more...

Kirsle
Channels
Creativity
Software
Web Tools
Subdomains
Miscellany
Links


Fan Club