Category: Software

Use Go as a Shell Scripting Language

Noah Petherbridge
Posted by Noah Petherbridge on Tuesday, November 29 2016 @ 01:43:14 PM

A very long time ago, I stumbled upon this article "Use Java for Everything". While I disagree that you should use Java for everything (or any programming language, for that matter), the author mentions that he wrote a wrapper script that lets him use Java for shell scripts (ones where you execute the Java source file directly, without the "write, compile, run" steps).

I wanted to do something similar for Go, because I had a very simple Go program I wanted to be able to throw into my .dotfiles repo and run without needing to do too many things first: a simple static HTTP server.


A "Dumb End User" Experiment

Noah Petherbridge
Posted by Noah Petherbridge on Friday, August 28 2009 @ 12:44:17 PM
Whilst using Windows operating systems and installing the software I tend to use, I've noticed that a good portion of Windows software comes bundled with crapware. During the installation of things such as AOL Instant Messenger, if you don't watch out, you'll also install the AIM Toolbar, set your Internet Explorer homepage to AIM's site, and set your default search engine to AIM's.

I, along with pretty much every other savvy computer user, never do the "Recommended" installation of software and always go with the "Custom Installation" route, so that I can opt out of installing unnecessary toolbars and other spyware/adware that comes with free Windows software. But does the Average Joe Windows user know that? Definitely not; the Average Joe just clicks through the install dialogs until the program he wants is installed, not knowing that he also just sold his soul to the devil by installing all manner of malicious spyware on his system.

So, I conducted an experiment.

I installed Windows XP on a virtual machine, and installed only a small selection of software that the average user would likely use, and went with all the "Recommended" installation options for every program installed. Altogether, I only installed 9 programs, and most of those were something everybody can say they've installed: instant messengers.

The Experiment

I used Sun VirtualBox as the virtual machine platform and installed Windows XP, Service Pack 2, on it. The specs of the virtual machine are as follows:

Memory: 256 MB
HDD Space: 10 GB

I installed a fresh copy of Windows XP, installed the VirtualBox guest additions, and used this as the baseline for a "vanilla" Windows XP installation -- a fresh, clean, pure instance of Windows with nothing really installed on it.

Vanilla Instance
[click for larger screenshot]

In our fresh vanilla Windows XP install, we see the default desktop, the start menu, the Task Manager with few enough tasks in it that we don't even need a scrollbar, and a default Internet Explorer 6 window with MSN as its homepage.

Then, I started installing some software.

Installing Software

I started with AOL Instant Messenger This installed AIM like I wanted, put an icon on my desktop, and also the AIM Toolbar for IE, and it also set my homepage and search engine to AIM.

Then I installed Yahoo! Messenger - this installed Yahoo Messenger, put an icon on my desktop, installed the Yahoo! Toolbar, and set my homepage and search engine to Yahoo.

Then, Windows Live Messenger 2009 (Build 14.0.8089.726) - this one didn't install a desktop icon, but it set my homepage in IE back to and changed my search engine back to Bing.

These are the three most common instant messengers that most people use. So, I went and installed other essential software:

Sun Java Runtime Environment, JRE 6 version 15. Java also took the liberty of installing the Bing Toolbar in my Internet Explorer.

Then I downloaded WinZip 12.1 Free Edition. Windows XP comes with built-in support for zip files, but Average Joe is bound to come across archives of other types and will be told to get WinZip. WinZip installed for me the Google Toolbar in Internet Explorer.

Then, the Adobe Flash Player - this is, so far, the only piece of software that installs what it says and nothing more. It's also the only thing I've installed in my experiment that installed only what I wanted it to.

Finally, I got a couple extra instant messengers installed: Skype 4.1 and ICQ 6.5 - Skype installed the Google Chrome web browser and ICQ installed the ICQ Toolbar and set my homepage and search engine to ICQ.

At this point, I have only installed 8 programs; 8 programs that Average Joe End User is likely to install. Using the default options on all the installers, my system is now fscked up already. But why stop there? Average Joe also needs an antivirus suite, with all this scare going around about viruses.

So, Average Joe installs AVG Free because Average Joe is a cheapass who can't afford Norton or McAfee. AVG may be well-intentioned, but that didn't stop it from installing the AVG Toolbar "Powered by Yahoo!" into my Internet Explorer as well as changing my search engine to AVG Search.

So, what's the damage? 9 programs, and this is what my system looks like:

After 9 Commonly Used Programs
[click for larger screenshot]

My Task Manager list has grown exponentially; I have to resize it vertically as tall as it will go, and even then there's still a scrollbar. And do you see the IE window in all that mess? It's completely being murdered under the weight of the 7 different toolbars taking up HALF of the vertical screen real estate.

This is only 9 programs being installed. For a quick list, here they are again:

  1. AOL Instant Messenger
  2. Yahoo! Messenger
  3. Windows Live Messenger
  4. Sun Java
  5. WinZip
  6. Skype
  7. Adobe Flash
  8. ICQ
  9. AVG Free
And this is the damage. Let's have a better look at that Internet Explorer window:

Internet Explorer Toolbar Hell!
[click for larger screenshot]

This, THIS is why Windows sucks. All Windows software installs all this crapware along with it, and all this crapware competes with each other (just look how many times my search engine had been changed).

This is the list of toolbars in IE, from top to bottom, which take up 50% of my 1024x768 vertical resolution:

  1. (IE Menu Bar)
  2. (IE Default Toolbar)
  3. (IE Address Bar & Links)
  4. AIM Toolbar
  5. Yahoo Toolbar
  6. Tabbed Browsing Toolbar
  7. Google Toolbar
  8. Bing Toolbar
  9. ICQ Toolbar
  10. AVG Toolbar
Perhaps more saddening is this: you may have noticed in the final screenshot that I'd installed LavaSoft AdAware - I was intending to scan my system with it and see how much spyware and adware it detected. Know what it found?

19 cookies in Internet Explorer. Cookies!!!

The only thing AdAware found were cookies left by ad banners. No adware? No spyware? Are you kidding me!?

So, how do the startup programs look? Well, I'll tell you that rebooting this virtual machine is miserable. With all these programs starting up when the desktop loads, nothing productive can be done for a full 10 minutes. Here's the breakdown:

Startup Items

There are three categories of startup items: current user items in the registry, system-wide items in the registry, and items in the user's Start Menu. At the beginning, the only startup item was VBoxTray, which is part of the VirtualBox Guest Additions, and it was in the system-wide registry list.

After this, the startup items were:

  • Current User Registry Items
    • AOL Instant Messenger
    • Windows Live Messenger
    • Yahoo Messenger
    • ICQ
    • Skype
    • Google Toolbar Notifier
  • System-wide Registry Items
    • AVG Tray
    • Microsoft Default Manager
    • Sun Java Update Scheduler
    • VBoxTray (VirtualBox Guest Additions)
  • Start Menu Items
    • WinZip Tray
This is absolutely ridiculous.

It should be noted here that free, open source software, almost never comes with crap like this. If you stick to fine programs like Firefox and Pidgin you can install them without worrying about what other crap they'll bring along with them.

I hate Windows.

Online TTF to EOT Font Converter

Noah Petherbridge
Posted by Noah Petherbridge on Wednesday, August 05 2009 @ 11:39:23 AM
After posting my initial blog post about embedding fonts in a way that works with Internet Explorer and Firefox 3.5, a reader has informed me that he had some trouble running the ttf2eot program on Windows XP.

So I've created a web-based converter tool to turn TrueType TTF fonts into OpenType EOT fonts, to go along with my other tools that turn images into favicons and turn images into XBM masks.

ttf2eot preview

You can use the new tool here. As with all the other tools, your converted files are cleared off the server after 24 hours, so don't think about hotlinking your embeddable fonts!

Convert MIDI Music to MP3

Noah Petherbridge
Posted by Noah Petherbridge on Wednesday, January 21 2009 @ 11:49:29 PM
Numerous years ago I was trying to find a way of converting MIDI files into WAV or MP3. I tried googling for "midi to wav" but found nothing useful. After going through 10 or 20 pages of search results I gave up. All there was out there were lame commercial products that were way more expensive than they're worth.

(On that note, I'm working on researching stuff for a long article I wanna write concerning the sad state of Windows software and the philosophy behind it).

This is one of many cases where after getting into Linux and the open source world, I discovered some free/open source software that does things that I've always wanted to do. In this case, I discovered TiMidity, a MIDI to WAV converter.

TiMidity is used in Linux for support for the MIDI audio format. Rather than have actual hardware drivers to deal with MIDI directly (like Windows does), TiMidity just converts it into WAV format on-the-fly and sends it straight off to your audio hardware. This is its default behavior, anyway. Last night I was digging through its manpages and found out how to save the output as a WAV file instead of sending it directly to the speakers.

Thus, I finally was able to convert MIDI audio to WAV. For reference here's how to do it:

$ timidity -Ow -o output.wav input.mid

WAV files are big and bulky though, so that's where LAME comes in handy. Instead of saving the output to a file, we can pipe it into LAME and save it as an MP3 on the other side.

Thus, here's a one-liner for converting any MIDI file to an MP3:

$ timidity -Ow -o - input.mid | lame - output.mp3

There are Windows ports of these programs available too.

How Linux Viruses Could Work

Noah Petherbridge
Posted by Noah Petherbridge on Thursday, January 08 2009 @ 05:18:05 PM
It's generally true that Linux and other unix-like operating systems don't have the problem of viruses and malware like Windows does. Linux and Mac users don't need virus scanners except in odd cases when they act as file servers where Windows clients are likely to receive files from them (a Linux mail server can easily store and transmit e-mail attachments containing viruses, in which case it'd want a virus scanner, only to look out for the Windows users it comes in contact with).

Unix-like systems do have small amounts of malware out there, but they're more commonly called "rootkits" and they tend to take the form of backdoors and trojans left behind after a hacker has already taken control of your system remotely. Thus they affect server systems more than client workstations. For instance if a server allows root login over SSH, and the root password is weak, a hacker could get into the server and once there installs some rootkits to guarantee further access in the future, even if the sysadmin changes the root password.

For desktop users, the following are commonly cited as to why we're generally safe from viruses:

1) Unix requires executable file permissions before files can be executed.

A Windows user reading their e-mail can easily click an attachment and run it immediately. All that Windows needs for a file to be executable is that it has a ".exe", ".scr", ".bat", or a small handful of other file extensions. With Unix, the file would need to be manually given permission to execute.

So for a user to get a virus via e-mail, they'd need to save the attachment to disk, then open its properties and change its permissions to be executable, and then double-click the file to run it (or, if they like the terminal, they'd need to cd to where they saved it, chmod it, and then execute it).

All of this eliminates the issue of accidentally executing e-mail attachments. If a user has to go through this much hassle to run a virus, they're more likely to think about it for a second and wonder how good of an idea it is.

2) Unix generally doesn't give users administrative rights.

With Windows operating systems (particularly Windows XP and older versions), all users of the PC have administrative rights. It means that, if you were so inclined, you could go to the system folders and start deleting stuff, and find your PC might not boot again after that.

Unix-like systems (including Mac) don't do this, and the user that you log on as for your day-to-day use doesn't have permission to do very much. You can download and modify things in your home directory and that's just about it. So, any programs you run are also stuck with these limited privileges. If you download an email attachment, give it executable permissions, and execute it, it's not gonna be allowed to do very much that you yourself aren't allowed.

Can it potentially get your saved passwords out of Firefox? Yes. So I wouldn't recommend trying to run things that are likely to be malicious. But can it affect your system as a whole? Can it get into other users' accounts and get their passwords? Can it infect your boot sector? No, no, no. They need root (administrative) privileges to do any such thing. If a normal user does run a malicious program, it's their own problem. Not like on Windows where it becomes everybody's problem because the system itself has become infected.

3) To do anything administrative, you need to have a password.

Usually you'll need to know the root user's password, as it's the only user with full rights to make system changes. Windows Vista prompts you for permission to do anything, but it doesn't ask for a password, and the prompt it gives you can be automatically accepted by a running program that can simulate mouse clicks on your behalf. So Vista's security model is hardly better than XP's.

On Linux systems the user passwords are typically kept in the file /etc/shadow, and are encrypted using a one-way hashing algorithm. If a hacker has a hashed password, it makes it easier for them to crack it, because it takes out the element of having to go through another system to do so (for instance, brute force login attempts can be handled by the server locking out the account after enough failed attempts). If the hacker has the hash, they can do their own cracking "offline" and only bug the server again once they know the password for sure.

But /etc/shadow is owned and read-only for the root user. So, the regular limited user account that's executing a malicious program doesn't have permission to even read this file, so the program can't even get the hashed passwords out of it.

So to do anything administrative, a password is needed (either the user's password or, more commonly, the root password), and the malicious program couldn't possibly know what those passwords are, and if it were to try guessing, any decently configured system would start to get suspicious of it.

Thus it's highly difficult for a user-executed program to gain root privileges. Sometimes they're able to do it, but they usually need to think way outside the box and exploit security holes in running services to do so. But it's a major hamper in their ability to do any harm.

How Linux viruses could work:

A computer is only as secure as the user who runs it.

I'm first going to talk about package management systems in Linux. Most mainstream distributions (Fedora, Ubuntu, Mandriva, etc.) have package management systems that control installed software. The distribution's vendor maintains a default repository of available software. The majority of software a user would ever want is usually available in these repositories, from Firefox to OpenOffice all the way to development libraries like GTK+ and GStreamer.

This eliminates the user's need to surf the web and bounce from site to site downloading installers for everything. Most things are available in the software repository, and better yet, they're all cryptographically signed by the vendor, so you can be reasonably sure you're installing trusted, safe software.

But, not all Linux software is available in the repositories. For instance, Sun's VirtualBox. To get VirtualBox you go to its website and download an RPM or Debian package file and install it. To install it, you enter a password (yours, or root's). Then, at least on Redhat-based systems, RPM will complain that the package has not been cryptographically signed using a trusted key, and asks for a second password to be entered to verify that you seriously want to install this.

And this is the point I'm getting at: most Linux software that isn't directly located in one of your trusted software repositories, frankly, can't be trusted. Recent Redhat-based systems give you a second prompt if you attempt to install untrusted software.

So how can Linux viruses be downloaded? If the end user is apathetic and just types in their passwords whenever asked. They could download a package from some random website that appears legit, give their root password to install it, and at that point the package installer has administrative privileges to install that package however it wants.

The package could, for instance, install a binary somewhere, owned as root, and with permissions set in a way that, when executed, it runs with root privileges automatically, regardless of what user executed the binary. And in this way, if it were a virus, it would already have root access to the system, and could do whatever it wanted.

A malicious hacker could take an RPM package such as VirtualBox, replace the main binary with a "wrapper" program (which could launch a second "virus" program and then launch the legitimate VirtualBox binary), repackage it as a new RPM, and post it on a website promoting VirtualBox, saying the download is provided as a convenience to its users so that they don't need to go and download VirtualBox themselves. And since such a wrapper program would launch the legitimate VirtualBox app, most of its users would never know anything was amiss.

So long story short, computers are only as secure as their users are.

P.S. this could also happen to Mac OS X, but it requires less explanation; Mac doesn't have a central software repository full of cryptographically signed packages; they buy or download software the same as Windows users. But they still need a password for installation, so everything after that point still applies. Mac is still a Unix-like operating system.

Web Tools

Fan Club