The Go codebase is a little rough around the edges and I'll be refactoring it over time. This is the first blog post on the new platform, so let me tell you about my open source Go blog!
I was reading this ACLU blog post about how DreamHost was served with a warrant to hand over IP addresses of some 1.3 million visitors to a website they host, and it got me thinking: do websites really need to store IP addresses of their visitors?
There are a lot of VPN companies such as Private Internet Access that advertise far and wide that they explicitly chose not to keep any logs. The idea is that if the VPN provider is served with a warrant for user activity, they would have no data to hand over, because they never stored anything in the first place. Why don't websites do that?
This is going to be yet another blog post in the "tabs vs. spaces" holy war that software developers like to fight about. I generally prefer tabs over spaces, but for certain types of programming languages I do use spaces instead of tabs.
A very long time ago, I stumbled upon this article "Use Java for Everything". While I disagree that you should use Java for everything (or any programming language, for that matter), the author mentions that he wrote a wrapper script that lets him use Java for shell scripts (ones where you execute the Java source file directly, without the "write, compile, run" steps).
I wanted to do something similar for Go, because I had a very simple Go program I wanted to be able to throw into my .dotfiles repo and run without needing to do too many things first: a simple static HTTP server.
I, along with pretty much every other savvy computer user, never do the "Recommended" installation of software and always go with the "Custom Installation" route, so that I can opt out of installing unnecessary toolbars and other spyware/adware that comes with free Windows software. But does the Average Joe Windows user know that? Definitely not; the Average Joe just clicks through the install dialogs until the program he wants is installed, not knowing that he also just sold his soul to the devil by installing all manner of malicious spyware on his system.
So, I conducted an experiment.
I installed Windows XP on a virtual machine, and installed only a small selection of software that the average user would likely use, and went with all the "Recommended" installation options for every program installed. Altogether, I only installed 9 programs, and most of those were something everybody can say they've installed: instant messengers.
Memory: 256 MB
HDD Space: 10 GB
I installed a fresh copy of Windows XP, installed the VirtualBox guest additions, and used this as the baseline for a "vanilla" Windows XP installation -- a fresh, clean, pure instance of Windows with nothing really installed on it.
In our fresh vanilla Windows XP install, we see the default desktop, the start menu, the Task Manager with few enough tasks in it that we don't even need a scrollbar, and a default Internet Explorer 6 window with MSN as its homepage.
Then, I started installing some software.
Then I installed Yahoo! Messenger 220.127.116.112 - this installed Yahoo Messenger, put an icon on my desktop, installed the Yahoo! Toolbar, and set my homepage and search engine to Yahoo.
Then, Windows Live Messenger 2009 (Build 14.0.8089.726) - this one didn't install a desktop icon, but it set my homepage in IE back to MSN.com and changed my search engine back to Bing.
These are the three most common instant messengers that most people use. So, I went and installed other essential software:
Sun Java Runtime Environment, JRE 6 version 15. Java also took the liberty of installing the Bing Toolbar in my Internet Explorer.
Then I downloaded WinZip 12.1 Free Edition. Windows XP comes with built-in support for zip files, but Average Joe is bound to come across archives of other types and will be told to get WinZip. WinZip installed for me the Google Toolbar in Internet Explorer.
Then, the Adobe Flash Player 10.0.32.18 - this is, so far, the only piece of software that installs what it says and nothing more. It's also the only thing I've installed in my experiment that installed only what I wanted it to.
Finally, I got a couple extra instant messengers installed: Skype 4.1 and ICQ 6.5 - Skype installed the Google Chrome web browser and ICQ installed the ICQ Toolbar and set my homepage and search engine to ICQ.
At this point, I have only installed 8 programs; 8 programs that Average Joe End User is likely to install. Using the default options on all the installers, my system is now fscked up already. But why stop there? Average Joe also needs an antivirus suite, with all this scare going around about viruses.
So, Average Joe installs AVG Free because Average Joe is a cheapass who can't afford Norton or McAfee. AVG may be well-intentioned, but that didn't stop it from installing the AVG Toolbar "Powered by Yahoo!" into my Internet Explorer as well as changing my search engine to AVG Search.
So, what's the damage? 9 programs, and this is what my system looks like:
My Task Manager list has grown exponentially; I have to resize it vertically as tall as it will go, and even then there's still a scrollbar. And do you see the IE window in all that mess? It's completely being murdered under the weight of the 7 different toolbars taking up HALF of the vertical screen real estate.
This is only 9 programs being installed. For a quick list, here they are again:
This, THIS is why Windows sucks. All Windows software installs all this crapware along with it, and all this crapware competes with each other (just look how many times my search engine had been changed).
This is the list of toolbars in IE, from top to bottom, which take up 50% of my 1024x768 vertical resolution:
19 cookies in Internet Explorer. Cookies!!!
The only thing AdAware found were cookies left by ad banners. No adware? No spyware? Are you kidding me!?
So, how do the startup programs look? Well, I'll tell you that rebooting this virtual machine is miserable. With all these programs starting up when the desktop loads, nothing productive can be done for a full 10 minutes. Here's the breakdown:
After this, the startup items were:
It should be noted here that free, open source software, almost never comes with crap like this. If you stick to fine programs like Firefox and Pidgin you can install them without worrying about what other crap they'll bring along with them.
I hate Windows.
You can use the new tool here. As with all the other tools, your converted files are cleared off the server after 24 hours, so don't think about hotlinking your embeddable fonts!
Unix-like systems do have small amounts of malware out there, but they're more commonly called "rootkits" and they tend to take the form of backdoors and trojans left behind after a hacker has already taken control of your system remotely. Thus they affect server systems more than client workstations. For instance if a server allows root login over SSH, and the root password is weak, a hacker could get into the server and once there installs some rootkits to guarantee further access in the future, even if the sysadmin changes the root password.
For desktop users, the following are commonly cited as to why we're generally safe from viruses:
So for a user to get a virus via e-mail, they'd need to save the attachment to disk, then open its properties and change its permissions to be executable, and then double-click the file to run it (or, if they like the terminal, they'd need to
cd to where they saved it,
chmod it, and then execute it).
All of this eliminates the issue of accidentally executing e-mail attachments. If a user has to go through this much hassle to run a virus, they're more likely to think about it for a second and wonder how good of an idea it is.
Unix-like systems (including Mac) don't do this, and the user that you log on as for your day-to-day use doesn't have permission to do very much. You can download and modify things in your home directory and that's just about it. So, any programs you run are also stuck with these limited privileges. If you download an email attachment, give it executable permissions, and execute it, it's not gonna be allowed to do very much that you yourself aren't allowed.
Can it potentially get your saved passwords out of Firefox? Yes. So I wouldn't recommend trying to run things that are likely to be malicious. But can it affect your system as a whole? Can it get into other users' accounts and get their passwords? Can it infect your boot sector? No, no, no. They need root (administrative) privileges to do any such thing. If a normal user does run a malicious program, it's their own problem. Not like on Windows where it becomes everybody's problem because the system itself has become infected.
On Linux systems the user passwords are typically kept in the file
/etc/shadow, and are encrypted using a one-way hashing algorithm. If a hacker has a hashed password, it makes it easier for them to crack it, because it takes out the element of having to go through another system to do so (for instance, brute force login attempts can be handled by the server locking out the account after enough failed attempts). If the hacker has the hash, they can do their own cracking "offline" and only bug the server again once they know the password for sure.
/etc/shadow is owned and read-only for the root user. So, the regular limited user account that's executing a malicious program doesn't have permission to even read this file, so the program can't even get the hashed passwords out of it.
So to do anything administrative, a password is needed (either the user's password or, more commonly, the root password), and the malicious program couldn't possibly know what those passwords are, and if it were to try guessing, any decently configured system would start to get suspicious of it.
Thus it's highly difficult for a user-executed program to gain root privileges. Sometimes they're able to do it, but they usually need to think way outside the box and exploit security holes in running services to do so. But it's a major hamper in their ability to do any harm.
I'm first going to talk about package management systems in Linux. Most mainstream distributions (Fedora, Ubuntu, Mandriva, etc.) have package management systems that control installed software. The distribution's vendor maintains a default repository of available software. The majority of software a user would ever want is usually available in these repositories, from Firefox to OpenOffice all the way to development libraries like GTK+ and GStreamer.
This eliminates the user's need to surf the web and bounce from site to site downloading installers for everything. Most things are available in the software repository, and better yet, they're all cryptographically signed by the vendor, so you can be reasonably sure you're installing trusted, safe software.
But, not all Linux software is available in the repositories. For instance, Sun's VirtualBox. To get VirtualBox you go to its website and download an RPM or Debian package file and install it. To install it, you enter a password (yours, or root's). Then, at least on Redhat-based systems, RPM will complain that the package has not been cryptographically signed using a trusted key, and asks for a second password to be entered to verify that you seriously want to install this.
And this is the point I'm getting at: most Linux software that isn't directly located in one of your trusted software repositories, frankly, can't be trusted. Recent Redhat-based systems give you a second prompt if you attempt to install untrusted software.
So how can Linux viruses be downloaded? If the end user is apathetic and just types in their passwords whenever asked. They could download a package from some random website that appears legit, give their root password to install it, and at that point the package installer has administrative privileges to install that package however it wants.
The package could, for instance, install a binary somewhere, owned as root, and with permissions set in a way that, when executed, it runs with root privileges automatically, regardless of what user executed the binary. And in this way, if it were a virus, it would already have root access to the system, and could do whatever it wanted.
A malicious hacker could take an RPM package such as VirtualBox, replace the main binary with a "wrapper" program (which could launch a second "virus" program and then launch the legitimate VirtualBox binary), repackage it as a new RPM, and post it on a website promoting VirtualBox, saying the download is provided as a convenience to its users so that they don't need to go and download VirtualBox themselves. And since such a wrapper program would launch the legitimate VirtualBox app, most of its users would never know anything was amiss.
So long story short, computers are only as secure as their users are.
P.S. this could also happen to Mac OS X, but it requires less explanation; Mac doesn't have a central software repository full of cryptographically signed packages; they buy or download software the same as Windows users. But they still need a password for installation, so everything after that point still applies. Mac is still a Unix-like operating system.
(On that note, I'm working on researching stuff for a long article I wanna write concerning the sad state of Windows software and the philosophy behind it).
This is one of many cases where after getting into Linux and the open source world, I discovered some free/open source software that does things that I've always wanted to do. In this case, I discovered TiMidity, a MIDI to WAV converter.
TiMidity is used in Linux for support for the MIDI audio format. Rather than have actual hardware drivers to deal with MIDI directly (like Windows does), TiMidity just converts it into WAV format on-the-fly and sends it straight off to your audio hardware. This is its default behavior, anyway. Last night I was digging through its manpages and found out how to save the output as a WAV file instead of sending it directly to the speakers.
Thus, I finally was able to convert MIDI audio to WAV. For reference here's how to do it:
$ timidity -Ow -o output.wav input.mid
WAV files are big and bulky though, so that's where LAME comes in handy. Instead of saving the output to a file, we can pipe it into LAME and save it as an MP3 on the other side.
Thus, here's a one-liner for converting any MIDI file to an MP3:
$ timidity -Ow -o - | lame -
There are Windows ports of these programs available too.
For now you can download it, or browse the source code and Javadocs, at /projects/Java/.
I started with Sun's tutorials, beginning with basic command-line apps to get the syntax down, then moving into the GUI tutorials with Swing. I'm not putting high priority on learning Java applet programming right at the moment, because nobody likes Java applets anymore.
And so now I'm at the point where I'm attempting to program my own things from scratch. A logical place to start was to create a Java class for the CyanChat protocol. The goal of it is to match the functionality of Net::CyanChat, and then one day I might even program a "Java CyanChat Client", to complement my current Perl CyanChat Client (and by Java CC Client, I don't mean an applet; the standard CyanChat client is an applet -- I mean a GUI application).
My CyanChat package is named
org.kirsle.network.CyanChat for right now. Eventually I intend to program a RiveScript interpreter in Java, to open the door up to Java developers to get into the world of RiveScript (and because the only RiveScript interpreter currently in existence is written in Perl). Then, one of my goals in C++ is to compile a "RiveScript.dll" file, which can be dynamically linked with C/C++ programs or any other language that can dynamically link a DLL. :)
Since I'm serious about Java development, I made a nice lil avatar for Java-related blog posts, and spent more than 5 minutes creating it too.